You bring up a excellent point that documents should be living documents and can evolve faster than the version of the code. With a community this small I think it's easiest to bundle the two together since anyone can contribute to anarki.

Maybe Anarki contributor should designate a collaborative location that can serve as the official site for documentation for atleast user guides, tutorials and faq's.

For discussions I think we would all agree that arclanguage.com is the best place.

I like that in concept. We could just have a GitHub Pages branch on Anarki, and GitHub would automatically let us view it as a web page. However, aw mentioned having trouble with GitHub Pages: http://arclanguage.org/item?id=12934

Someone could rig up a website to serve GitHub raw file views as HTML, but I don't know if that's nice to GitHub. :)

Someone could instead have a website that somehow keeps an up-to-date clone of Anarki (perhaps triggering a "git pull" not only as a cron job but also every time a certain page was viewed) and somehow uses that to determine the website content.

One thing to consider is security: If anyone can show their own JS code on this page, they could set tracking cookies or something. If anyone can run Arc code on the server, there's even more to worry about (albeit nothing the racket/sandbox module isn't designed for).

---

"Maybe Anarki contributor should designate a collaborative location that can serve as the official site for documentation for atleast user guides, tutorials and faq's."

With any luck, this is what arclanguagewiki is for. http://sites.google.com/site/arclanguagewiki/

However, having a separate place for documentation is only one part of what I'm suggesting. I'm not sure it's worth it unless the separate parts are somehow integrated again--for instance, by showing docs and discussions as you browse code, or by letting user guide writers say {{doc:anarki:lib/util.arc:afnwith}} or somesuch to include a piece of the API reference.

---

"For discussions I think we would all agree that arclanguage.com is the best place."

Speaking of which, are arclanguage.com and arclanguage.org both legitimate? Both of their WHOIS entries list Paul Graham, but I don't know whether that means anything. I've never logged in anywhere but arclanguage.org, just because it's what most people link to.

However exactly for the reason of implementing our own features at some point such as the cross references you mention I expect that we're going to want to do our own processing. Which suggests that GitHub Pages or the arclanguagewiki on Google Sites might be part of the right long term solution, but only if there's a way to e.g. insert the piece of an API reference... which we're generating.

Here's a thought. What if we had a server which went out and gathered documentation source material from various places such as Anarki. (GitHub has http://help.github.com/post-receive-hooks/ so the server could get notified of new pushes to Anarki instead of having to poll).

---

"The server would work on the text of the sources, such as docstrings found in the Anarki source code."

With this approach, people might be pushing to Anarki way more, sometimes using in-browser file edits on GitHub, and the server would have to scrape more and more things each time. Then again, that would be a good problem to have. :-p

---

"That way even if someone pushed something malicious to Anarki then we wouldn't have a security problem (either on the server or in the reader's browser)."

By the same token, it would be harder for just anyone to update the server, right? Eh, that might be a necessity for security anyway.

I'd just have the server code do that.

For instance, someone might write a DSL in Arc and want to set up a whole subsite covering the DSL's own API.

Sorry, not following you here. How would this be different?

Besides that, it would just be nifty to have the Arc documentation improve as people improved the Arc libraries and vice versa.

  (defstarc word4
    "Demonstrates stuff."
    word1 word2 word3)

So every time I come up with one of these requirements for the documentation, I should submit a patch to the server or something? Fair enough--the code implementing the documentation oughta be documented somewhere too, and keeping it close to the project also makes it more ad hoc and inconsistent--but I think this would present a bit of an obstacle to working on the documentation. I'd rather there be a compromise, where truly ad hoc and experimental things were doable in independent projects and the most useful documentation systems moved to the server code gradually.

- you run a copy of the server code you're working on locally, until you see that your "Stark" documentation is being integrated into the rest of the documentation in the way that you want it to

- you push your changes to the server (say, via github for example) and they go live

OK, but what if you're a completely random person, you've never posted anything to arclanguage.org, no one knows who you are, and you want write access to the server so that you "can do stuff". Alright, fork the repo on github, push your changes there, and send a pull request. Then when you turn out to be someone who isn't trying to install malicious Javascript you are given write access to the server repo yourself. (This is pretty standard approach in open source projects, by the way).

But... what if write access to the server repo ends up being controlled by an evil cabal of conservatives who reject having any of this "Starc" stuff added? Fire up your own server, publish the documentation pages yourself, and people will start using your documentation pages because they are more complete than the old stuff.

My concern with the sandbox idea is that I imagine it's going to be hard to create a sandbox that is both A) powerful enough to be actually useful, and B) sufficiently constrained so that there's no possible way for someone to manage to generate arbitrary Javascript.

I'm finding this discussion very helpful, by the way. What I'm spending my time on now is the "pastebin for examples" site. I've been wondering if this project would stay focused on just the examples part (with the ability for other documentation sites to embed examples from the pastebin site) or if it would expand to be a site for complete documentation itself (the "code site for Arc" idea).

For the pastebin site I've thrown away several designs that weren't working and I've found one that so far does look like it's going to work. But, the catch is that by design it allows the site to execute arbitrary code in the target machine that's running the example. This isn't too terrible by itself (you can always run the example in a virtual machine or on an Amazon EC2 instance etc. instead of on your own personal computer if you want), but it does mean that the "pastebin for examples" site is going to need a higher level of security than an Arc documentation site.

The main thing I'm afraid of is the documentation site becoming stagnant. Too often, someone finds the arclanguage.org website and asks "How do I get version 372 of MzScheme?" Too often, someone who's been reading arcfn.com/doc the whole time finally looks at the Arc source and starts a forum thread to say "Look at all these unappreciated functions!" ^_^

I don't blame pg or kens; I blame the fact that they don't have all the time in the world to do everything they want. I'm in the same position, and I bet it's pretty universal.

---

"Fire up your own server, publish the documentation pages yourself, and people will start using your documentation pages because they are more complete than the old stuff."

That could be sufficient. But then while I'm pretty active on this forum, I'm not sure I have the energy to spare on keeping a server up. If the community ends up having only people as "let someone else own it" stingy as me, we'll be in trouble. >.>;

---

"My concern with the sandbox idea is that I imagine it's going to be hard to create a sandbox that is both A) powerful enough to be actually useful, and B) sufficiently constrained so that there's no possible way for someone to manage to generate arbitrary Javascript."

All I'm thinking of is some hooks where Arc code can take as input an object capable of querying the scrape results and give as output a BBCode-esque representation that's fully verified and escaped before use. But then I don't know if that would be sophisticated enough for multi-page layouts or custom styles or whatnot either. ^^;

There could also be another Arc hook that helped specify what to scrape in the first place... but in a limited way so that it couldn't do denial-of-service attacks and stuff. ^^; lol

Partly it's just a curiosity for me. I like the thought of letting Arc code be run in a sandbox for some purpose, even if it's only marginally useful. :-p

---

I'd be happy to run the server, and set up some kind of simple continuous deployment system so that when someone makes a code push to the server repo the code goes live.

Depending on availability and motivation I may (or may not...) end up having time myself to get Ken's documentation into a form where it can be edited (he generously offered last year to let us do this).

A part that I don't have motivation to do myself is writing the code that would crawl Anarki and generate documentation from the docstrings.

I like the thought of letting Arc code be run in a sandbox for some purpose, even if it's only marginally useful.

I think if contributing is open and flexible people will contribute to keep the site up todate. Complete and simple instructions must exist to help and encourage people to contribute. Some is social where people feel they need "permission" to contribute.

Penknife's a programming language. I have a very specific opinion of what programming languages will become (http://arclanguage.org/item?id=12993), and I'm making Penknife in the hope that it's a well-designed first foray into that world—well-designed in the gemlike sense of not being everything to everybody but being as flawless as possible for what it is. Penknife has two very specific purposes:

- To have a customizable syntax that's fine at the start but almost limitlessly excellent once it's been customized.

- To be great at code generation, so that even if you think you want to use another language, you're likely to be more productive if you generate that code using Penknife instead.

The point is for Penknife to be a language useful to as many people as possible for as long as possible, so that it can continue to be more and more useful as its library support improves. This is sort of a world domination plot, but it isn't really (I promise :-p ). Penknife will probably only dominate syntax design (not platform design, where security and performance are hot topics), and even then it'll probably only dominate sequence-of-commands-style syntax design (the kind good at REPLs, as opposed to the landscape-of-declarations style good in IDEs). Even then, there'll be people who can't stand the foundations upon which their code generators are built; you can't please everyone. That being said, I think Penknife does stand a very real chance of monopolizing a big segment of the programming experience, so I think it's important for a gemlike language to get there first, before a hackish one has the benefit of entrenchment.

To minimize the number of fiddly mistakes (and other hackishness) in Penknife, any complexity-introducing feature that's only rarely needed for the above purposes is to be left out. Two examples are reentrant continuations and concurrency. A lack of continuation support may force certain syntaxes to be implemented without the help of continuations for coroutines or backtracking, and a lack of concurrency may hobble some theorem-proving-heavy compilers, but I doubt that'll be everyday Penknife programming. When such a nightmarish situation does come up, it's probably time to use an FFI or system call, and since you can use Penknife to generate the program you're calling that way, it shouldn't be too bad. :-p

It's quite a bit arrogant of me to think Penknife will be gemlike anytime soon, let alone that it'll still be seen as gemlike in the future. I'm still in the process of adding language features I dearly hope won't make it to the final product in their current form. Still, I think I'm the person who best knows where the heck I'm going with this project, and I intend to use Penknife myself all the time, so I'm following my passion. Feedback and help are welcome, of course!

For syntax experimentation, my favorite existing language is Arc. The Arc Forum community's very particular about every little syntactic detail. :-p So it should be no surprise that Penknife's design borrows heavily from things I like about Arc, while fixing things I don't like. For instance:

- Infix syntax is now seamless with the language. Penknife infix operators follow the normal scoping rules, as shown in the example.

- Penknife syntaxes take arbitrary []-balanced textual code, not just s-expressions, as their bodies. This allows for custom syntaxes to behave the same way as built-in "literal" syntaxes like q[...]. Case in point: the example's sym macro.

(There's more information about the above two points at http://arclanguage.org/item?id=13079.)

- Penknife has a hygienic macro system built on top of its more general-purpose syntax system (although only this high-level macro system is exposed to Penknife for now, and I'm pretty sure it's buggy despite it currently passing every test I've come up with :-p ). It's designed to work harmoniously with a module system I've also been working on. I consider both hygiene and modules to be scoping issues, so I aim to tackle them using environments, and given that premise, I figure a module import is going to be a matter of replacing the interaction environment with a local environment that shadows it, so I generally try to minimize the idea that there's a global environment at all when I can. Thus, for hygiene's sake across modules, I have each macro capture its lexical environment like a closure, and this is why [let arcsym arc.sym [mac* ...]] works in the example.

  arc> (def hello ((o name "stranger"))
         (string "Greetings, " name "!"))
  #<procedure:zz>
  arc> (hello)
  "Greetings, stranger!"
  arc> (hello "Steve")
  "Greetings, Steve!"

  wart> (def hello (? greeting "Greetings" name "stranger")
          (string greeting ", " name "!"))
  #<procedure:zz>
  wart> (hello)
  "Greetings, stranger!"
  wart> (hello :greeting "Hello" :name "Steve")
  "Hello, Steve!"
  wart> (hello :name "Steve" :greeting "Goodbye")
  "Goodbye, Steve!"