I really like the auto-documentation feature that arc currently has. However, I've recently thought that it would be a good idea to add support for not just looking up docstrings, but source code as well. Being able to look up source code by function name would be a major feature.
Therefore, I propose a global "source" table, that gets added to by every def and mac, just like the global help* table. It could be left as just source, so that the lookup would be a direct table lookup, i.e. (source fn), but if you all think it would be a better idea, we can make wrapper functions for a source* table instead.
1) def and mac now add entries to a table named source*
2) there are now two macs fro accessing those entries: lsrc and src. lsrc returns the source as a list, so that you can modify it, etc. src pretty-prints lsrc, so that you can read it better from the repl.
3) I also changed fns so that it a) takes both arguments as optional and b) will accept symbols as arguments. Here's a few examples on how to use fns:
arc> (fns)
*returns the signatures of all macros and functions currently defined*
arc> (fns f)
*returns the signagures of all functions and macros beginning with f*
arc> (fns nil [fancy-filter _ ])
*returns the sigs of all fns and macs which pass fancy-filter.*
In the last form, the br-fn cannot reference the first argument.
It could possibly be modified so that it tests if the single argument is a function or a symbol, and if it is a function it treats it as the filter, otherwise it uses prefix with the string of the symbol.
One of the other ideas I've had for the auto-doc feature of arc is a modification to the help op on the web server that instead of just looking at names between [[ ]] a) prints the source if in verbose mode and b) if so searches the source tree of the fn or mac for all symbols bound to fns or macs and makes them links. That would provide a pretty cool code explorer tool that would (almost) rival arc-fn
So I have a few questions:
1) What is the best way to search a source tree.
2) Currently src works by having a hook in def save the source to the table just like help, but that means that only things defined by def and mac actually have source entries. What about all of the xdefed functions? Should I add a hook to xdef as well? We could add docstring support to xdef too I suppose.
3) Since src saves the source after read, it has all of the reader macros already expanded, such as backquote and br-fns. Is there a way to get around that so that the source is more readable?
Is it possible to condense the form (quasiquote l), (unquote l), (quote l) and (make-br-fn l) into their respective string based syntaxes? How would you go about doing this?
What is the arc method for mapping a function across a tree?
>I don't know precisely how you're doing this, though, so that might not work.
All my source table does is
(sref source* '(def ,name ,parms ,@body) ',name)
for def and
(sref source* '(mac ,name ,parms ,@body) ',name)
for mac.
Since name, parms, and body are read by the reader, reader macros have already been expanded. I'm looking for a way to print the source in a more readable fashion.
Your functions looks pretty close to what I want. Maybe stringify could just be syntaxify, and make it recursive? I don't know if tail recursion is possible because of make-br-fn, but the rest of it looks pretty simple.
If the current object is a cons, but not one of the above, return "(" (syntaxify:cdr _) ")", and if it's an atom return (string _).
I'll do some more thinking on it, but it looks like you're on the right track.
Adding spaces shouldn't be too hard; are you saying that we don't know how to handle indentation and newlines? I don't think that should be too hard. Looking at ppr and friends in string.arc should give us a clue as to that.
Is it possible to create a list in which the syntax is not expanded? I.e. preserve our syntaxification, after we've completed it? So that ', `, , ,@ [] become part of their symbols? That would probably be helpful, because then we could just use the built in ppr function, and not have to worry about formatting.
Does anyone use the "fns" function? I am considering either changing it or replacing it with a better function lister function.
If you had to pick a name for a function that listed all the currently defined functions or macros, or allowed you to filter them, what would it be? Would you stick with the current name "fns" or not?
'fns is nice, but I'd like to be able to pass it symbols, not just strings. (Just run 'string on the argument.) I'll push this to anarki next chance I get, unless there are objections.
It would also probably be nicer if passing the 'test argument didn't make the 'pfx argument irrelevant - it's not really an optional argument, it's a bimodal function. Perhaps just testing whether the 'pfx argument was a function and using it as test if it were, and otherwise using [prefix string.pfx _]? This is a bigger change, though, and I won't push it unless people think it's a good idea.
I've done most of that, but I haven't pushed it yet. The only thing different is that I made pfx optional too, so that it would return all functions if you called fns with no args.
Another difference is that I made fns a macro so that it would quote the symbol passed to it, instead of evaluating it. Maybe that's what you mean. Unfortunately, the scheme of taking in bare symbols as prefixes is rather incompatible with the test to see if it's a function.
Right now the way you'd call it if you wanted your own function without a prefix is (fns nil [ex _]).
currently fns takes an optional filter argument, and defaults to prefix. i.e., you give it a string, and it keeps all of the keys from the "sig" table that begin with that string. So it finds both macs and defs.
I've modified my version so that a) if you don't give any arg it returns the signatures all currently defined functions and macros, and b) it can take unquoted symbols as input instead of just strings. That makes use from repl easier.
I don't know how I would filter defs from macs. Maybe have def and mac store them in a different table? How do you get the object associated with a symbol? If I knew that, we could filter based on (type key).
I just found the macro varif on Anarki, which returns the value if bound. Would that be better, or worse? I should think it might be a little better, but I don't know.
True, this is running, I think, in the repl. So I don't think it matters that much. But having a good way to look up the value of a symbol is kind of important, I think. I want to do similar things that might be used in a "performance-sensitive" environment, not that arc is really performance sensitive in general. ;)
rntz wanted two separate functions to search macs and defs, so I suggested filtering the sig table based on whether each symbol was a mac or fn; that would only be possible if there was a way to turn the symbol into a value we could take the type of.
Are you thinking to have at runtime or at edit-time? I ask because my IntelliJ plugin for Arc does this - you just hit cmd-shift-I (on a Mac, at any rate), and a little window pops up with the source code for whatever the cursor is on. Is this what you mean...?
I was thinking at runtime, I.e. in the repl or available via the online help op. It would add greatly to the automatic documentation that currently exists.
What do you suggest we do about all of the foundation functions? Most are defined in scheme using xdef, some using set (def and mac). I could add hooks to those (especially xdef) but I don't know how well that would work.
Hard coding it probably isn't a good idea; the source would get out of date, and be hard to maintain. Besides, it violates the DRY principle.
Would you guys prefer that the access function to the source table be named "src" or "source"? I don't know if src would cause any confilcts, but it fits with the current abbrev. style ;)
It should be named 'source* . Naming conventions are a good thing. Abbreviation for things like this is unnecessary. Also 'src or 'source might be a good name for a macro to get the source of a given function, much like 'help is now:
