This is pretty awesome. So... I'm not trying to rain on your parade, but I noticed two things that break Lathe right away:
- When you call (split "hello" 1), it doesn't call the Arc strings.arc split function; instead it tries to call the Java String.split() method, which doesn't accept a numeric parameter. It doesn't help to say (let split ...) or (apply split "heyo" 1 nil) either. Seems like after http://www.arclanguage.org/item?id=11730 you changed your mind or something. :)
- There's a subtle difference in the implementation of setforms that messes me up. It isn't as hackable:
(Arc 3.1)
Use (quit) to quit, (tl) to return here after an interrupt.
arc> (mac foo (x) 'bar)
#(tagged mac #<procedure: foo>)
arc> (setforms 'foo.0)
((gs1722 bar) gs1722 (fn (gs1723) (assign bar gs1723)))
arc> (let s setforms (= setforms [do (prn "trace: " _) s._]))
#<procedure: setforms>
arc> (setforms 'foo.0)
trace: foo.0
trace: (foo 0)
((gs1724 bar) gs1724 (fn (gs1725) (assign bar gs1725)))
arc>
(Jarc)
Jarc> (mac foo (x) 'bar)
#3(tagged mac #<procedure>)
Jarc> (setforms 'foo.0)
((gs20 bar) gs20 (fn (gs21) (assign bar gs21)))
Jarc> (let s setforms (= setforms [do (prn "trace: " _) s._]))
#<procedure>
Jarc> (setforms 'foo.0)
trace: foo.0
((gs22 bar) gs22 (fn (gs23) (assign bar gs23)))
Notice how the Arc 3.1 implementation is self-recursive and that if it's overwritten, the old version ends up calling the new version instead of itself. Jarc's worked this way for a while too, but now either you've changed your definition of 'setforms or else you've made one too many assumptions in your occasional tail-call optimization.
Neither of these issues is a big deal for me with Lathe; I can just define another function called "xsplit" or something, and I can just ssexpand the parameter of my setforms extension myself rather than relying on the recursion to do it. But until I do those things and get farther, these potential incompatibilities may be worth your attention.
I'm really optimistic about Jarc 12+. Thanks a lot, again, for all you've done. ^_^
I've got Lathe's buggy-jarc branch working with Jarc 12 now. Woo!
I ended up reasoning that Lathe would be better off not needing to redefine basic Arc functionality at all, so I fiddled around and got it working that way. The biggest changes are a) that the codebase now uses '=fn and '=mc in place of 'def and 'mac, and b) that namespace applications like my.foo now expand to (global 'gs1234-foo) rather than just gs1234-foo, which lets (= my.foo 2) work without the 'setforms extension I was using.
These design changes aren't in the Lathe master branch yet, but they should get there eventually; whether or not hackability is important to Jarc, I feel it's important for Lathe not to depend on it, so that it's as compatible as possible with other codebases that do depend on redefinition (or indeed, Arc implementations which don't support it).
Anyway, back to Jarc. There were only a couple more hurdles I came across. The first is that cut's optional parameter isn't really optional in Jarc; it was easy enough to just call 'cut with three parameters all the time. The other issue was more insidious: Quasiquotation is still messed up.
- Giving `,2 should result in 2 rather than (unquote 2).
- Giving `(1 `(2 ,(3 ,4))) should result in (1 (quasiquote (2 (unquote (3 4))))) rather than (1 (quasiquote (2 (unquote (3 (unquote 4)))))). This isn't even consistent with Jarc 11. :)
- The following should result in (1) rather than (2):
(mac foo () 1)
(def bar () `(,(foo))) ; remember ,@ and ssyntax too
(mac foo () 2)
(bar)
- The following should result in (2) rather than (1):
(def foo () `(1))
(= ((foo) 0) 2)
(foo)
I thought about implementing quasiquote myself as an Arc macro to give you a reference implementation, but as I started in on that I noticed just how inconsistent Arc's own quasiquotation is.
In particular, in Arc 3.1, (iso ``',,@'(x) '`',x) works, but (iso ``',@,@'(x) '`',@x) gives an error. Also, (iso ``(unquote x y) ``,x) is true, which entails that not every expression which can be quote-mutated (the way (def foo () `(1)) is mutated above) is copied verbatim from the original quasiquote form.
Given these inelegances, it's quite a bit more difficult to emulate Arc than I expected. That said, had I understood these details going in, I probably would have gotten a lot farther by now, so I hope by mentioning them I can help you out too.
I've just released Jarc 13 which addresses most of these issues.
- quasiquote
I've learned that quasiquote is quite a deep rabbit hole! Jarc 13
handles quasiquote much better. I wrote a Java-based qq-expand
that is simpler than qq.arc. It doesn't handle dotted lists and
it doesn't do any optimizing, but it does handled nested quasiquotes
properly. And qq.arc is now included in Jarc 13. So if you want it,
do (use 'qq).
In Jarc 13, quasiquote is now a macro instead of a special
form. This makes it much easier for the interpreter and
compiler to share the code for handling it. Since neither
has to do any special handling for it. I was able to
rip out some code from Closure and from compile.arc. Always
like it when I can do that!
- split, cut and compiled functions with optional params
Fixed. Jaclyn now handles dotted destructed args too.
- strings
Jarc 13 includes strings.arc, code.arc and pprint.arc when
it starts.
- eval
Unrelated to this thread, but when I "fixed" eval to
ignore the lexical environment it broke the pr-tag macro
in jtml.arc. Since pr-tag is a macro it didn't need
to call eval anyway, so I just rewrote it.
- Changing the definition of a function after it's defined
This scenario is still problematic:
(def foo () `(1))
(= ((foo) 0) 2)
(foo)
While I agree with the hackability of Arc, I think it's
perhaps a problem that you can change the definition of
a function after it's defined. You are essentially
reaching into foo and changing the cons inside it.
The behavior of this is dependent on how you
optimize qq-expand.
Arc 3.1 yeilds (2) for the above code.
Jarc 13 yields (1) for the above code.
Jarc 13 with (use 'qq) yields (2).
Jarc 13 with (use 'qq) (toggle-optimize) yields (1)
As CatDancer pointed out in http://arclanguage.org/item?id=9962
there are multiple ways to expand quasiquote that all
result in the same expression when eval'd.
Unless there's a precise definition for how `(1) should
be expanded, I don't see how we can define what the output of
the above code should be.
While I agree with the hackability of Arc, I think it's perhaps a problem that you can change the definition of a function after it's defined. You are essentially reaching into foo and changing the cons inside it.
Yeah, I don't like this feature either. It makes me a bit paranoid about using quasiquote at all sometimes, since I might be unintentionally opening up parts of my macro code to being modified. I don't let that affect my coding style--I just don't usually mutate macro-expansions and stuff--but it's sort of a pet peeve.
I'm sure there's a pg post somewhere around here about it being a bit of effort to get things to work this way, but I can't find it.
Unless there's a precise definition for how `(1) should be expanded...
I think the obvious one is '(1), which is to say, replacing the 'quasiquote symbol with 'quote. If 'quasiquote is taken out of the core, then 'quote is really the only fundamental form left that you can "reach into" this way. Furthermore, Jarc 12 already worked for '(1):
If (use 'qq) gives you something that yields (2) and it's a macro, then it probably expands `(1) into '(1), and so the work is probably done for you.
This probably also means that your optimizer stops (quote (1)) from returning a reference to a single mutable (1) and makes it cons up a whole new (1) instead. If that's the case, then it isn't even much of an optimization. ^_^ I wonder if the compiled JVM bytecode can contain a sort of quote-closure field for each 'quote form, so that the compiled Java object/class can be initialized with a direct reference to a subtree of foo's syntax.
On the other hand, if this is something you don't want to fix, I'll agree with you there. :-p
Okay, I've brought Lathe's buggy-jarc branch up-to-date with Jarc 13. The code I had for Jarc 12 already worked, so I took the opportunity to remove all the unnecessary workarounds that were still in there. It sure is an awesome relief having the diff with the master branch get so small, but there are still two bugs in my way right now:
- Jarc's metafns aren't metafns. This has been a problem for quite a while, but usually I phrase it in terms of (my:foo ...) not working, since that's the way my namespace system encourages macro calls to look.
Each of the following should result in t rather than trying to apply 1 as a function:
(iso ((compose do) '(1 2 3)) '(1 2 3))
(iso ((andf do) '(1 2 3)) '(1 2 3))
For compatibility, this should work even if someone rebinds 'compose, 'andf, or some other metafn name to a completely different value. Metafn forms are identified purely by name in official Arc.
- Jarc's a&b ssyntax no longer works. The result of (ssexpand 'a&b) is (andf a b) as it should be, and (andf a b) does work correctly in most cases (the above bug being an exception), but for some reason using a&b in the code itself causes an error. It tries to look up the symbol a&b.
This isn't particular to quasiquote. You can reach into any function that returns a literal cons or string.
If this is something you don't want to fix, I'll agree with you there
Yeah, I'm disinclined to try to emulate Arc precisely in this regard because it appears to be an arbitrary artifact of how Arc does quasiquote expansion, and not a defined feature.
Yeah, I'm disinclined to try to emulate Arc precisely in this regard because it appears to be an arbitrary artifact of how Arc does quasiquote expansion, and not a defined feature.
I think the only thing going on here is quotation. Generally, quoted things work like pointers (see http://arclanguage.org/item?id=10248). Because you can still do
arc> (def foo () (cons 1 nil)) ; note: cons creates new data each call
*** redefining foo
#<procedure: foo>
arc> (= ((foo) 0) 2)
2
arc> (foo)
(1)
Then, the reason the quasiquoted thing gets weird results with the qq.arc I ported is that optimizing `(1) manages to switch between always consing new data with (list '1) and trying to reduce runtime consing with '(1).
I'm not (terribly) surprised that the ability to modify literals was intentional. Thanks for that link. I'm still wondering if `(1) expanding to '(1) is intentional or not.
Clearly '(foo) is a literal and `(foo ,bar) is not a literal, right?
I guess it's not a huge stretch to say that `(foo) is a literal. Unfortunately, qq-expand doesn't always expand `(...) with no commas into '(...)
I'm still wondering if `(1) expanding to '(1) is intentional or not.
A quasiquote expander can expand `(1) into '(1) or (list 1), or even (join '(1) ' nil), which is what Bawden's simple, correct, but inefficient quasiquote expander does.
The expander may choose '(1) as being more efficient than (list 1), but isn't required to do so to be a quasiquote expander.
Since it's an optimization, I wouldn't write code that relies on a particular instance of `(1) evaluating to the same list every time, that's just an accidental result of the optimization. Instead, use a plain quote for that.
There. Just updated qq.arc, removing almost 100 lines of cruft in the process. It still passes all the old tests, and (with any luck) is much more pleasant to work with.
This isn't particular to quasiquote. You can reach into any function that returns a literal cons or string.
I know, hence why I talked about 'quote. On principle, I'm equally paranoid about 'quote, 'quasiquote, and literal strings, but I've never mutated strings in my own projects, and I almost exclusively quasiquote my "escaping" cons cells rather than quoting them, so I just glossed over those facets of my paranoia. :-p
it appears to be an arbitrary artifact of how Arc does quasiquote expansion, and not a defined feature.
Like I said, I think there's a relevant pg comment around here somewhere. I've searched some more, but I still can't find it. Maybe it was just my imagination. ^_^;
Oh, wait, fallintothis's post links to it.
Arrgh, it would help if Google weren't so inconsistent. It's a result for ("behavior can be quite useful in some contexts"), but it doesn't show up in the results for ("behavior can be quite useful in some"), and ("can be quite useful in some contexts") has no results at all. Of course, observing this property and posting about it will no doubt change the search results. XD
I haven't looked at your quasiquote examples in particular, but note that standard Arc uses MzScheme's quasiquote expander, which is buggy for nested quasiquote expansions.
Fortunately it's easy to use a working quasiquote expander: it's a one line change to the Arc compiler so that if a "quasiquote" macro has been defined, it gets used instead of the builtin MzScheme version. Then we can load a quasiquote implementation written in Arc, such as fallintothis' port of the Common Lisp quasiquote expander.
The problem with cut is the reason why 'split' wasn't working also--it calls cut without the optional parameter. Fixed now. I'm going to try fallintothis' quasiquote in Jarc. I'll need the Java-based one for bootstrapping, I think, so will need to replace this one in. Shouuld be okay. Will need some testing.
You're not creating any rain. You're just reporting the weather, and I appreciate the information.
Yeah, even non-tail recursive functions get optimized by the compiler to avoid any run-time lookup of syms in the global environment. I'm not sure that's the right trade-off, but it seems okay for a compiler. Although hackability is important too. I'll have to think about that and research what other Lisp compilers do.
The split problems seems to be a compiler bug. I'll have to investigate that more.
And Jarc should include strings.arc, there's a conflict with java.lang.String.trim() but that should be okay.
I talk about Lathe at http://arclanguage.org/item?id=11610, which links to a blog post, which links to the GitHub repo thaddeus already linked to. ^_^
The blog post is still a pretty comprehensive look at Lathe. A bit has happened since then, but the most useful thing is Jarc compatibility, and even that isn't on the master branch yet; Lathe's approach to namespaces still doesn't harmonize with Jarc as well as I'd like.
