Arc Forumnew | comments | leaders | submitlogin
arc-exe.scm : now faster loading, deals with compiled scheme code
6 points by sacado 6105 days ago | 17 comments
I updated arc-exe.scm (the arc standalone exe) on the git. It is now faster on startup : 1.2 s to get the prompt, vs 2.3 previously (on my machine).

The previous startup procedure was the following :

1. load arc.arc and libs

2. transform arc code to scheme code

3. compile scheme code to bytecode

4. evaluate bytecode

Step 2 and 3 are quite expensive and never change from one execution to another.

That's why I compiled code from arc.arc and the libs to mzscheme bytecode and saved this bytecode in arc.arcc. Now, only arc.arcc has to be loaded (only steps 1 and 4). This is not perfect, but it is a little better.

As a side effect, it is now possible to compile and/or run your own code this way:

- "arc-exe" gives you the prompt (default behavior)

- "arc-exe foo.arc" runs the code from foo.arc and exits (no prompt)

- "arc-exe -cc foo.arc" runs the code from foo.arc, compiles it, prints the result on stdout (a bad thing, I know) and exits

- "arc-exe -lc foo.arcc" runs the bytecode saved in foo.arcc and exits

These behaviors will seem familiar to Python users. Note that compilation only speeds up startup (just for big files actually) as you don't have to compile the code everytime. But once loaded, compiled code is as fast as raw code.



3 points by eds 6105 days ago | link

It seems that you need to do an initial

  arc-exe -cc arc.arc > arc.arcc
in order to make this work. Else scheme might complain if the version the user is using is different than what the distributer used. Perhaps arc.arcc shouldn't actually be part of Anarki? (And add something to arc-exe.scm that will automatically compile arc.arc the first time if it isn't there already?)

It might also be nice to have a feature where the .arc file would be automatically compiled to its .arcc version whenever the date on the source is newer than the bytecode. (I think Python does this with .py and .pyc files.) Although it is true that you don't get any speed benefit from doing so, and only save startup time after the first startup.

-----

3 points by sacado 6105 days ago | link

Oops. You're right. It might not work with different versions of mzscheme.

So I removed arc.arcc from Anarki. Now, arc-exe generates it automatically if it cannot find it on startup. And the -cc flag doesn't write anymore on stdout but directly in file foo.arcc (if you called arc-exe -cc foo.arc).

Your point about date diffs between .arc and .arcc files is also interesting. Python does work this way. I might add it later (if nobody does it before me :)

-----

2 points by eds 6104 days ago | link

Thats better. But its a little strange that when you run it the first two times, it compiles the arc.arc and libs.arc instead of starting up, I would expect it to both compile and start up.

I get the following errors when I try to start arc-exe:

  C:\User\Programming\Arc\arc-wiki>arc-exe
  reference to undefined identifier: _ref
  
   === context ===
  c:\User\Programming\Arc\arc-wiki\arc-exe.scm:1005:0: cload1
  c:\User\Programming\Arc\arc-wiki\arc-exe.scm:1032:0: cload
  #f::354: loop
arc.arcc now exists

  C:\User\Programming\Arc\arc-wiki>arc-exe
  reference to undefined identifier: _map
  
   === context ===
  c:\User\Programming\Arc\arc-wiki\arc-exe.scm:1005:0: cload1
  c:\User\Programming\Arc\arc-wiki\arc-exe.scm:1032:0: cload
  #f::354: loop
libs.arcc now exists

  C:\User\Programming\Arc\arc-wiki>arc-exe
  Use (quit) to quit, (tl) to return here after an interrupt.
  arc>
now arc-exe starts properly.

EDIT: Actually, arc-exe doesn't seem to be working properly. I can't access any of the predefined functions:

  C:\User\Programming\Arc\arc-wiki>arc-exe
  Use (quit) to quit, (tl) to return here after an interrupt.
  arc> prn
  Error: "reference to undefined identifier: _prn"
  arc> (prn "hello world")
  Error: "reference to undefined identifier: _prn"
  arc> quit
  Error: "reference to undefined identifier: _input-history-update"
  arc> (quit)
although at least I can quit. This problem doesn't occur with mzscheme -mf as.scm.

This is using Anarki cloned on Wed Feb 27 22:36:04 PST 2008.

-----

1 point by sacado 6104 days ago | link

That is very strange, I have none of the problems you mention. Even the first time, everything works fine. What version of mzscheme are you using ?

-----

2 points by eds 6104 days ago | link

I'm using version 352 on Windows.

-----

2 points by eds 6103 days ago | link

The problem ended up being that the definition (xdef 'ref ...) was missing from arc-exe.scm for some reason. Copying it over from ac.scm fixed all the problems I was encountering. Pushed the fix to Anarki.

-----

3 points by nex3 6103 days ago | link

That would be because I defined ref, and only thought to put it in ac.scm. We should come up with a way of dealing with this. I suppose we could admonish everyone to make changes to both files, but that seems a little annoying and apt to fall victim to forgetfulness.

Would you be willing to take on the responsibility of running "git log ac.scm" periodically and copying stuff over?

-----

2 points by almkglor 6103 days ago | link

Wouldn't it be possible for arc-exe to just (load "ac.scm") somewhere? Admittedly I don't know enough about the mzscheme compilation process to figure out why it won't work that way - something to do with modules not being able to use (load ...), maybe?

-----

1 point by sacado 6103 days ago | link

No, it's not possible. There are problems with mzscheme's namespaces. I had to fight with them to make arc-exe work, and simply importing ac.scm in an englobing file will just not work.

-----

3 points by almkglor 6103 days ago | link

Hmm. How about a macro which reads in the code from ac.scm and inserts it into a progn form? Would that work?

  (defsyntax insert-file-here ()) ; dunno, never figured out scheme macros

  (module arc-exe mzscheme
    (insert-file-here "ac.scm")
  )
Where (insert-file-here "ac.scm") would expand to:

  (progn
    (provide (all-defined))
    ...
    (define (ac s env)
    ...)
  )
Would putting the defines in (progn ...) work? Sorry, I'm not very good at macros in scheme, never managed to grok hygiene.

The alternative is to have the macros generate the (module ...) code, and insert the code inside the modules of ac.scm and as.scm and what else arc needs.

Basically what I'm proposing is writing a macro which does what you originally did in creating arc-exe.scm. The unfortunate part is that I never managed to grok Scheme macros, otherwise I'd offer to do it for you if you describe how you built arc-exe.scm.

-----

3 points by sacado 6103 days ago | link

Looks like an excellent proposition. I'm not very good with scheme macros either, however I think mzscheme has an à la Common Lisp define-macro extension. I'll look about that more precisely.

-----

1 point by eds 6103 days ago | link

Is it really namespaces that are a problem, or modules?

I hacked together a short version of arc-exe.scm that worked just fine importing from ac.scm. The only problem with that version was that it wasn't a module, and thus couldn't be compiled by mzc. I couldn't make it a module because scheme complained when brackets.scm tried to override read and read-syntax. (Apparently overriding imported bindings only works outside modules.) And when I tried to modify or not include brackets.scm, it would complain that it couldn't find the module ac.

So did you encounter this particular problem or something else? My guess is that if I could make brackets.scm behave, I could make the rest of it fall in line. But maybe I am mistaken about this.

-----

2 points by eds 6103 days ago | link

Me? I'd be fine with that.

-----

1 point by almkglor 6105 days ago | link

Hmm. If arc.arc is modded, would it be possible to use:

  arc-exe -cc arc.arc > arc1.arcc
  mv arc1.arcc arc.arcc
in order to update arc.arcc on our machines without waiting for your update?

-----

1 point by sacado 6105 days ago | link

I think it would work. But you will have to put both arc.arc and libs.arc in arc.arcc :

  cat arc.arc libs.arc >all.arc
  arc-exe -cc arc.arc > arc.arcc

-----

2 points by steadicat 6103 days ago | link

You mean:

    arc-exe -cc all.arc > arc.arcc

-----

1 point by sacado 6103 days ago | link

Yep. Sorry about that. Anyway, this is not required anymore. Just run arc-exe and arc.arcc will be generated automatically.

-----