Arc Forumnew | comments | leaders | submitlogin
Modules : a proposition
4 points by sacado 6153 days ago | 15 comments
I was just thinking about a minimalist module system for Arc. I tried to implement it, but it's far to be done.

Here's the idea : a macro called module defines a module (sic). In this module, a new (virtual) namespace is created. Both the module and the global namespace can then be accessed :

  (module 'foo
    (m= a 'in-foo)
    (= a 'in-top)
    (mdef bar (x) (+ x 1))
    (pr a)
  )
This declares a module (a default namespace) called "foo". There are two more macros : m= and mdef ; they are the counterpart of = and def, but the defined variables / functions are prefixed by "foo@". Thus, the above code is the equivalent of

(= foo@a 'in-foo) (= a 'in-top) (def foo@bar (x) (+ x 1)) (pr foo@a)

As you can see, m= defines a package variable (that's the new part), = a global one (as usal, since you must sometimes modify global values).

Inside the module, a is equivalent to foo@a (to make it work, I had to hack a little arc-var-ref in ac.scm). In foo, a is hidden by foo@a.

Once outside the module, we're back in the global namespace. So the variable has to be prefixed by foo@, or else we're talking about the global one :

  arc> a
  'in-top
  arc> foo@a
  'in-foo
There is no magic here, the @ is just an arbitrary character I chose, foo is not a hash table or something like that. It is just a string-append.

mmac should be added I guess.

Good points :

  - modules can be ignored if you don't want them. symbols from modules are just plain symbols with an @ in the middle
  - the mechanism can be written in a few macros only
Bad points :

  - not yet totally implemented
  - needs to modify the axioms in *ac.scm*, which could drive pg crazy as this is not the priority :)
What do you think of this idea ? Do I miss very important things ? Should I spend my time on more valuable things ?


8 points by raymyers 6153 days ago | link

Perhaps modules should be valid containers. Thus instead of introducing a new syntax foo@a, we can use (foo 'a) and foo!a using sugar already in place. (I realize I am not the first person to say that.) I also would not like having to use mdef and m= within a module. Ideally, I should just use def, mac, and = as normal and export the symbols I want visible from outside.

    (module foo (export a bar)
      (= a 'in-top)
      (def foo (x) (+ x 1))
      (def bar (x) (+ x 1))
      (pr a))
    (prn foo!a)
    (foo!bar 0)
    (= bar foo!bar) ; an import.
    (= a2 foo!a)    ; a qualified import.
This could all be implemented as an optional library, except for those pesky macros. And of course, we do need a module system with adequate macro support.

-----

1 point by sacado 6152 days ago | link

What about something like what you are proposing, but where you have to explicitly state when you use the module namespace, since we obviously can't overwrite = without sad effects ?

For example, let's take the $ symbol (another ugly one :) to mean "the current module". The above could be written :

  (module foo (export a bar)
      (= $!a 'in-top)
      (def $!foo (x) (+ x 1))
      (def $!bar (x) (+ x 1))
      (pr a))
    (prn foo!a)
    (foo!bar 0)
    (= bar foo!bar) ; an import.
    (= a2 foo!a)    ; a qualified import.
Advantages : - very simple - written in pure Arc (thus candidate to the core language)

Here's a macro implementing part of that behavior : (= $ nil) (= $path* '()) ; Current module hierarchy

  (mac module (name . body)
        (w/uniq old-$
                `(with (,old-$ $)
                        (= $ (table))
                        (push $ $path*)
                        ,@body
                        (pop $path*)
                        (if $path*
                                (= (,old-$ ',name) $) ; Put it in the parent
                                (= ,name $))          ; Put it in global namespace
                        (= $ ,old-$))))
It supports imbricated modules. To import a module, or do a qualified import, the classical table manipulation functions work.

  (module foo
    (module bar
      (= $!baz 'arc)))

  arc> foo
  #hash((bar . #hash((baz . arc))))
  arc> (= bar foo!bar)
  #hash((baz . arc))
  arc> bar!baz
  arc

-----

2 points by raymyers 6152 days ago | link

It sounds like several of us are working on different module ideas. How about we start throwing our prototypes in a `lib/module' directory on the git?

-----

1 point by almkglor 6151 days ago | link

Done, hacking off yours since it was already there ^^

-----

2 points by almkglor 6152 days ago | link

This requires us to have decent code-traversal.

Question (dunno the answer, not on my hacking computer): how will a macro receive the expression (foo:bar foo)? As (foo (bar foo))? as ((compose foo bar) foo)? As (|foo:bar| foo)? (|| is used to enclose a literal in symbols)

-----

1 point by sacado 6152 days ago | link

  arc> (macex 'foo:bar)
  foo:bar
Too bad :(

-----

3 points by cooldude127 6153 days ago | link

i like the looks of this. i fully support taking advantage of the simplicity arc already has.

-----

1 point by sacado 6152 days ago | link

There's a problem with using = and def. I tried it, but many parts of the core and libraries assume a unique namespace. As commands can have many side-effects, everything breaks easily.

But the idea of using tables is excellent.

-----

1 point by sacado 6152 days ago | link

Anyway, we must be able to access the global namespace too. If you overwrite = and def, you can't. Or you have to define g= and gdef :)

-----

4 points by almkglor 6152 days ago | link

Here's a different idea. Suppose that instead we build a basic modulesystem which transforms:

  (modules-base
     ;name of module.
     foo
     ;set of functions in this module
     (bar)
     ;set of module variables
     (nitz)
     ;set of functions from other modules
     ((module2 hmm niawniaw))
     (def bar (x) (hmm) (niawniaw) (do1 nitz (= nitz x))))
to:

  (= foo
     (with
        (bar nil nitz nil hmm module2!hmm niawniaw module2!niawniaw)
       (= bar (fn (x) (hmm) (niawniaw) (do1 nitz (= nitz x))))
       (fill-table (table) 'bar bar)))
Then we create another macro which simply scans through the code for (def ...) forms and transforms the following code:

   (module foo
     (use module2)
     (module-vars nitz)
     (def bar (x) (do1 nitz (= nitz x))))
to:

  (modules-base
     foo
     (bar)
     (nitz)
     ;gotten by taking (keys module2)
     ((module2 hmm niawniaw))
     (def bar (x) (do1 nitz (= nitz x))))
Weaknesses: (1) we can't make module-variables accessible outside. If we had access to environments, though, we could.

(2) macros are impossible as yet, whether shared or not. Possibly, we need macrolet, and adding some mechanism to store macros separately from the module table - possibly in a table-of-tables module-macros.

Implementation: simple scanning would be nice. However, modules-basic would be better implemented by a 'macrolet form.

Conclusion: We need macrolet.

-----

3 points by eds 6151 days ago | link

I know this doesn't address many of the issues with a module system in arc... but there is currently an import.arc file in the arc-wiki that has a few interesting points.

In the current import.arc, when you import a module, the name of the module gets set to a macro which allows you to access functions in the module via composition. Thus you get to use : rather than another arbitrarily chosen character. I just thought that was kind of cool.

  arc> (load "lib/import.arc")
  nil
  arc> (imp array)
  #3(tagged mac #<procedure>)
  arc> array
  #3(tagged mac #<procedure>)
  arc> array:array?
  #<procedure>
  arc> (array:array? 1)
  nil
Maybe there is a problem with just simple composition that doesn't scale when applied to a full blown module system. But if you could incorporate this into any ideas currently going around, I think it would clean up the syntax for accessing internals of a module at the very least.

-----

2 points by pau 6153 days ago | link

I think that discussing a module system is valuable in itself. I also think it must be done sooner rather than later. For missing things, take a look at http://arclanguage.org/item?id=2580.

There is some convergence, but only in the ugliness of '@'... ;)

-----

1 point by vrk 6152 days ago | link

Most module system proposals so far suffer from the same problem: macros won't work properly. However, this is not my conclusion, but what many people have stated. It's not entirely clear to me what the exact problems are. Could someone elaborate?

-----

5 points by soegaard 6152 days ago | link

Matthew Flatt.

"Composable and Compilable Macros: You Want it When?".

International Conference on Functional Programming

(ICFP'2002)

http://www.cs.utah.edu/plt/publications/macromod.pdf

And you are right, making modules and macros work together turns out to be harder than most people realize. Schemers have thought long and hard about the problem. It would be wise to look at the papers at

    http://library.readscheme.org/page5.html
before designing a module system.

-----

2 points by cpfr 6152 days ago | link

There needs to be a mechanism of merging modules together. Without this modules that import modules are going to end up with very nasty long names. Being able to give a module a nickname is handy.

-----