Arc Forumnew | comments | leaders | submitlogin
Help needed with macro
3 points by skenney26 5983 days ago | 24 comments
I've been trying to write a version of rand-choice that chooses between weighted choices:

  arc> (biased-choice 3 'black 3 'white 1 (randcolor))
  white
  arc> (biased-choice 3 'black 3 'white 1 (randcolor))
  "#3c4ea4"
  arc> (biased-choice 3 'black 3 'white 1 (randcolor))
  black
I've tried writing this macro several different ways but the only one that works so far uses eval. OnLisp mentions several times that its usually a bad sign when eval is called at runtime.

This is the macro so far:

  (def unzip (xs)
    (let ps (pair xs)
      (list (map car ps) (map cadr ps))))

  (def sum (xs)
    (apply + xs))

  (mac biased-choice args
    `(withs ((bs cs) ',(unzip args)
             r (rand (sum bs)))
       ((afn (bs cs i)
          (if (< r (sum (cut bs 0 i)))
              (eval (cs (-- i)))
              (self bs cs (++ i))))
        bs cs 1)))
Any ideas on how to correct or improve it?


3 points by tokipin 5982 days ago | link

here's my take

  (def wrand args
      (withs (weights (sort (compare > car) (pair args))
              unit   (/ 1.0 (reduce + (map car weights))))
          
         (ccc (fn (return)
             (with (acc 0 r (rand))
  
                 (each (weight val) weights
                     (= acc (+ acc weight))
                     (if (< r (* acc unit))
                         (return val))))))))
i think the sort might be unnecessary but my mind isn't functioning atm

-----

1 point by skenney26 5982 days ago | link

Shouldn't there be a simple, elegant solution to this problem that doesn't necessitate using ccc and return? Maybe I'm resistant to using continuations because I'm still coming to grips with understanding them... but there's alot of code in arc.arc and only one call to ccc.

-----

1 point by rkts 5982 days ago | link

You can write a tail-recursive version of tokipin's code pretty easily. Personally though I think this could be an addition to the "examples of LOOP" thread from a while back.

  (defun wrandf (xs weights)
    (loop with r = (random (apply #'+ weights))
          for x in xs
          for w in weights
          for cw = w then (+ cw w)
          when (> cw r) return x))

  ; assumes pair
  (defmacro wrand (&rest args)
    `(funcall
       (wrandf
         (list ,@(mapcar (lambda (x) `(lambda () ,(cadr x))) (pair args)))
         (list ,@(mapcar #'car (pair args))))))

-----

1 point by almkglor 5982 days ago | link

The sort is indeed unnecessary, and the solution above is indeed the classic solution to the weighted random choice

-----

1 point by fallintothis 5982 days ago | link

Besides the other solutions noted that are more "classic" answers to the problem, I think it's important to ask: why does this have to be a macro? Using eval within a macro, in my experience, is often a sign that you don't need it; you want a function instead. I can't see any signs of the need here either. To say nothing of the soundness of the original solution's method, I believe that this would work:

  (def unzip (xs)
    (let ps (pair xs)
      (list (map car ps) (map cadr ps))))

  (def sum (xs)
    (apply + xs))

  (def biased-choice args
    (withs ((bs cs) (unzip args)
             r (rand (sum bs)))
      ((afn (bs cs i)
         (if (< r (sum (cut bs 0 i)))
             (cs (-- i))
             (self bs cs (++ i))))
       bs cs 1)))
If I'm mistaken for some reason, please beat some sense into me.

-----

1 point by skenney26 5982 days ago | link

The problem with using a function is all the arguments are evaluated.

  arc> (biased-choice 3 'black 3 'white 1 (pr 'blue))
  blueblack

-----

1 point by almkglor 5982 days ago | link

If all weights are ints:

  (mac biased-choice args
    (w/uniq (choice nums sum fns rsum)
      (givens (pairs        (pair args)
               weight-exps  (map [_ 0] pairs)
               choice-exps  (map [_ 1] pairs)
               )
        `(givens (,nums   (list ,@weight-exps)
                  ,sum    (apply + ,nums)
                  ,choice (rand sum)
                  ,fns    (list ,@(map [idfn `(fn () ,_)]
                                       choice-exps))
                  ,rsum   0)
           (while (< rsum choice)
             (zap + rsum (car nums))
             (zap cdr nums)
             (zap car fns))
           ((car fns))))))

The above macro supports that the weights are expressions instead of constants

(untested)

-----

1 point by almkglor 5981 days ago | link

ah crick: here's a working debugged version:

  (mac biased-choice args
    (w/uniq (choice nums sum fns rsum)
      (givens pairs        (pair args)
              weight-exps  (map [_ 0] pairs)
              choice-exps  (map [_ 1] pairs)
        `(withs (,nums   (list ,@weight-exps)
                 ,sum    (apply + ,nums)
                 ,choice (rand ,sum)
                 ,fns    (list t ,@(map [idfn `(fn () ,_)]
                                        choice-exps))
                 ,rsum   0)
           (while (<= ,rsum ,choice)
             (zap + ,rsum (car ,nums))
             (zap cdr ,nums)
             (zap cdr ,fns))
           ((car ,fns))))))
Also: the reason it needs ints is because of the 'rand function. We could also define a rand-float function which creates a random floating point number and use that instead:

  (def rand-float (lim) (* lim (rand)))

  (mac biased-choice args
    (w/uniq (choice nums sum fns rsum)
      (givens pairs        (pair args)
              weight-exps  (map [_ 0] pairs)
              choice-exps  (map [_ 1] pairs)
        `(withs (,nums   (list ,@weight-exps)
                 ,sum    (apply + ,nums)
                 ,choice (rand-int ,sum)
                 ,fns    (list t ,@(map [idfn `(fn () ,_)]
                                        choice-exps))
                 ,rsum   0)
           (while (<= ,rsum ,choice)
             (zap + ,rsum (car ,nums))
             (zap cdr ,nums)
             (zap cdr ,fns))
           ((car ,fns))))))
the above now works with weight expressions that return real numbers. Also as specified, only the chosen expression is executed; however, all weight expressions are executed.

-----

1 point by fallintothis 5982 days ago | link

Ah, I misread that you wanted something akin to random-elt rather than rand-choice -- i.e., you'd want to use this as a control structure, in which case a macro indeed is what you'd need. My bad. That's what I get for commenting on an empty stomach (well, empty brain is more like it, but excuses are entertaining).

-----

1 point by skenney26 5982 days ago | link

Still sucks but slightly better:

  (mac bias args
    (withs (ps (pair args)
            bs (map car ps)
            cs (map cadr ps))
     `(let r (rand (apply + ',bs))
        ((afn (b c i)
           (if (< r (apply + (cut b 0 i)))
               (eval (c (- i 1)))
               (self b c (+ i 1))))
         ',bs ',cs 1))))

-----

1 point by skenney26 5981 days ago | link

  (mac bias args
    (withs (ps (pair args)
            bs (map car ps))
     `(let r (rand (apply + ',bs))
        (if ,@(mappend list
                (rev (accum a
                       (for i 1 (len bs)
                         (a (list '< 'r
                                  (apply + (cut bs 0 i)))))))
                (map cadr ps))))))

-----

1 point by skenney26 5981 days ago | link

  (mac bias args
    (withs (ps (pair args)
            bs (map car ps))
     `(let r (rand (apply + ',bs))
        (if ,@(mappend list
                (map [list '< 'r (apply + (cut bs 0 _))]
                     (range 1 (len bs)))
                (map cadr ps))))))

-----

1 point by skenney26 5981 days ago | link

  (mac bias args
    (withs (ps (pair args)
            bs (map car ps))
     `(let r (rand (apply + ',bs))
        (if ,@(let i 0
                (mappend
                  [list (list '< 'r (apply + (cut bs 0 (++ i)))) _]
                  (map cadr ps)))))))
This one is very similar to the definition of rand-choice. (Why does the mappend expression work? Aren't the values of the biases unavailable during macro-expansion?)

-----

1 point by almkglor 5980 days ago | link

> (Why does the mappend expression work? Aren't the values of the biases unavailable during macro-expansion?)

They are if the biases are constant numbers.

See my solution above instead for a way of making it work with non-constant biases

-----

1 point by skenney26 5980 days ago | link

This version still needs some massaging but it works with non-constant biases:

  (mac bias args
    (let bs (map car (pair args))
     `(let r (rand (+ ,@bs))
        (if ,@(let i 0
                (rev (accum a
                       (each c (map cadr (pair args))
                         (a `(< r (+ ,@(cut bs 0 (++ i)))))
                         (a c)))))))))


  arc> (with (a 1 b 2 c 3) (bias a 'red b 'white c 'blue))
  white

-----

3 points by rkts 5980 days ago | link

Your macro has problems with variable capture and multiple evaluation (see chapters 9-10 of On Lisp). Here's a version that should work properly:

  (mac bias args
    (w/uniq r
      (withs (ws (map car  (pair args))
              xs (map cadr (pair args))
              us (map [uniq] ws))
        `(with ,(mappend list us ws)
           (let ,r (rand (+ ,@us))
             (if ,@(mappend
                     (fn (u x) `((< (-- ,r ,u) 0) ,x))
                     us xs)))))))
IMO, though, the use of a macro here is a premature optimization. I think you should try to get a function working first, and then wrap a macro around it if you know that's what you need. See my comment http://arclanguage.org/item?id=7760 for an example of such a wrapper macro (in CL, but the Arc is similar).

-----

1 point by skenney26 5978 days ago | link

  (mac bias args
    (w/uniq (bs r)
     `(withs (,bs (list ,@(map car (pair args)))
              ,r  (rand (apply + ,bs)))
        (if ,@(mappend
                (fn (c) `((< (-- ,r (pop ,bs)) 0) ,c))
                (map cadr (pair args)))))))

-----

1 point by skenney26 5979 days ago | link

Interesting solution. I like the clever use of --

-----

1 point by almkglor 5980 days ago | link

I still suggest you take a look at how I do it http://arclanguage.com/item?id=7765 , which (1) avoids multiple evaluation, and (2) avoids variable capture.

(1) is the hard part here, which is why I had to use a list.

-----

1 point by rincewind 5982 days ago | link

this only works for integer weightings

  (mac biased-choice choices
     (cons 'rand-choice (apply join (map [n-of car._ cadr._] pair.choices))))

  (def biased-choice choices
     (random-elt (apply join (map [n-of car._ cadr._] pair.choices))))
the second evaluates all of its arguments, which may or may not be what you want

-----

1 point by rincewind 5982 days ago | link

this works for fractional weightings also

  (def reducecollect (sum list start)
    (if acons.list
        (let csum (sum start car.list)
            (cons csum (reducecollect sum cdr.list csum)))))

  (def zip args
   (when (all acons args) 
      (cons (map car args) (apply zip (map cdr args)))))

  (mac biased-choice choices
   (with (ranges (reducecollect + (map car pair.choices) 0)
          items (map cadr pair.choices)
          random (uniq))
          `(let ,random (* (rand) ,last.ranges)
               (if ,@(apply join (zip (map [list '< random _] ranges) items))))))

-----

3 points by rincewind 5977 days ago | link

zip (like that in python) can be written shorter with map

  (def zip args (apply map list args))

-----

1 point by skenney26 5977 days ago | link

And here's a better definition of unzip:

  (def unzip (xs (o n 2))
    (apply map list (tuples xs n)))

  arc> (unzip '(a b c d e f) 3)
  ((a d) (b e) (c f))

-----

1 point by tokipin 5982 days ago | link

you can multiply by 10^n to get n-digit decimal precision

-----