Inconsistent for syntax

[Quelklef]

I've noticed that for takes the iteration variable and iterable like [var iterable] whereas lfor, dfor, and gfor take it like var iterable:

(for [item iterable] body)
(lfor item iterable result)
(dfor item iterable [key val])
(gfor item iterable result)

while this isn't huge, it would be nice for these to be consistent, so lfor dfor and gfor would also look like [item iterable].

[Kodiologist]

More generally, these forms look like:

(for [clause1 clause2…] body)
(lfor clause1 clause2… result)
(dfor clause1 clause2… [key val])
(gfor clause1 clause2… result)

Are you asking for every iteration clause to get a pair of square brackets, or for the whole set of iteration clauses to get square brackets even when it's not necessary (e.g., for forms other than for itself)? In other words, do you want

(for [x xs  y ys] foo)
(lfor [x xs  y ys] foo)

or

(for [[x xs]  [y ys]] foo)
(lfor [x xs]  [y ys] foo)

? Neither really seems like an improvement to me.

[Quelklef]

Didn't know about multiple clauses, awesome!

To answer your question, of the two, I'd prefer the former, because it's consistent. Another option would be to change for to drop the braces to match lfor, dfor, and gfor:

(for x xs y ys foo)
(lfor x xs y ys foo)

(By the way, I'm aware that I'm opening a lot of issues and giving a lot of feedback while being someone very new to Hy. If I'm overreaching, please let me know.)

[Kodiologist]

(It's okay. Everybody's new to Hy except for the small handful who stick around.)

I don't think we can have for without the mandatory brackets because then you couldn't tell body forms from iteration clauses; that is, (for a b c d e) could be interpreted as (for [a b c d] e) or as (for [a b] c d e). You could remove the ability to include an arbitrary number of body forms, so (for [a b] c d e) would have to be written (for a b (do c d e)), but I'm not a big fan of that. for loops tend to have only a few iteration clauses and a lot of body forms, so setting off the iteration clauses, rather than the body forms, makes sense to me.

[yuhan0]

How about requiring brackets around the bindings in lfor, gfor and allowing multiple body forms, for consistency with the for signature?

Essentially allowing this:

(lfor [x xs  y ys]
  (body1)
  (body2)
  (result))

Which would be equivalent to the current syntax:

(lfor x xs  y ys
      (do (body1)
          (body2)
          (result)))

It seems more "lisp-like" to treat the first argument specially (the binding vector) rather than having 2N + 1 arguments and treating the last differently.

[Quelklef]

Seconded. I like that solution.

[Kodiologist]

Generally, lfor is used to produce a list rather than for side-effects; if side-effects were the point, you'd use for. So, why would you want more body forms in lfor?

[yuhan0]

For constructing intermediate values, especially since there isn't a built-in let form in Hy.
eg.

(lfor
  n (range 5)
  (do (setv sq (* n n))
      (.format "{} squared is {}" n sq)))
;; => ['0 squared is 0', '1 squared is 1', '2 squared is 4', '3 squared is 9', '4 squared is 16']

[Kodiologist]

But that's what :setv is for. Have you read the documentation for these forms?

[Quelklef]

I recognize this is a bit of a one-off usage, but it will make it much nicer to stick a debug print into a loop:

(lfor [n some-iterable]
    (print n)  ; <-- just put it in there with no worries
    (some-function n))

though I am mostly in favor for the sake of consistency.

[Kodiologist]

But that's what :do is for.

[yuhan0]

Sorry for the noise - I did read the documentation some time ago, but forgot that these special clauses existed. In a way it seems that the lfor family is much more similar to the CL loop macro than an actual Python list comprehension.

Personally I would prefer a natural way of writing these things, i.e just being able to use the standard(setv ... ) form, instead of using a specialized DSL with :setv , :do etc. clauses that don't exist elsewhere in the language.

But it's true that the :if clause exists in Python too, and I haven't thought entirely about how that would fit into my proposed syntax. Thanks for your patience though :)

[Kodiologist]

Personally I would prefer a natural way of writing these things, i.e just being able to use the standard(setv ... ) form, instead of using a specialized DSL with :setv , :do etc. clauses that don't exist elsewhere in the language.

That was my original plan; the default clause type would be to just evaluate the given form, like :do clauses, and you'd mark iteration clauses with :for instead of not having to mark them, as is currently the case. @gilch was against this, primarily on the argument that one uses iteration clauses a lot more than do-clauses. I relented once we hit on a syntax for for that doesn't require a bunch of :dos but still lets you use the other clause types.

[yuhan0]

Thanks for the explanation, it's always interesting to hear the rationale behind these syntax decisions :)

I guess you had also considered Clojure's syntax for its for loop, from what I understand these are equivalent:

;; ===== Clojure =====
(for [x xs
      :when (condition x)
      :let [p (f1 x)
            q (f2 p)]
      y ys
      :let [r (f3 y)]]
  (body1)
  (body2)
  (g x y))

;; ===== Hy =====
(gfor x xs
      :if (condition x)
      :setv p (f1 x)
      :setv q (f2 p)
      y ys
      :setv r (f3 y)
      :do (body1)
      :do (body2)
      (g x y))

It's strange how aggressively Hy tends to "flatten" its syntax, considering Clojure already does that to some extent – I find the first one more readable but guess it's a matter of getting used to..

[gilch]

I think that I had originally suggested keeping the square brackets for all the comprehensions to make it match Clojure's for (which would be my vote), but @Kodiologist didn't think they were necessary. I also considered adding a :body clause and dropping the brackets in the normal for.

But you shouldn't need multiple :do clauses to make a body. Just use a single (do) form.

(gfor x xs
      :if (condition x)
      :setv p (f1 x)
      :setv q (f2 p)
      y ys
      (do (setv r (f3 y))
          (body1)
          (body2)
          (g x y)))

If I recall, I was kind of opposed to having a :do clause at all. Even though I suggested it in the first place (inspired by Common Lisp's loop macro), it was only because @Kodiologist wanted arbitrary expressions at any point, though I still don't understand why. By the time you need one, you might as well use an explicit generator function with yield. If you just need to insert a print call, you could always replace any foo with (do (print whatever) foo). This would even work on xs or ys, in the above example, even in Clojure. :do was never necessary, just convenient. And it is convenient.

[peaceamongworlds]

I think that the syntax would be nicer if square brackets were used around the arguments for lfor, just because that would make it more consistent with other forms, like for, let, with etc. It would also put more emphasis on the actual body, distinguishing it from the other clauses.

Another case were the syntax is inconsistent is with loop. It seems to me that the extra pair of brackets around each symbol-value pair is redundant. So it should be

(loop [n 5
       acc 1]
  (if (zero? n) acc (recur (dec n) (* n acc))))

instead of

(loop [[n 5]
       [acc 1]]
  (if (zero? n) acc (recur (dec n) (* n acc))))

[asemic-horizon]

The [[n 5] [acc 1]] type syntax is what's used to declare default arguments in functions. So it matches

(defn looped [&optional [n 5] [acc 1]]
    (if (zero? n) acc (looped (dec n) (* n acc))))

(Apologies if I'm missing any brackets, I'm used to rainbow parens coloring...)

[asemic-horizon]

As for for, this took me a half while to get used to, but isn't it better ultimately to have hints that we're looking at the imperative, not functional construct?

Hy in general has a very "flat" syntax (consider how {2:3, 4:5} translates to {2 3 4 5}), but that's kind of an acquired taste, and goes better with Parinfer + rainbow brackets. Also: (lfor x xs (f x)) is a Pythonism that most of the time reads better in Hy as (ap-map (f it) xs (the irregular [f(x) for x in xs] is kind of nice in a way that doesn't translate).

[gilch]

As for for, this took me a half while to get used to, but isn't it better ultimately to have hints that we're looking at the imperative, not functional construct?

That is my feeling as well. I got used to it. I'm pretty satisfied now with how the xfor family turned out overall. It's kind of like Hy's version of the Common Lisp loop macro.

But keeping the square brackets for all the comprehensions to make it match Clojure's for was my original suggestion, and I'm still fine with that too.

[scauligi]

I also have a preference towards square brackets for all the comprehensions — it makes it visually consistent with other forms that also assign values to symbols.

I'm currently going through the language special forms and macros to generally see what is/isn't consistent and will an issue documenting the results. Some even appear to be bugs, for example the form for with doesn't seem to properly handle multiple non-assigned items.

[allison-casey]

i'm also in favor of square brackets around comprehensions since they are effectively binding pairs like (let [..] ..), (with [...] ...), and (except [...] ...) and I like the idea of all these binding forms being the same. For this reason though i'd also want to change loop to drop the brackets around the bindings since the (loop [...] ...) brackets are a paired binding form and not a lambda-list a la function definitions

[gilch]

setv also binds multiple pairs, and like setq in Common Lisp, it doesn't have any inner brackets. We don't really want to add them there, do we?

[allison-casey]

setv doesn't have an implicit do or following FORM where the bindings are intended to be used so i'd say no

[Kodiologist]

With the demise of #2102, this is unlikely to change.