:keywords as objects

[algernon]

The main reason keywords are invalid unicode strings is that we didn't want to leak Hy objects into python. Having used Hy in production for a while, and planning to deploy it yet another place, I do not see an issue with leaking Hy objects into python space.

Therefore, I propose we turn keywords into proper objects!

Pros:

• (string? :keyword) would do the right thing.
• We could make keywords callable, which'd give us quite a bit of neat syntax: (map :keyword [{:keyword "found me!" :closet :skeleton}]) => ("found me")
• Less keyword-specific magic in the compiler

Cons:

• Hy objects leak through into python.

[rwtolbert]

👍 👍

[agentultra]

The other alternative is to not have keyword objects. Python doesn’t have keyword objects.

On Jul 24, 2015, at 10:17 AM, Gergely Nagy [email protected] wrote:

The main reason keywords are invalid unicode strings is that we didn't want to leak Hy objects into python. Having used Hy in production for a while, and planning to deploy it yet another place, I do not see an issue with leaking Hy objects into python space.

Therefore, I propose we turn keywords into proper objects!

Pros:

• (string? :keyword) would do the right thing.
• We could make keywords callable, which'd give us quite a bit of neat syntax: (map :keyword [{:keyword "found me!" :closet :skeleton}]) => ("found me")
• Less keyword-specific magic in the compiler
Cons:

• Hy objects leak through into python.
—
Reply to this email directly or view it on GitHub.

[algernon]

That is certainly an option too. But it looks a bit weird to have (import [foo :as bar]), but not having keywords. If we get rid of keywords, we'll need to figure out a way to change imports too, I'd say.

[algernon]

Almost two years in, and having Hy depolyed into production at various places, it turns out that I can live without keywords as objects. As noone else stepped up to implement them either, lets get rid of this issue!

[gilch]

After #1352, I want to think about this some more. Keywords as (invalid) strings are kind of weird. They're awkward in macros. Leaking \ufdd0 into Python isn't that nice either. A proper object would be better-behaved even in Python. We can already leak HySymbols but without much issue.

The other alternative is to not have keyword objects. Python doesn’t have keyword objects.

I think Python could have used a good symbol type. But keywords on top of that actually don't seem that useful in Hy. But then how would kwargs work? Clojure does have a separate keyword type, but in other Lisps, keywords are just symbols that start with :, and these are special-cased to evaluate to themselves. We can still have a predicate to tell if a symbol starts with :.

Unifying keywords and symbols would probably simplify the compiler and macros.

We could also make symbols callable, so (foo {'foo 42}) would call the function named by foo, but ('foo {'foo 42}) would call the symbol itself and return 42. Clojure actually does this. But since keywords would just be self-evaluating symbols, (:bar {:bar 42}) would also work.

[refi64]

Is there a reason we can't just make it a compile error to use keywords outside of a keyword context or macro? IMO I really can't think of a major use case for them that wouldn't be complicated by their existing usage.

[gilch]

Is there a reason we can't just make it a compile error to use keywords outside of a keyword context or macro?

I don't understand how that could possibly work.

Hy can use keywords to name kwargs in function calls, and as control words special forms, like :as in import. These won't always follow the same key/value format used by kwargs. See, for example, #1371 which can usefully repeat the same keyword multiple times in a list.

Macros should at least be able to do the same as special forms.

Macros must be able to build any HyExpression. And we have to have some object to represent keywords for macros to be able to build HyExpressions that contain keywords. And they have to be first class. That is, functions must be able to take these keyword objects as arguments and return one as a result, or macros won't be able to delegate work involving them to functions. The only way to avoid this is to get rid of keywords altogether. But then how do you write Python kwargs in Hy? We need that for interop.

We could use a separate HyKeyword Hy model type, following Clojure, or we could make it a special case of HySymbol somehow, like Emacs Lisp and Common Lisp do.

[gilch]

Now that #1543 is merged, what do we think about making them callable like in Clojure? We can do the same for symbols, but strings and displays would be harder, requiring compile-time hacks that might not even work that consistently #963. An alternative might be a tag macro that makes the basic collections callable. E.g.

=> (deftag ? [form] `(. ~form __getitem__))
from hy import HyExpression, HySymbol
import hy
hy.macros.tag('?')(lambda form: HyExpression([] + [HySymbol('.')] + [form] +
    [HySymbol('__getitem__')]))

<function <lambda> at 0x0000019D4FB927B8>
=> (#?[10 11 12] 2)
[10, 11, 12].__getitem__(2)

12
=> (tuple (map #?[10 11 12] [1 0]))
tuple(map([10, 11, 12].__getitem__, [1, 0]))

(11, 10)

Of course, this would make a string work like a list of characters.

=> (#?"spam" 2)
"""spam""".__getitem__(2)

'a'

Which may or may not be what the user expects. Strings are not callable in Clojure, unlike symbols and keywords.

But if keywords become callable, perhaps they could look up the equivalent string instead of itself?

=> (:spam {"spam" 1  "eggs" 2})
HyKeyword('spam')({'spam': 1, 'eggs' 2})

1

But since a symbol is-a string in Hy, if we make symbols callable, they'll do that naturally. So keywords could be restricted to looking up keywords as in Clojure. Another option is to make HyString callable. This way you can use a string literal as a lookup function if you quote it.

=> ('"spam" {"spam" 1  "eggs" 2})
HyString('spam')({'spam': 1, 'eggs' 2})

1

[Kodiologist]

Keep in mind that using a callable keyword literal to a higher-order function would require quoting it, since (map :key foo) would be interpreted as map(key=foo).

[gilch]

Ah, yep. That does make them seem a bit less convenient as functions, since that would be easy to forget. Until you do it a lot. But it would certainly be possible to pass them to higher-order functions.

But I think we're already using a compile-time hack to make them seem callable.

=> (:a {:a 1  :b 2})
from hy import HyKeyword
{HyKeyword('a'): 1, HyKeyword('b'): 2}[HyKeyword('a')]

1

I think we could remove that special case from the compiler without losing the functionality if keywords were properly callables.

It seems weird that the above works, but not as higher-order functions and not with symbols.

I'm not sure how useful keyword-keyed dicts are in Hy anyway. Python nearly always uses strings for this kind of thing. Maybe it's a bit easier to type, but I'm not really seeing an advantage over using symbols, which count as strings anyway.

[Kodiologist]

Man, I completely forgot that was implemented. I wonder if it's even documented.

[mknecht]

Two questions about keywords not being available in macros. (Seems to be part of this issue here, sorry, if wrong.)

Keywords as (invalid) strings are kind of weird. They're awkward in macros.

Why is that?

Also, the merged PR above does not provide keywords in macros, right?

[Kodiologist]

But keywords are available in macros. Do you just mean that macros can't have keyword arguments (in the Python sense of the word "keyword")? In that case, yes, #1581 doesn't change that as #1581 currently stands.

Keywords as (invalid) strings are kind of weird. They're awkward in macros.

Why is that?

This is no longer the case since #1543. What was annoying about it was that it meant keywords didn't evaluate to themselves (as in other Lisps), and so you had to distinguish between HyKeyword objects and the result of evaluating a keyword, which was a string distinguished from ordinary strings by its value rather than its type.