clean up rules for ?

[gilch]

Hy converts a trailing ? to an is_ prefix. This means that you can use a more lispy style for common predicate methods:

=> (.integer? 1.0)
1.0.is_integer()
True
=> (.integer? 1.5)
1.5.is_integer()
False

But due to an unfortunate inconsistency in Python's naming conventions, it doesn't always work:

=> (.lower? "foo")
'foo'.is_lower()
Traceback (most recent call last):
  File "<input>", line 1, in <module>
AttributeError: 'str' object has no attribute 'is_lower'
=> (.islower "foo")
'foo'.islower()
True

There's no underscore in islower! This appears to be the case for all string predicate methods. It may also be true elsewhere in the Python standard library, and therefore in other Python libraries.

We may be able to adjust the rules to make this more consistent.

option 1

Make a prefixed ? become a prefix is.

=> (.?lower "foo")
'foo'.islower()
True

This way, we can leave the suffix case (and any code using it) as-is_. But now there's three ways to write is_integer, ?-integer, and integer?.

option 2

Make a suffix ? become a prefix is and suffix -? become a prefix is_.

=> (.integer-? 1.0)
1.0.is_integer()
=> (.lower? "foo")
'foo'.islower()

This can break existing Hy code, but we're doing that anyway.

option 3

Remove the special treatment of ?. Use the same names in Hy as in Python. Hy becomes more consistent and less magical. Hy names ending in ? become as difficult to use in Python as Hy names with an embedded ? are now. But it's still possible with getattr.

We could probably come up with more options, but in any case, it needs to be documented. I didn't see the current ? rule in the docs either.

[refi64]

Wow I didn't even realize this was a thing. Personally, I prefer either option 1 or 2.

[gilch]

Yup, you can see all the conversions here. Note also that a ! suffix becomes a _bang suffix. Hy also uses punycode, but that's in the docs.

Automagic really should be documented.

Another trick--since it happens at the parser level, it is possible to keep the question mark:

=> 'query?
from hy import HySymbol
HySymbol('is_query')
'is_query'
=> (HySymbol "query?")
HySymbol('query?')
'query?'

This kind of automagic makes me uncomfortable (it causes headaches, like #1100, etc.), but it really does make common Hy method names easier to use from Python.

But if it's not a suffix, it doesn't count:

=> 'fo?o
from hy import HySymbol
HySymbol('fo?o')
'fo?o'

It's really awkward to use a method named like that in Python.

[gilch]

On second thought, the first two options are no good. Neither is the status quo.

Recall that underscore prefixes have meaning in Python.

_single_leading_underscore: weak "internal use" indicator. E.g. from M import * does not import objects whose name starts with an underscore.

-- From PEP-8

Any identifier of the form __spam (at least two leading underscores, at most one trailing underscore) is textually replaced with _classname__spam, where classname is the current class name with leading underscore(s) stripped. This mangling is done without regard to the syntactic position of the identifier, so it can be used to define class-private instance and class variables, methods, variables stored in globals, and even variables stored in instances. private to this class on instances of other classes.

-- Python docs

Hy behaves the same way, except when the identifier ends with ?! This could definitely confuse users for no good reason. Why? Because the mangling adds an is_ prefix, and it's therefore no longer a _ prefix.

This inconsistency is unacceptable.

Option 3 still works.

Option 4

Take the Canadian approach and use an _eh suffix instead.

Option 5

Take Clojure's approach and replace all ? with _QMARK_. We could also do this kind of thing for other characters.

[Kodiologist]

You're right that that inconsistency is no good, but it can also be addressed by altering mangling rules as follows: the "generated" is or is_ is put after any leading underscores.

[gilch]

On third thought, _QMARK_ and friends could also introduce a leading underscore where none was intended, if the symbol begins with a special character. We could special case that, so ?foo? becomes QMARK_foo_QMARK_ and not _QMARK_foo_QMARK_. You'd also have to take care that _?foo retains exactly one leading underscore, and __?foo has at least two.