Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[parsing] is "++" an identifier or not ? #42076

Closed
o314 opened this issue Sep 1, 2021 · 7 comments · Fixed by #42266
Closed

[parsing] is "++" an identifier or not ? #42076

o314 opened this issue Sep 1, 2021 · 7 comments · Fixed by #42266
Labels
docs This change adds or pertains to documentation

Comments

@o314
Copy link
Contributor

o314 commented Sep 1, 2021

are "++", ".." valid identifiers or not ?

FOR "++"

using Test

# pass @ v1.6
@test Base.isidentifier("++") == false  # it should not be
@test (++ = 1) == 1                     # but it can be assigned,
@test let ++ = 1 ; 2 * ++ end == 2      # be doubled,
@test let ++ = 1 ; 1 + ++ end == 2      # be incremented at right,

# do not pass @ v1.6
@test let ++ = 1 ; ++ + 1 end == 2      # but not be incremented at left # will syntaxically throws

FOR ".."

@test Base.isidentifier("..") == false  # it should not be
@test (.. = 1) == 1                     # but it can be assigned,
@test let .. = 1 ; 2 * .. end == 2      # be doubled,
@test let .. = 1 ; 1 + .. end == 2      # be incremented at right,


# do not pass @ v1.6
@test let .. = 1 ; .. + 1 end == 2      # but not be incremented at left # will syntaxically throws

BTW, "--" seems to throws on assign coherently with

@test Base.isidentifier("--") == false

Edited: add the ".." case

@mcabbott
Copy link
Contributor

mcabbott commented Sep 2, 2021

The same tests pass for ^, -, &, %, ⊗ and more. They are all infix operators, but you can still assign them meanings.

Depending on precedence, sometime the parsing relative to other infix operators will not be the same as for identifiers: :(^ + 1) is ^(+1) which is illegal, "^" is not a unary operator, but :((^) + 1) is fine. That's your last test.

@stevengj
Copy link
Member

stevengj commented Sep 3, 2021

isidentifier returns false for infix/prefix/postfix operators, which are a very special form of identifier that can only be used in certain contexts.

@o314
Copy link
Contributor Author

o314 commented Sep 13, 2021

Thanks for your answers and sorry for the delay in mine, i currently works on two jobs those days. It's sometime tought to catch everything on time.

isidentifier doc needs an update

isidentifier returns false for infix/prefix/postfix operators, which are a very special form of identifier that can only be used in certain contexts.

One may need to update doc for Base.isidentifier. Here is the doc for of v1.6

help?> Base.isidentifier
isidentifier(s) -> Bool

Return whether the symbol or string s contains characters that are parsed as a valid identifier in Julia code.

Internally Julia allows any sequence of characters in a Symbol (except \0s), and macros automatically use variable names containing # in order to avoid naming collision with the surrounding code.
In order for the parser to recognize a variable, it uses a limited set of characters (greatly extended by Unicode). isidentifier() makes it possible to query the parser directly whether a symbol
contains valid characters.

Examples
≡≡≡≡≡≡≡≡≡≡

julia> Meta.isidentifier(:x), Meta.isidentifier("1x")
(true, false)

-- and ++ need to be not so different

One may be clearer about the diff between -- and ++. -- is even not parsable. Why such a difference ?

((((an op is a function) maybe a special form) and a literal) barely documented) needs to be a bit more shaved

Dismissed considering #42076 (comment)

Assigning operator eg. allowing both to be literal and function is IMHO an hazardous choice (one can even add a third way: special form, too much of a good thing)

In Julia, most operators are just functions with support for special syntax. (The exceptions are operators with special evaluation semantics like && and ||. These operators cannot be functions since Short-Circuit Evaluation requires that their operands are not evaluated before evaluation of the operator.) Accordingly, you can also apply them using parenthesized argument lists, just as you would any other function:
https://docs.julialang.org/en/v1/manual/functions/#Operators-Are-Functions

Operators are named by construction, they should behave like method, not lambda as in [method vs lambda](https://gist.github.com/StefanKarpinski/4509b6499f0c76446a4ba4f0f7bf1354)

It may seem great to allow more and more choices, but without real value added, it's mainly more complexity and hard to fix bug.

using Test
+ = 1                           # is it a feature or a bug ?
@test_throws MethodError 1 + 1 # ugly. callable is not what you're looking for.

# ---
f(x,y) = x+y
@test_throws ErrorException (f = 1) # reassignment is forbidden

@stevengj stevengj added the docs This change adds or pertains to documentation label Sep 13, 2021
@stevengj
Copy link
Member

stevengj commented Sep 15, 2021

Note that Julia is not accepting fundamental language changes anymore, so we aren't really going to debate fundamental features like the ability to assign operators to arbitrary expressions.

Reassignment of a constant is forbidden in the module that defines it (as in the case of your f above — functions are constants). However, you can re-assign names that were imported from other modules — otherwise, if I have a variable named foo, my code would suddenly become invalid if another module I'm using exports a function foo. That's why you're allowed to assign a new "shadow" definition of + in the code you quoted above — you are defining a new + meaning in the current scope, you aren't changing the existing + as it is used in other modules. If you're confused by this, you can ask more about it on discourse.

@stevengj
Copy link
Member

(I added the "doc" tag to this issue because it is really about improving the documentation for isidentifier.)

@o314
Copy link
Contributor Author

o314 commented Sep 16, 2021

You 're right, thanks for pointing out correctly this case

using Test
@test (+ = 1; +) == 1
@test (sin = 1; sin) == 1

@test_throws ErrorException Base.sin = 1
@test_throws ErrorException Base.:(+) = 1

It's a feature, not an error :)

@o314
Copy link
Contributor Author

o314 commented Sep 17, 2021

Thanks,

Considering the boundary and similarity between id and op, I remain a bit puzzled by some changes at https://github.com/JuliaLang/julia/blob/v1.6.2/test/syntax.jl#L1945 introduced by #33158

@test Meta.parse("import Base.Foo.:(==).bar") == :(import Base.Foo.==.bar)

IMHO operator should be atomical eg. do not be able to handle a getfield request

May also concern #37583 afterwards isoperator may accept more than symbol

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
docs This change adds or pertains to documentation
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants