Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

The lexer needs to be more informative #657

Closed
rcarmo opened this issue Sep 9, 2014 · 2 comments
Closed

The lexer needs to be more informative #657

rcarmo opened this issue Sep 9, 2014 · 2 comments
Labels
complaint / disgust

Comments

@rcarmo
Copy link
Contributor

rcarmo commented Sep 9, 2014

I've just spent a long while trying to debug the following situation:

  • a.hy had a single-quoted string (yeah, I know...)
  • b.hy imported a

When trying to run b.hy, I got:

LexException: Could not identify the next token.

...without a single clue as to the file or line the error was actually in. So a little more information might save a lot of time...
@rcarmo
Copy link
Contributor Author

rcarmo commented Oct 8, 2014

I've since managed to reproduce this in another condition:

(defn [foo] (+1 foo))
"

That's right, a "lost" quote near EOF will completely screw up the lexer. Fiddling with LexException a bit and removing the guard for positive line numbers, I got this:

  File "/Users/rcarmo/.pyenv/versions/2.7.8/lib/python2.7/site-packages/hy/lex/__init__.py", line 37, in tokenize
    pos.lineno, pos.colno)
hy.lex.exceptions.LexException:   File "main.hy", line -1, column -1

  "
  ^
LexException: Could not identify the next token.

So it is possible to provide a little more info (and at least point to the offending character). I'm at a loss as to why the line and column are set to negative values here, but I haven't looked at the tokenizer properly yet -- it seems to be a bug in rply, actually, given that getsourcepos() is the one setting those values.

@tx46
Copy link

tx46 commented Aug 20, 2017

This needs to be fixed as it is still an issue. I did some manual "binary search" by deleting chunks of code in a source file until finding the issue, but that is not always viable either. How about at least outputting the file, row and column of the previous token!?

brandonwillard added a commit to brandonwillard/hy that referenced this issue Oct 10, 2018
@brandonwillard
Generate more informative syntax errors
2dc416f 
Closes hylang#657
brandonwillard added a commit to brandonwillard/hy that referenced this issue Oct 11, 2018
@brandonwillard
Generate more informative syntax errors
5ca669d 
This commit refactors the exception/error classes and their handling, keeps Hy
source string and their originating file (if any) information closer to the
origin of an exception (so that calling code isn't responsible for annotating
exceptions), and provides minimally intrusive traceback print-out filtering via
a context manager that temporarily alters `sys.excepthook`.

It also provides an environment variable, `HY_COLORED_ERRORS`, and package
variable, `hy.errors.__colored_errors`, that enables/disables manual error
coloring.

Closes hylang#657, closes hylang#1510, closes hylang#1429.
@brandonwillard brandonwillard mentioned this issue Oct 11, 2018
Merged
5 tasks
brandonwillard added a commit to brandonwillard/hy that referenced this issue Oct 11, 2018
@brandonwillard
Generate more informative syntax errors
1b796e5 
This commit refactors the exception/error classes and their handling, keeps Hy
source strings and their originating file (if any) information closer to the
origin of an exception (so that calling code isn't responsible for annotating
exceptions), and provides minimally intrusive traceback print-out filtering via
a context manager that temporarily alters `sys.excepthook` (enabled by default
for the Hy interpreter).

It also provides an environment variable, `HY_COLORED_ERRORS`, and package
variable, `hy.errors.__colored_errors`, that enables/disables manual error
coloring.

Closes hylang#657, closes hylang#1510, closes hylang#1429.
brandonwillard added a commit to brandonwillard/hy that referenced this issue Oct 11, 2018
@brandonwillard
Generate more informative syntax errors
79c54e1 
This commit refactors the exception/error classes and their handling, keeps Hy
source strings and their originating file information (if any) closer to the
origin of an exception (so that calling code isn't responsible for annotating
exceptions), and provides minimally intrusive traceback print-out filtering via
a context manager that temporarily alters `sys.excepthook` (enabled by default
for the Hy interpreter).

It also provides an environment variable, `HY_COLORED_ERRORS`, and package
variable, `hy.errors.__colored_errors`, that enables/disables manual error
coloring.

Closes hylang#657, closes hylang#1510, closes hylang#1429.
brandonwillard added a commit to brandonwillard/hy that referenced this issue Oct 13, 2018
@brandonwillard
Generate more informative syntax errors
d817bc2 
This commit refactors the exception/error classes and their handling, keeps Hy
source strings and their originating file information (if any) closer to the
origin of an exception (so that calling code isn't responsible for annotating
exceptions), and provides minimally intrusive traceback print-out filtering via
a context manager that temporarily alters `sys.excepthook` (enabled by default
for the Hy interpreter).

It also provides an environment variable, `HY_COLORED_ERRORS`, and package
variable, `hy.errors.__colored_errors`, that enables/disables manual error
coloring.

Closes hylang#657, closes hylang#1510, closes hylang#1429.
brandonwillard added a commit to brandonwillard/hy that referenced this issue Oct 13, 2018
@brandonwillard
Generate more informative syntax errors
bc8bb0b 
This commit refactors the exception/error classes and their handling, keeps Hy
source strings and their originating file information (if any) closer to the
origin of an exception (so that calling code isn't responsible for annotating
exceptions), and provides minimally intrusive traceback print-out filtering via
a context manager that temporarily alters `sys.excepthook` (enabled by default
for the Hy interpreter).

It also provides an environment variable, `HY_COLORED_ERRORS`, and package
variable, `hy.errors.__colored_errors`, that enables/disables manual error
coloring.

Closes hylang#657, closes hylang#1510, closes hylang#1429.
brandonwillard added a commit to brandonwillard/hy that referenced this issue Oct 14, 2018
@brandonwillard
Generate more informative syntax errors
8ead992 
This commit refactors the exception/error classes and their handling, keeps Hy
source strings and their originating file information (if any) closer to the
origin of an exception (so that calling code isn't responsible for annotating
exceptions), and provides minimally intrusive traceback print-out filtering via
a context manager that temporarily alters `sys.excepthook` (enabled by default
for the Hy interpreter).

It also provides an environment variable, `HY_COLORED_ERRORS`, and package
variable, `hy.errors.__colored_errors`, that enables/disables manual error
coloring.

Closes hylang#657, closes hylang#1510, closes hylang#1429.
brandonwillard added a commit to brandonwillard/hy that referenced this issue Oct 17, 2018
@brandonwillard
Generate more informative syntax errors
f39c325 
This commit refactors the exception/error classes and their handling, keeps Hy
source strings and their originating file information (if any) closer to the
origin of an exception (so that calling code isn't responsible for annotating
exceptions), and provides minimally intrusive traceback print-out filtering via
a context manager that temporarily alters `sys.excepthook` (enabled by default
for the Hy interpreter).

It also provides an environment variable, `HY_COLORED_ERRORS`, and package
variable, `hy.errors.__colored_errors`, that enables/disables manual error
coloring.

Closes hylang#657, closes hylang#1510, closes hylang#1429.
brandonwillard added a commit to brandonwillard/hy that referenced this issue Oct 28, 2018
@brandonwillard
Generate more informative syntax errors
b550672 
This commit refactors the exception/error classes and their handling, keeps Hy
source strings and their originating file information (if any) closer to the
origin of an exception (so that calling code isn't responsible for annotating
exceptions), and provides minimally intrusive traceback print-out filtering via
a context manager that temporarily alters `sys.excepthook` (enabled by default
for the Hy interpreter).

It also provides an environment variable, `HY_COLORED_ERRORS`, and package
variable, `hy.errors.__colored_errors`, that enables/disables manual error
coloring.

Closes hylang#657, closes hylang#1510, closes hylang#1429.
brandonwillard added a commit to brandonwillard/hy that referenced this issue Oct 29, 2018
@brandonwillard
Retain source and stack information for exceptions
b624ccc 
This commit refactors the exception/error classes and their handling.
It also retains Hy source strings and their originating file information, when
available, all throughout the core parser and compiler functions.

As well, with these changes, calling code is no longer responsible for providing
source and file details to exceptions,

Closes hylang#657.
brandonwillard added a commit to brandonwillard/hy that referenced this issue Oct 29, 2018
@brandonwillard
Retain compiled source and file information for exceptions
fc6b94e 
This commit refactors the exception/error classes and their handling.
It also retains Hy source strings and their originating file information, when
available, all throughout the core parser and compiler functions.

As well, with these changes, calling code is no longer responsible for providing
source and file details to exceptions,

Closes hylang#657.
brandonwillard added a commit to brandonwillard/hy that referenced this issue Oct 30, 2018
@brandonwillard
Retain compiled source and file information for exceptions
67b839b 
This commit refactors the exception/error classes and their handling.
It also retains Hy source strings and their originating file information, when
available, all throughout the core parser and compiler functions.

As well, with these changes, calling code is no longer responsible for providing
source and file details to exceptions,

Closes hylang#657.
brandonwillard added a commit to brandonwillard/hy that referenced this issue Nov 10, 2018
@brandonwillard
Retain compiled source and file information for exceptions
7b5e30a 
This commit refactors the exception/error classes and their handling.
It also retains Hy source strings and their originating file information, when
available, all throughout the core parser and compiler functions.

As well, with these changes, calling code is no longer responsible for providing
source and file details to exceptions,

Closes hylang#657.
brandonwillard added a commit to brandonwillard/hy that referenced this issue Nov 28, 2018
@brandonwillard
Retain compiled source and file information for exceptions
c7de22e 
This commit refactors the exception/error classes and their handling.
It also retains Hy source strings and their originating file information, when
available, all throughout the core parser and compiler functions.

As well, with these changes, calling code is no longer responsible for providing
source and file details to exceptions,

Closes hylang#657.
brandonwillard added a commit to brandonwillard/hy that referenced this issue Nov 29, 2018
@brandonwillard
Retain compiled source and file information for exceptions
e965fd0 
This commit refactors the exception/error classes and their handling.
It also retains Hy source strings and their originating file information, when
available, all throughout the core parser and compiler functions.

As well, with these changes, calling code is no longer responsible for providing
source and file details to exceptions,

Closes hylang#657.
brandonwillard added a commit to brandonwillard/hy that referenced this issue Dec 1, 2018
@brandonwillard
Retain compiled source and file information for exceptions
a334115 
This commit refactors the exception/error classes and their handling.
It also retains Hy source strings and their originating file information, when
available, all throughout the core parser and compiler functions.

As well, with these changes, calling code is no longer responsible for providing
source and file details to exceptions,

Closes hylang#657.
brandonwillard added a commit to brandonwillard/hy that referenced this issue Dec 2, 2018
@brandonwillard
Retain compiled source and file information for exceptions
0adcf6e 
This commit refactors the exception/error classes and their handling.
It also retains Hy source strings and their originating file information, when
available, all throughout the core parser and compiler functions.

As well, with these changes, calling code is no longer responsible for providing
source and file details to exceptions,

Closes hylang#657.
brandonwillard added a commit to brandonwillard/hy that referenced this issue Dec 7, 2018
@brandonwillard
Retain compiled source and file information for exceptions
f9e9fd8 
This commit refactors the exception/error classes and their handling.
It also retains Hy source strings and their originating file information, when
available, all throughout the core parser and compiler functions.

As well, with these changes, calling code is no longer responsible for providing
source and file details to exceptions,

Closes hylang#657.
Kodiologist pushed a commit to brandonwillard/hy that referenced this issue Dec 8, 2018
@brandonwillard @Kodiologist
Retain compiled source and file information for exceptions
6d912ca 
This commit refactors the exception/error classes and their handling.
It also retains Hy source strings and their originating file information, when
available, all throughout the core parser and compiler functions.

As well, with these changes, calling code is no longer responsible for providing
source and file details to exceptions,

Closes hylang#657.
Kodiologist pushed a commit to brandonwillard/hy that referenced this issue Dec 12, 2018
@brandonwillard @Kodiologist
Retain compiled source and file information for exceptions
0f10d86 
This commit refactors the exception/error classes and their handling.
It also retains Hy source strings and their originating file information, when
available, all throughout the core parser and compiler functions.

As well, with these changes, calling code is no longer responsible for providing
source and file details to exceptions,

Closes hylang#657.
brandonwillard added a commit to brandonwillard/hy that referenced this issue Dec 17, 2018
@brandonwillard
Retain compiled source and file information for exceptions
f95df70 
This commit refactors the exception/error classes and their handling.
It also retains Hy source strings and their originating file information, when
available, all throughout the core parser and compiler functions.

As well, with these changes, calling code is no longer responsible for providing
source and file details to exceptions,

Closes hylang#657.
brandonwillard added a commit to brandonwillard/hy that referenced this issue Jan 17, 2019
@brandonwillard
Retain compiled source and file information for exceptions
a674ca3 
This commit refactors the exception/error classes and their handling.
It also retains Hy source strings and their originating file information, when
available, all throughout the core parser and compiler functions.

As well, with these changes, calling code is no longer responsible for providing
source and file details to exceptions,

Closes hylang#657.
Kodiologist pushed a commit to Kodiologist/hy that referenced this issue Jan 20, 2019
@brandonwillard @Kodiologist
Retain compiled source and file information for exceptions
cb11995 
This commit refactors the exception/error classes and their handling.
It also retains Hy source strings and their originating file information, when
available, all throughout the core parser and compiler functions.

As well, with these changes, calling code is no longer responsible for providing
source and file details to exceptions,

Closes hylang#657.
Kodiologist pushed a commit to Kodiologist/hy that referenced this issue Jan 26, 2019
@brandonwillard @Kodiologist
Retain compiled source and file information for exceptions
10691c7 
This commit refactors the exception/error classes and their handling.
It also retains Hy source strings and their originating file information, when
available, all throughout the core parser and compiler functions.

As well, with these changes, calling code is no longer responsible for providing
source and file details to exceptions,

Closes hylang#657.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
complaint / disgust
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@rcarmo @Foxboron @tx46