Wrong locations in semantic highlighting when using preprocessors

[jboillot]

Hello!

I was using ocaml-lsp with cppo, a C-style preprocessor for OCaml while I noticed the semantic highlighting in files containing cppo code was broken:

After some investigation in the ocaml-lsp code I found this PR #932 that corrected the function lident

ocaml-lsp/ocaml-lsp-server/src/semantic_highlighting.ml

Line 354 in 45b3198

let lident ({ loc; _ } : Longident.t Loc.loc) rightmost_name

that parses long identifiers.
In order to remember spaces that may appear in long identifiers, it ignores the long identifier object given as a parameter (that already has spaces trimmed) and instead reads again the source M.source using the function source_excerpt

ocaml-lsp/ocaml-lsp-server/src/semantic_highlighting.ml

Line 341 in 45b3198

let source_excerpt ({ loc_start; loc_end; _ } : Loc.t) =

.

The problem is that, while using preprocessor directives, the location of the long identifier given as an argument is invalid (the location is specific to the code already preprocessed). Then the semantic highlighting returned is invalid.

I have tested to add to the lident function some logs after computing the source excerpt:

let lid = source_excerpt loc in
let lid2 = join (Longident.flatten lid2) in
debug (Printf.sprintf "I read \"%s\" while I could have read \"%s\"" lid lid2);

And here is an extract of the result:

I read "ec" while I could have read "()"
I read "e you" while I could have read "false"
I read "re" while I could have read "&&"
I read "warne" while I could have read "debug"
I read " i" while I could have read "()"

I am not sure what could be acceptable to do. Would it be possible to keep track of the source location? Or to keep the input text in the long identifier structure?

Thank you for reading!

PS: Maybe you have some clues @ulugbekna since you did the PR? :)

[jboillot]

Fixed by #1049