Python lexer broken for raw strings in v3.18.0

[tuxu]

Name of the lexer
Python

Code sample

The two below examples show that escaped characters in raw strings confuse the lexer.

import re
s = "Hello World"
m = re.findall(r"^[\w]+", s)
print(m)

(rouge.jneen.net)

devid_line = status_line.split('STATUS:')[0]
result_line = status_line.split('STATUS:')[1]
print(f'Parsing "{result_line}"...')

# parse devid string
pattern = re.compile(r'.*\[p00t_(d\d{4})\].*')
data = pattern.match(devid_line)
devid = data.group(1)

(rouge.jneen.net)

Additional context

There is an issue in the GitLab issue tracker which describes the same problem:
https://gitlab.com/gitlab-org/gitlab/-/issues/215643

I traced the problem back to commit 652a622: Use generic string states in Python lexer (#1477)

[pyrmont]

@tuxu Thanks for the detailed report and sorry for the trouble :( Will have a look at that now.

[Spindel]

It's not only raw strings, this also seems to set it off:

var = "\w+"
foo = "bar"

[pyrmont]

@Spindel Thanks for the report! The code in the PR will fix this too :)