Zero-width characters cause unbounded memory use in PyREPL

[trag1c]

Bug report

Bug description:

Trying to paste zero-width characters (U+200B, U+200C, U+200D, U+FEFF) into the new REPL causes it to get stuck and keep allocating memory. Python 3.12 handles them fine.

This seems to happen during screen calculation, so the amount of characters needed to reproduce will depend on the number of cells in a terminal emulator window (I got it working with as few as 3 with a big enough zoom, but 100 should be enough).

A table with all of them x100 to copy for convenience:

ZWSP	ZWNJ	ZWJ	ZWNBSP
​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​	‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌‌	‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍	

To reproduce

1. Open the new REPL
2. Paste in some zero-width characters
3. The REPL should get stuck and keep allocating memory
4. Can be stopped with double ^C or a single ^\

The culprit seems to be this loop in Lib/_pyrepl/reader.py:Reader.calc_screen and its screen/screen_info appends:

cpython/Lib/_pyrepl/reader.py

Lines 388 to 411 in f4e5643

	while l:
	prelen = lp if i == 0 else 0
	index_to_wrap_before = 0
	column = 0
	for character_width in l2:
	if column + character_width >= self.console.width - prelen:
	break
	index_to_wrap_before += 1
	column += character_width
	pre = prompt if i == 0 else ""
	if len(l) > index_to_wrap_before:
	offset += index_to_wrap_before
	post = "\\"
	after = [1]
	else:
	offset += index_to_wrap_before + 1 # Takes the newline
	post = ""
	after = []
	last_refresh_line_end_offsets.append(offset)
	screen.append(pre + l[:index_to_wrap_before] + post)
	screeninfo.append((prelen, l2[:index_to_wrap_before] + after))
	l = l[index_to_wrap_before:]
	l2 = l2[index_to_wrap_before:]
	i += 1

An additional demo

Screen.Recording.2024-11-11.at.13.52.32.mov

CPython versions tested on:

3.12, 3.13, CPython main branch

Operating systems tested on:

Linux, macOS

[bswck]

Excellent find. Needs a backport to 3.13. Gonna fix soon.