Freeze on processing text with some punctuation marks

[aiharisov]

Trying with this table test.txt
The process starts to consume a lot of processor time (near 25%) and RAM. RAM consumption increases over time, looks like memory leak.
WeasyPrint==48

[Tontyna]

No freeze here. 3 pages generated in a flash.

[liZe]

The process starts to consume a lot of processor time (near 25%) and RAM. RAM consumption increases over time, looks like memory leak.

If it never finishes, it's probably an infinite loop.

That's strange, I've received a personal mail from someone with quite the same problem, but I can't reproduce either. It includes tables too.

@aiharisov Could you please give your OS and the versions of Cairo and Pango you're using? It would be also useful to check that the number of pages is growing using the -v option when launching WeasyPrint.

[aiharisov]

new example in attachment test1.txt

for test i've been using this script:

import os

from weasyprint import HTML

with open(os.path.join('D:', 'test.txt'), 'r', encoding='utf-8') as f:
text = f.read()
pdf_file = HTML(string=text).write_pdf()
with open(os.path.join('D:', 'test.txt.pdf'), 'wb') as file:
file.write(pdf_file)

system: Windows 10 Pro x64
used gtk3-runtime-3.24.10-2019-08-05-ts-win64.exe
pip:
Package Version

---

allure-pytest 2.6.2
allure-python-commons 2.6.2
apipkg 1.5
atomicwrites 1.3.0
attrs 19.1.0
beautifulsoup4 4.7.1
cached-property 1.5.1
cairocffi 1.0.2
CairoSVG 2.4.0
certifi 2019.6.16
cffi 1.12.3
chardet 3.0.4
colorama 0.4.1
coverage 4.5.4
cssselect2 0.2.1
Cython 0.29.10
defusedxml 0.6.0
Django 2.1.5
django-cors-headers 2.4.0
djangorestframework 3.9.2
dnspython 1.15.0
execnet 1.7.0
gevent 1.4.0
greenlet 0.4.15
gunicorn 19.9.0
html5lib 1.0.1
idna 2.8
importlib-metadata 0.19
Jinja2 2.10
jsonschema 3.0.1
MarkupSafe 1.1.1
more-itertools 7.2.0
numpy 1.16.0
pandas 0.23.4
pep8 1.7.1
Pillow 5.2.0
pip 19.2.2
pluggy 0.12.0
psycopg2-binary 2.7.6.1
py 1.8.0
pycparser 2.19
pydevd 1.5.1
PyPDF2 1.26.0
Pyphen 0.9.5
pyrsistent 0.15.4
pytest 3.10.1
pytest-cache 1.0
pytest-cov 2.6.0
pytest-django 3.4.4
pytest-pep8 1.0.6
python-dateutil 2.8.0
pytz 2018.9
requests 2.21.0
setuptools 40.6.2
six 1.12.0
soupsieve 1.9.3
tinycss2 1.0.2
urllib3 1.24.3
WeasyPrint 48
webencodings 0.5.1
XlsxWriter 1.1.8
zipp 0.5.2

[Tontyna]

With test1.txt I can reproduce the freeze. Will dive into it tonight...

@aiharisov to create a pdf you don't need to open/write the files yourself. Let WeasyPrint do that for you:

import os
from weasyprint import HTML

input_filename = os.path.join('D:', 'test.txt')
output_filename = os.path.join('D:', 'test.txt.pdf')

HTML(input_filename).write_pdf(output_filename)

[Tontyna]

It's not the table but the strange cell content:

<p>
Adverse drug experience 
[see <u>
<a href="">21 CFR Part 314.80 (a)</a>
<idrac> (IDRAC 8891)</idrac></u> ]: 
</p>

If this funny paragraph is rendered with a certain width the line iterator iter_line_boxes(), called in blocks.block_container_layout, never finishes yielding.

The surrounding table just generates the certain width that triggers the inifinite loop.

Minimal example to freeze WeasyPrint:

<p style="width:130px">
<span>
<span>xxxxxx YYY yyyyyy yyy</span>
<span>ZZZZZZ zzzzz
</span></span>)x
</p>

Of course, the special width value depends on the font size.

The three spans are required. It's also required that the block ends with </span></span>)x -- no space between </span> and </span> , followed by a bracket and another letter.

@liZe as always: I'm able to encircle the root of evil in the layout source but cannot fix it. It's probably another split_first_line jumping back and forth, unable to decide where to cut, cf. #660 (comment)

[liZe]

@liZe as always: I'm able to encircle the root of evil in the layout source but cannot fix it. It's probably another split_first_line jumping back and forth, unable to decide where to cut, cf. #660 (comment)

Reading test_breaking_linebox_regression_* makes me sad, but it's way better than reading split_first_line. I know I have to rewrite text.py when I add rtl and bidi support.

[Tontyna]

More lurking dragons inside innocent nested spans followed by bracket-and-letter. Slight modifications generate

IndexError list index out of range in skip_first_whitespace()

<p  style="width:130px">
<span>
<span>xxxxxx YYY yyyyyy yyy</span>
ZZZZZZ zzzzz
</span> )x 
</p>

That's the prime number bug #783, resurrected. Not joking. The skip stack is (0, (0, (13, None)))

AssertionError assert next_skip_stack is None in skip_first_whitespace()

<p style="width:130px">
<span>
xxxxxx YYY yyyyyy yyy
<span>ZZZZZZ zzzzz
</span></span>)x: 
</p>

Thats prime number bug again, disguised.

The prime number bug was resolved by introducing same_broken_child in split_inline_box.

This bug requires something similar, named is_this_child_a_box_we_already_failed_to_break, but I don't know how to implement that.

Fact is: If I set broken_child = True in split_inline_box()

WeasyPrint/weasyprint/layout/inlines.py

Line 863 in b8c19a2

broken_child = same_broken_child(

then neither infinite loop nor IndexError nor AssertionError is raised when the certain width hits the nested-spans-followed-by-bracket-etc