StackOverflowError with long base64 image and LINKS_ALLOW_MATCHED_PARENTHESES disabled

[mrombout]

Describe the bug

Most likely the same, or similar issue as #364.

Using a Parser with LINKS_ALLOW_MATCHED_PARENTHESES = false.

When parsing a reasonably sized base64 image in markdown the parser has trouble matching and ends up with a StackOverflowError. Decreasing the size of the base64 image will eventually allow the regex to cope.

I found this because for the time being, I am using PegdownOptionsAdapter, which has that option enabled by default.

To Reproduce

Running:

openjdk version "11.0.5" 2019-10-15
OpenJDK Runtime Environment (build 11.0.5+10-post-Ubuntu-0ubuntu1.118.04)
OpenJDK 64-Bit Server VM (build 11.0.5+10-post-Ubuntu-0ubuntu1.118.04, mixed mode, sharing)

See https://github.com/mrombout/flexmark-so-sscce, or below.

MutableDataSet options = new MutableDataSet().set(Parser.LINKS_ALLOW_MATCHED_PARENTHESES, false);

Parser parser = Parser.builder(options).build();
HtmlRenderer renderer = HtmlRenderer.builder(options).build();

Document parse = parser.parse("![](data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAEAAAABACAYAAACqaXHeAAAHsklEQVR42u1bC1BUVRi+ZGKapfaUpgzHxrGcAlr23gVB0MiATMkHmqmgxpIoyt27PESRVTFJhUVFXkoQhpPmA/M9ZjKVmpqmNtNjcprGnB7WFDNWM5VxOv/lHl1pd++5955dhPHOfMMwe+855/vP/zrn/Ifjbj23nk7zmM3mUIvFEuuKLk04NDS0LyadyvN8MwbygrOCIGTB+12GPCblwGhRId4e8H7Szccm2zSIE83JnI13YLzJicI+/LcZ4yQnCYfw3x349zLOZp4dlGxOw7N+XiPxG9DxpjFxYjdO4p/jbEINJ/LfYYKIBr3SeWSKdD+ziv0Hg7pTCKGpY4hnYRuU+HxM5qIrsTvzolGYcwJ6vi4NzdpuR9K+xSj/0FLkeP81lHdwCZq3eyGaUGVFQpTFLaFHx/IXZIHiRxGCqmn4l7hjaCAeYA4m+xshfW/BSJTU8Cpa2lyMGs5vQJs+2+gVieMTPJLpkaEIU+KPcXaLoPiHm0QAIh+NB/c5IT64eAyy7y9EDSqEXSGVzvdIZGT8CDR+UwbqvWA40ah/A+cIdfi3ix1rAmDnNr4Q4yoMrL/jWZRzwEFNmmDjiQoUFRPlUQCT05Pl9zacqUSj66zodrulzazS+Eu8wF9x9w34C9+Sz4jtjQdxAAYSIAloTH06qjtXo5k8YG7RbK+qDL+7vl/84So0cHlim9O08n+FRAlfuOYDviefGXa/HL7wAO7KH65r1mlnH1C8ben/vqs7W43ialKJSbTiECr5x95zTX1wrD4PHT9QGIdWHy3VTR6QX5WjGtNBSJ6+n/L2PBRAoo3IZ/ve04v8YejsQUccKj+51hB5Nc8PeCZhpGob1p05RAitnGR+2XcCsPHrgPw9BSNQ6TGnYfKle19XnX3iANWQ+o6NmMOfnF14ij15yTwOOgjMjpTjulHygLT8maoCaO8AvSG2ajrJFb6UnTTT7M7GX4bGQdIsyANAvdUE4M4BekLtp1XokWXxRBNKWKp+BTQ6ZOVYTcmNN6w/4qRa1HhzgO6wDGvnbTgs4/H+w8YUbPxgSHS64eRjxQcrmc2+t8yPAMKjnrbjNswgUWEPC9t/AxqLwfZlhDCoMtgzAY36wzuu3wBAc9T6qjhVjnrkRLZFhSzL0wbW8Ob+uJG/YfZLjhmL9y9OG4uMrO+1mkR87SziCxqNqL8NGjGvnWSIPAyaBXnIGWj7dB53ktzgDy5TuFuvAM6AAGx47e7rbI8G4De09Pv4yiTFFwgzdIQ+UxB8fEfuMN2LHCPxX8tse8LM7XZiBlv1rPGnwMchpeOZkqf1ByAko/04j5cRAfyMGQVo9P7Cevh48uZM5gKgUXcwGxZ9wa5UW3ZoGqJVA+RFT/aBQqbkIRzSCIAm3NEgpHSCogWWMVod4CX4kMWix18JkEo4zNUqAHmbq56xA0zJmspsBUgD2C/QvjYQI3qSlR9r+6fJALWsAKkjgWiuphdAXlQ/spfPkjxtQqRlBaiG2bvydGSEbRuecg7AUgBLGhZRCYBln69czwUatfqAVviwgeFg1HZ/WSVArpi6JUsRgLlcqwB+gg/XnljT6RIgV7xQbyU+YJFGAQgfwYdwfsdqMGrb3ywTIAJTWTLRgElaNaAWPkzZKjIZCM0GKMsEiCBoySjiA0K0CiAdPhTWTWYyEJoVIcsECFD5STlZEv/OWU3dNabCEY+Rk5+GTpgAAea8u4Ash/fp3Q/4BhooeK/I5wcgetb7aghfk0y2yUW9AiiCBkZUp/glAQI/wVL9lVPkq/LehoEd4VZIiGCj0ZcJEGv7n7R5LnF++41ui++FhuD425cJEEv7rz1bhfosjFHUPzzB4KlQ+DBoqGdulO7DUJoEiGX8T268NvunWJ0M7YYGI8un+GwHSOsJkCeUfVxGzgTA+49iIwA7P1A+ddWxQ0yTALHK/yFcP1kyjnj+LYyPxgU7NAyFSiBllgkQq/B3TfUl/hcuZ9hDrA/IA4hDDF6eiKpPVzBLgFioP9QdKoeieBUrJPqmQEIy3Yc7+BqE8MSqJLlOx+gOEAvvv/jwchRI7B5yF9/WBMop8g9ECDVnKgwlQEZ3f6Aescd18m9p3//XJQTzUNzZ99DpgKIE+QBCTwJEU/+jttvTTakZ5ES+Qa5b9F9NsCUYd/wVcYxQ66vV/vXGftC66Mqp14usJWG1f2ZeucAApejyxYS20pmdZCBhzonyqSyN/eud/YxdeajvolhC/gomP81vV1Xc3NpoDo8ID8NhJ1MeDB5U9+wIuUJj8bZCr7MP5qElvot7C9CgFaNdS+tPaT/uMkbeU3l6i6wN800D8KA2QyEzDDDoJcEjeUiNaYhDUQYUSUMNsgvxy9gHZfjV3tXu68BdnXYOsjE0mm91925EdCTK21Egk6s6vV4GrDFWHS1BuQcdaPpWEcVUTpOrUG+4TCFfusD9WE29OuLujlo4a26nMame3u2XQndzRMGvcmgTzfH+9fDaBdBCbmvBX/z/t+7ee3gcvwcTasKzeU7ZbQKCLRg/YlzARI/IR1gw05Jg6lDS7QRAc3OrCW5rebrPA1rBddaH5nqKN+3oEpcbMZF6Hff3HF3tImOSouqeSIPtN4G6dyni3jJCBcHcrafzPv8B13NkQkVaYmEAAAAASUVORK5CYII=)\n");
String render = renderer.render(parse);

System.out.println(render);

Expected behavior

Output an image tag with the base64 data.

<p><img src="data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAEAAAABACAYAAACqaXHeAAAHsklEQVR42u1bC1BUVRi+ZGKapfaUpgzHxrGcAlr23gVB0MiATMkHmqmgxpIoyt27PESRVTFJhUVFXkoQhpPmA/M9ZjKVmpqmNtNjcprGnB7WFDNWM5VxOv/lHl1pd++5955dhPHOfMMwe+855/vP/zrn/Ifjbj23nk7zmM3mUIvFEuuKLk04NDS0LyadyvN8MwbygrOCIGTB+12GPCblwGhRId4e8H7Szccm2zSIE83JnI13YLzJicI+/LcZ4yQnCYfw3x349zLOZp4dlGxOw7N+XiPxG9DxpjFxYjdO4p/jbEINJ/LfYYKIBr3SeWSKdD+ziv0Hg7pTCKGpY4hnYRuU+HxM5qIrsTvzolGYcwJ6vi4NzdpuR9K+xSj/0FLkeP81lHdwCZq3eyGaUGVFQpTFLaFHx/IXZIHiRxGCqmn4l7hjaCAeYA4m+xshfW/BSJTU8Cpa2lyMGs5vQJs+2+gVieMTPJLpkaEIU+KPcXaLoPiHm0QAIh+NB/c5IT64eAyy7y9EDSqEXSGVzvdIZGT8CDR+UwbqvWA40ah/A+cIdfi3ix1rAmDnNr4Q4yoMrL/jWZRzwEFNmmDjiQoUFRPlUQCT05Pl9zacqUSj66zodrulzazS+Eu8wF9x9w34C9+Sz4jtjQdxAAYSIAloTH06qjtXo5k8YG7RbK+qDL+7vl/84So0cHlim9O08n+FRAlfuOYDviefGXa/HL7wAO7KH65r1mlnH1C8ben/vqs7W43ialKJSbTiECr5x95zTX1wrD4PHT9QGIdWHy3VTR6QX5WjGtNBSJ6+n/L2PBRAoo3IZ/ve04v8YejsQUccKj+51hB5Nc8PeCZhpGob1p05RAitnGR+2XcCsPHrgPw9BSNQ6TGnYfKle19XnX3iANWQ+o6NmMOfnF14ij15yTwOOgjMjpTjulHygLT8maoCaO8AvSG2ajrJFb6UnTTT7M7GX4bGQdIsyANAvdUE4M4BekLtp1XokWXxRBNKWKp+BTQ6ZOVYTcmNN6w/4qRa1HhzgO6wDGvnbTgs4/H+w8YUbPxgSHS64eRjxQcrmc2+t8yPAMKjnrbjNswgUWEPC9t/AxqLwfZlhDCoMtgzAY36wzuu3wBAc9T6qjhVjnrkRLZFhSzL0wbW8Ob+uJG/YfZLjhmL9y9OG4uMrO+1mkR87SziCxqNqL8NGjGvnWSIPAyaBXnIGWj7dB53ktzgDy5TuFuvAM6AAGx47e7rbI8G4De09Pv4yiTFFwgzdIQ+UxB8fEfuMN2LHCPxX8tse8LM7XZiBlv1rPGnwMchpeOZkqf1ByAko/04j5cRAfyMGQVo9P7Cevh48uZM5gKgUXcwGxZ9wa5UW3ZoGqJVA+RFT/aBQqbkIRzSCIAm3NEgpHSCogWWMVod4CX4kMWix18JkEo4zNUqAHmbq56xA0zJmspsBUgD2C/QvjYQI3qSlR9r+6fJALWsAKkjgWiuphdAXlQ/spfPkjxtQqRlBaiG2bvydGSEbRuecg7AUgBLGhZRCYBln69czwUatfqAVviwgeFg1HZ/WSVArpi6JUsRgLlcqwB+gg/XnljT6RIgV7xQbyU+YJFGAQgfwYdwfsdqMGrb3ywTIAJTWTLRgElaNaAWPkzZKjIZCM0GKMsEiCBoySjiA0K0CiAdPhTWTWYyEJoVIcsECFD5STlZEv/OWU3dNabCEY+Rk5+GTpgAAea8u4Ash/fp3Q/4BhooeK/I5wcgetb7aghfk0y2yUW9AiiCBkZUp/glAQI/wVL9lVPkq/LehoEd4VZIiGCj0ZcJEGv7n7R5LnF++41ui++FhuD425cJEEv7rz1bhfosjFHUPzzB4KlQ+DBoqGdulO7DUJoEiGX8T268NvunWJ0M7YYGI8un+GwHSOsJkCeUfVxGzgTA+49iIwA7P1A+ddWxQ0yTALHK/yFcP1kyjnj+LYyPxgU7NAyFSiBllgkQq/B3TfUl/hcuZ9hDrA/IA4hDDF6eiKpPVzBLgFioP9QdKoeieBUrJPqmQEIy3Yc7+BqE8MSqJLlOx+gOEAvvv/jwchRI7B5yF9/WBMop8g9ECDVnKgwlQEZ3f6Aescd18m9p3//XJQTzUNzZ99DpgKIE+QBCTwJEU/+jttvTTakZ5ES+Qa5b9F9NsCUYd/wVcYxQ66vV/vXGftC66Mqp14usJWG1f2ZeucAApejyxYS20pmdZCBhzonyqSyN/eud/YxdeajvolhC/gomP81vV1Xc3NpoDo8ID8NhJ1MeDB5U9+wIuUJj8bZCr7MP5qElvot7C9CgFaNdS+tPaT/uMkbeU3l6i6wN800D8KA2QyEzDDDoJcEjeUiNaYhDUQYUSUMNsgvxy9gHZfjV3tXu68BdnXYOsjE0mm91925EdCTK21Egk6s6vV4GrDFWHS1BuQcdaPpWEcVUTpOrUG+4TCFfusD9WE29OuLujlo4a26nMame3u2XQndzRMGvcmgTzfH+9fDaBdBCbmvBX/z/t+7ee3gcvwcTasKzeU7ZbQKCLRg/YlzARI/IR1gw05Jg6lDS7QRAc3OrCW5rebrPA1rBddaH5nqKN+3oEpcbMZF6Hff3HF3tImOSouqeSIPtN4G6dyni3jJCBcHcrafzPv8B13NkQkVaYmEAAAAASUVORK5CYII=" alt="" /></p>

Additional context

Enabling Parser.LINKS_ALLOW_MATCHED_PARENTHESES avoid the regular expressions which I think is the culprit in InlineParserImpl#parseLinkDestination:1168:

if (options.linksAllowMatchedParentheses) {
    // ... 
} else {
    // spec 0.27 compatibility
    BasedSequence matched = match(myParsing.LINK_DESTINATION);
    return matched != null && spaceInUrls ? matched.trimEnd(BasedSequence.SPACE) : matched;
}

[vsch]

@mrombout, I think you are right.

I have been trying to figure out which option causes this and even have a hand-rolled parser to replace the regex one, which does not have the issue. However, in some cases the stack overflow is still caused and I have not been able to isolate exactly what causes this.

I will take another look through the code to see what other parts of code use this regex but in the mean time I will disable this feature by default.

[vsch]

@mrombout, my bad. I completely forgot about the matched parens option and my hand-rolled parser, which does not suffer from stack overflow, is bypassed if this option is not selected. Thus the continued stack overflow occurrences.

I am adding a flag to the state machine parser for no parens and will have it parsing in both cases without regex.

Thank you for bringing this up. It was on my to do list and nagging me to figure it out. Your finding pointed me in the right direction.

[vsch]

Fix for this is available. Repo updated, maven updated but may take a while to show up in maven central.