fix: fix IndexError when partioning a pdf with starting_page_number
#3246
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The Issue: When extracting images from pdfs, we use the metadata page number to index into a list of the images. However, the metadata page number can now be changed via `starting_page_number`. To get the true page index, we need to subtract this value. Testing: Run this snippet in a python shell. Before the fix, this throws an IndexError. On this branch, it will return the elements. ``` from unstructured.partition.auto import partition filename = "example-docs/layout-parser-paper-with-table.pdf" partition(filename, strategy="hi_res", extract_image_block_types=["Image", "Table"], starting_page_number=20) ```
MthwRobinson
suggested changes
Jun 19, 2024
| @@ -1223,6 +1223,8 @@ def test_partition_pdf_element_extraction( | |||
| if file_mode == "filename": | |||
| elements = pdf.partition_pdf( | |||
| filename=filename, | |||
| # Image extraction shouldn't break by setting this | |||
| starting_page_number=20, | |||
There was a problem hiding this comment.
Could we also add a test that asserts that the resulting page number in metadata is correct?
MthwRobinson
changed the title
starting_page_numberIndexError when partioning a pdf with starting_page_number
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The Issue:
When extracting images from pdfs, we use the metadata page number to index into a list of the images. However, the metadata page number can now be changed via
starting_page_number. To get the true page index, we need to subtract this value.Testing:
Run this snippet in a python shell. Before the fix, this throws an IndexError. On this branch, it will return the elements.