Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

.extractText() reads / as 1. #789

Closed
dakotabartell opened this issue Apr 21, 2022 · 8 comments
Closed

.extractText() reads / as 1. #789

dakotabartell opened this issue Apr 21, 2022 · 8 comments
Assignees
@MartinThoma
Labels
Has MCVE A minimal, complete and verifiable example helps a lot to debug / understand feature requests is-bug From a users perspective, this is a bug - a violation of the expected behavior with a compliant PDF workflow-text-extraction From a users perspective, text extraction is the affected feature/workflow

Comments

@dakotabartell
Copy link

dakotabartell commented Apr 21, 2022 •
edited by MartinThoma
Loading

I'm trying to automate sorting pdfs by the date on the pdf. However the issue I continue having is that the /'s in the dates continually get read as 1's. Wouldn't be a problem 90% of the time unfortunately it reads a lot of January and November dates as the same

1/11/2022
11/1/2022

Both end up as 111112022

I tried getting the new pdfs to change to a new format to have 01/11/2022 but they aren't able to do that. Is there a way to fix this?

from PyPDF2 import PdfReader

reader = PdfReader("TestPackingSlip637860440227283947.pdf")
print(f"Total pages= {len(reader.pages)}")

for i, page in enumerate(reader.pages, start=1):
    print(f"Page: {i}")
    print(page.extract_text())

The info on the pdf I'm uploading is randomized and does not represent anyone's real info.

TestPackingSlip637860440227283947.pdf
@dakotabartell dakotabartell added the is-bug From a users perspective, this is a bug - a violation of the expected behavior with a compliant PDF label Apr 21, 2022
@MartinThoma MartinThoma added Has MCVE A minimal, complete and verifiable example helps a lot to debug / understand feature requests workflow-text-extraction From a users perspective, text extraction is the affected feature/workflow labels Apr 21, 2022
@MartinThoma
Copy link
Member

@dakotabartell For me, PyPDF2 doesn't extract anything. That is not the bug you described, but for sure an issue as well.

May I add your example to PyPDF2 (Resources, under BSD 3-clause) and a test that shows that it fails?

@dakotabartell
Copy link
Author

dakotabartell commented Apr 21, 2022 via email

@pubpub-zz
Copy link
Collaborator

I've started profiling of the extractText with your file and I found a Do operation : it means that the text is part of a postscript image/XObject. the extraction/analysis of this XObject looks like a hudge job and this type of object is considered as obsolete...
I will personnaly not engage into this task

@pubpub-zz
Copy link
Collaborator

@dakotabartell
I've reviewed my comment : I've found how to extract Xform object. This is in progress in PR #929 for alpha evaluation
I still have some issues with the true type font : you will get extra \x00 to remove for the moment

@dakotabartell
Copy link
Author

dakotabartell commented May 31, 2022 via email

@MartinThoma
Copy link
Member

#924 fixed this issue. I'll create a release today with it :-)

MartinThoma added a commit that referenced this issue Jun 6, 2022
@MartinThoma
REL: 2.1.0
4e44122 
The highlight of the 2.1.0 release is the most massive improvement to the
text extraction capabilities of PyPDF2 since 2016 🥳🎊 A very big thank you goes
to [pubpub-zz](https://github.com/pubpub-zz) who took a lot of time and
knowledge about the PDF format to finally get those improvements into PyPDF2.
Thank you 🤗💚

In case the new function causes any issues, you can use `_extract_text_old`
for the old functionality. Please also open a bug ticket in that case.

There were several people who have attempted to bring similar improvements to
PyPDF2. All of those were valuable. The main reason why they didn't get merged
is the big amount of open PRs / issues. pubpub-zz was the most comprehensive
PR which also incorporated the latest changes of PyPDF2 2.0.0.

Thank you to [VictorCarlquist](https://github.com/VictorCarlquist) for #858 and
[asabramo](https://github.com/asabramo) for #464 🤗

New Features (ENH):
-  Massive text extraction improvement (#924). Closed many open issues:
    - Exceptions / missing spaces in extract_text() method (#17) 🕺
      - Whitespace issues in extract_text() (#42) 💃
      - pypdf2 reads the hifenated words in a new line (#246)
    - PyPDF2 failing to read unicode character (#37)
      - Unable to read bullets (#230)
    - ExtractText yields nothing for apparently good PDF (#168) 🎉
    - Encoding issue in extract_text() (#235)
    - extractText() doesn't work on Chinese PDF (#252)
    - encoding error (#260)
    - Trouble with apostophes in names in text "O'Doul" (#384)
    - extract_text works for some PDF files, but not the others (#437)
    - Euro sign not being recognized by extractText (#443)
    - Failed extracting text from French texts (#524)
    - extract_text doesn't extract ligatures correctly (#598)
    - reading spanish text - mark convert issue (#635)
    - Read PDF changed from text to random symbols (#654)
    - .extractText() reads / as 1. (#789)
-  Update glyphlist (#947) - inspired by #464
-  Allow adding PageRange objects (#948)

Bug Fixes (BUG):
-  Delete .python-version file (#944)
-  Compare StreamObject.decoded_self with None (#931)

Robustness (ROB):
-  Fix some conversion errors on non conform PDF (#932)

Documentation (DOC):
-  Elaborate on PDF text extraction difficulties (#939)
-  Add logo (#942)
-  rotate vs Transformation().rotate (#937)
-  Example how to use PyPDF2 with AWS S3 (#938)
-  How to deprecate (#930)
-  Fix typos on robustness page (#935)
-  Remove scripts (pdfcat) from docs (#934)

Developer Experience (DEV):
-  Ignore .python-version file
-  Mark deprecated code with no-cover (#943)
-  Automatically create Github releases from tags (#870)

Testing (TST):
-  Text extraction for non-latin alphabets (#954)
-  Ignore PdfReadWarning in benchmark (#949)
-  writer.remove_text (#946)
-  Add test for Tree and _security (#945)

Code Style (STY):
-  black, isort, Flake8, splitting buildCharMap (#950)

Full Changelog: 2.0.0...2.1.0
@dakotabartell
Copy link
Author

dakotabartell commented Oct 11, 2022 via email

@pubpub-zz
Copy link
Collaborator

If you upgrade to the latest version, you should be able to extract the text. This upgrade may be done with
pip install --upgrade PyPDF2

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@MartinThoma MartinThoma

Labels
Has MCVE A minimal, complete and verifiable example helps a lot to debug / understand feature requests is-bug From a users perspective, this is a bug - a violation of the expected behavior with a compliant PDF workflow-text-extraction From a users perspective, text extraction is the affected feature/workflow
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@MartinThoma @pubpub-zz @dakotabartell