Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: check ole storage content to differentiate filetypes #3581

Merged
merged 10 commits into from
Aug 30, 2024

Conversation

MthwRobinson
Copy link
Contributor

Summary

Updates the file detection logic for OLE files to check the storage content of the file to more reliable differentiate between DOC, PPT, XLS and MSG files. This corrects a bug that caused file type detection to be incorrect in cases where the filetype library guessed and incorrect MIME type, such as 'application/vnd.ms-excel' for a .msg file.

As part of this work, the "msg" extra was removed because the python-oxmsg package is now a base dependency.

Testing

Using a test .msg file that returns 'application/vnd.ms-excel' from filetype.guess_mime.

from unstructured.file_utils.filetype import detect_filetype

filename = "test-file.msg"
detect_filetype(filename=filename) # result should be FileType.MSG

@MthwRobinson MthwRobinson requested a review from scanny August 30, 2024 17:03
("fake-email.msg.outlook", FileType.UNK),
],
)
def test_it_can_only_detect_MSG_format_by_extension(
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Test was removed because the extension is no longer required to detect MSG files.

Copy link
Collaborator

@scanny scanny left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sweet! LGTM :)

@MthwRobinson MthwRobinson enabled auto-merge August 30, 2024 18:07
@MthwRobinson MthwRobinson added this pull request to the merge queue Aug 30, 2024
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Aug 30, 2024
@MthwRobinson MthwRobinson merged commit 6ba8135 into main Aug 30, 2024
50 checks passed
@MthwRobinson MthwRobinson deleted the fix/msg-detection branch August 30, 2024 19:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants