feat: batch multiple files in a single Unstructured API request

[MthwRobinson]

Submit Multiple Files to the Unstructured API

Enables batching multiple files into a single Unstructured API requests. Support for requests with multiple files was added to both UnstructuredAPIFileLoader and UnstructuredAPIFileIOLoader. Note that if you submit multiple files in "single" mode, the result will be concatenated into a single document. We recommend using this feature in "elements" mode.

Testing

The following should load both documents, using two of the example docs from the integration tests folder.

    from langchain.document_loaders import UnstructuredAPIFileLoader

    file_paths = ["examples/layout-parser-paper.pdf",  "examples/whatsapp_chat.txt"]

    loader = UnstructuredAPIFileLoader(
        file_paths=file_paths,
        api_key="FAKE_API_KEY",
        strategy="fast",
        mode="elements",
    )
    docs = loader.load()

[dev2049]

could these be part of unstructured_kwargs?

[dev2049]

(as in file_filename and file_filenames)

[MthwRobinson]

Yep those can just be unstructured_kwargs. I'll update that.

[MthwRobinson]

Updated!

[eyurtsev]

Could we add * operator.

[MthwRobinson]

Updated file_path to a Union, so this kwarg doesn't exist anymore.

[eyurtsev]

would it make sense to union this field instead?

file_path: Union[str, Sequence[str]]

and ideally

FileLike: Union[str, Path, bytes.IO]
`file_path`: Union[FileLike, Sequence[FileLike]]

[MthwRobinson]

Made file_path and file both Unions.

[eyurtsev]

Favor Sequence instead of List on inputs

[MthwRobinson]

Update this to Sequence.

[eyurtsev]

It might make sense to lift the path into the global namespace -- the code that finds the file path is duplicated -- which will make it more difficult to refactor locations of data fixtures

[MthwRobinson]

Yeah that makes sense to me, I'll make that update