Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Re-enable support for ListingTables with custom file formats #8637

Closed
pgwhalen opened this issue Dec 23, 2023 · 3 comments
Closed

Re-enable support for ListingTables with custom file formats #8637

pgwhalen opened this issue Dec 23, 2023 · 3 comments
Labels
enhancement New feature or request

Comments

@pgwhalen
Copy link
Contributor

Is your feature request related to a problem or challenge?

After I upgraded my application from 28.0.0 to 34.0.0, I noticed that as a result of #7390, it is no longer possible to register your own listing table with a custom file format, because the code that supports writing assumes a fixed list of FileTypes. Specifically, the when implementing FileFormat, the enum must be provided.

Describe the solution you'd like

Ideally FileFormat would not have to return a FileType, and the ability to write to listing tables would be:

  • optional
  • not coupled to an enumerated list of file types

As it stands currently, it looks like this method is only used to help out code that supports writing, so perhaps it can be refactored to be more coupled with that functionality. I don't have any specific proposal at the moment, but it seems achievable (if it's something maintainers would be okay with).

Describe alternatives you've considered

Several alternatives seem to work for me:

  • Give up on ListingTable as an extensible API and just use TableProvider instead. This would be disappointing, because it does provide a lot of nice functionality that would have to reimplemented.
  • Provide a garbage FileType in FileFormat::file_type because it doesn't seem to cause problems unless you're trying to write. This is what I'm doing for now.

Additional context

No response

@alamb
Copy link
Contributor

alamb commented Dec 26, 2023

Thank you for this comment @pgwhalen

I think another idea that has come up (e.g. in #8345) is to combine FileType into the FileFormat -- I filed #8657 with a more specific proposal

@pgwhalen
Copy link
Contributor Author

That looks great, I'll keep an eye on it, thank you!

@pgwhalen
Copy link
Contributor Author

Closing due to fix: #10499

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants