Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement native schema inference for PostgreSQL #153

Merged
merged 1 commit into from
Oct 25, 2024

Conversation

phillipleblanc
Copy link
Collaborator

Closes #127

This PR implements native schema inference for PostgreSQL using a combination of postgres native system tables and information_schema. It handles all primitive types, array types, complex types and enums. It also handles quite a few types that the row conversion itself doesn't support yet - but those can be added over time (i.e. TSVECTOR, INET, CIDR, MACADDR, UUID, JSON, JSONB, XML).

This allows the schema to be retrieved even for tables without any rows.

* wip

* test cleanup

* Implement native schema inference for PostgreSQL

* handle bpchar

* fix uuid

* Fix test

* align

* fix snapshot
@phillipleblanc phillipleblanc merged commit a9e1469 into main Oct 25, 2024
3 checks passed
@phillipleblanc phillipleblanc deleted the phillip/241024-schema-inference branch October 25, 2024 08:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Query postgres information_schema to construct datafusion table schema
2 participants