Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add iceberg feature #256

Merged
merged 16 commits into from
Dec 12, 2024

Conversation

matthewmturner
Copy link
Collaborator

No description provided.

Comment on lines 119 to 131
pub async fn register_extensions(&mut self) -> Result<()> {
let ctx = &mut self.session_ctx;
let config = &self.config;
let extensions = enabled_extensions();
// Apply any additional setup to the session context (e.g. registering
// functions)
for extension in &extensions {
extension.register_on_ctx(config, ctx)?;
}

Ok(())
}

Copy link
Collaborator Author

@matthewmturner matthewmturner Dec 3, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

New method for registering extensions

@matthewmturner matthewmturner changed the title Start adding iceberg Add iceberg feature Dec 3, 2024
@matthewmturner
Copy link
Collaborator Author

for this initial pr im just going to register the table provider factory, i will elicit feedback from the iceberg community about how to best integrate iceberg catalogs.

@matthewmturner
Copy link
Collaborator Author

I am merging this as is for now as there isnt a clear easy solution to adding testing for iceberg - i anticipate coming back to this though since i do plan on eventually turning dft as a library in an app i will be building that will likely use iceberg as a catalog.

I am keeping my eye on lakekeeper as part of these efforts and may end up utilizing their rest catalog implementation for testing purposes.

@matthewmturner matthewmturner merged commit a1345e1 into datafusion-contrib:main Dec 12, 2024
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant