Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Partitioned Iceberg models are not created correctly using Athena #3683

Closed
sean-eyre opened this issue Jan 22, 2025 · 2 comments
Closed

Partitioned Iceberg models are not created correctly using Athena #3683

sean-eyre opened this issue Jan 22, 2025 · 2 comments
Assignees
Labels
Bug Something isn't working

Comments

@sean-eyre
Copy link

sean-eyre commented Jan 22, 2025

I have a model I'm running on Athena using Iceberg with INCREMENTAL_BY_TIME_RANGE, however the CREATE TABLE statement executed on Athena uses a partitioned_by property, rather than the partitioning property used for Iceberg tables (docs).

This results in the model plan failing with Catalog 'awsdatacatalog$iceberg-aws' table property 'partitioned_by' does not exist.

Here's my model config:

MODEL (
    name model.name,
    kind INCREMENTAL_BY_TIME_RANGE (
      time_column business_date
   ),
    cron '@daily',
    table_format iceberg,
    start '2025-01-15'
);

And here's what is run in Athena:

CREATE TABLE IF NOT EXISTS ...
WITH (
    table_type='iceberg', 
    partitioned_by=ARRAY['business_date'], 
    location='...', 
    is_external=false
) 
AS ...

Thanks

@treysp
Copy link
Contributor

treysp commented Jan 22, 2025

@georgesittas
Copy link
Contributor

Fixed by linked sqlglot PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Something isn't working
Projects
None yet
Development

No branches or pull requests

4 participants