Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] Microbatch error #1376

Closed
2 tasks done
Tracked by #10624
jschintz-nytimes opened this issue Oct 18, 2024 · 0 comments · Fixed by dbt-labs/dbt-core#10917
Closed
2 tasks done
Tracked by #10624

[Bug] Microbatch error #1376

jschintz-nytimes opened this issue Oct 18, 2024 · 0 comments · Fixed by dbt-labs/dbt-core#10917
Assignees
Labels
bug Something isn't working

Comments

@jschintz-nytimes
Copy link

jschintz-nytimes commented Oct 18, 2024

Is this a new bug in dbt-bigquery?

  • I believe this is a new bug in dbt-bigquery
  • I have searched the existing issues, and I could not find an existing issue for this bug

Current Behavior

When running microbatch incremental models, receive the following error

Encountered an error while running operation: Compilation Error argument of type 'ModelNode' is not iterable

When I disable persist docs this error goes away

Expected Behavior

Model should run

Steps To Reproduce

with this model config

{{ config(
    materialized='incremental',
    incremental_strategy='microbatch',
    partition_by={
      'field': 'date_field',
      'granularity': 'day'
    },
    event_time='date_field',
    batch_size='day',
    begin=modules.datetime.datetime(2024, 10, 16, 0, 0, 0)
    )
}}
select 
    DATE(2024, 10, 16) as date_field

dbt run

Relevant log output

20:20:06 Re-using an available connection from the pool ()
20:20:06 Encountered an error while running operation: Compilation Error
  argument of type 'ModelNode' is not iterable
  
  > in macro bigquery__create_table_as (macros/adapters.sql)
  > called by macro create_table_as (macros/relations/table/create.sql)
  > called by macro bq_create_table_as (macros/materializations/incremental.sql)
  > called by macro statement (macros/etc/statement.sql)
  > called by macro materialization_incremental_bigquery (macros/materializations/incremental.sql)
  > called by 
20:20:06 Encountered an error while running operation: Compilation Error
  argument of type 'ModelNode' is not iterable
  
  > in macro bigquery__create_table_as (macros/adapters.sql)
  > called by macro create_table_as (macros/relations/table/create.sql)
  > called by macro bq_create_table_as (macros/materializations/incremental.sql)
  > called by macro statement (macros/etc/statement.sql)
  > called by macro materialization_incremental_bigquery (macros/materializations/incremental.sql)
  > called by 
20:20:06 Encountered an error while running operation: Compilation Error
  argument of type 'ModelNode' is not iterable
  
  > in macro bigquery__create_table_as (macros/adapters.sql)
  > called by macro create_table_as (macros/relations/table/create.sql)
  > called by macro bq_create_table_as (macros/materializations/incremental.sql)
  > called by macro statement (macros/etc/statement.sql)
  > called by macro materialization_incremental_bigquery (macros/materializations/incremental.sql)
  > called by 
20:20:06 Opening a new connection, currently in state closed
20:20:06 Connection 'master' was properly closed.
20:20:06 Finished running 1 incremental model in 0 hours 0 minutes and 1.09 seconds (1.09s).
20:20:06 Command end result
20:20:06 Completed with 1 error, 0 partial successs, and 0 warnings:
20:20:06 Done. PASS=0 WARN=0 ERROR=1 SKIP=0 TOTAL=1
20:20:06 Resource report: {"command_name": "build", "command_wall_clock_time": 1.4175539, "process_user_time": 8.919335, "process_kernel_time": 3.221199, "process_mem_max_rss": "323776", "process_out_blocks": "106816", "command_success": false, "process_in_blocks": "0"}
20:20:06 Observability Metric: command_success=0.0
20:20:06 Observability Metric: command_wall_clock_time=1.4175539016723633
20:20:06 Observability Metric: process_user_time=8.91933536529541
20:20:06 Observability Metric: process_kernel_time=3.2211990356445312
20:20:06 Observability Metric: process_mem_max_rss=323776.0
20:20:06 Command `cli build` failed at 20:20:06.992467 after 1.42 seconds
Running dbt...

Environment

dbt-bigquery: 1.9.0

Additional Context

No response

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants