Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feat(BigQuery): Add support for column descriptions in nested fields #3887

Merged
merged 5 commits into from
Feb 27, 2025

Conversation

themisvaltinos
Copy link
Contributor

This PR introduces support for column descriptions in nested types in BigQuery, closes #3854

The syntax follows BigQuery's conventions for nested and repeated columns, using dots to indicate the appropriate level of nesting:

MODEL (
  name schema.record_model,
  column_descriptions(
    id = "Unique identifier for the event",
    repeated_record.sub_repeated_record.sub_field = "Sub-field within nested repeated record",
    record.nested_record.field = "Field within nested record",
    record.nested_record.array = "Array of integers",
    repeated_record.sub_repeated_record = "Nested repeated record",
    record.nested_record = "Nested record"
  )
);

SELECT
  id,
  [STRUCT([STRUCT(sub_field AS sub_field)] AS sub_repeated_record)] AS repeated_record,
  STRUCT(
    STRUCT([2, 3] AS array, field AS field) AS nested_record
  ) AS record
FROM
  schema.seed_model;

The column definitions are built using the helper method _build_column_def, which is applied recursively in the BigQuery adapter to handle the RECORD and REPEATED RECORD columns. For views which are registered in _create_column_comments we traverse the fields and apply comments at each appropriate level.

Copy link
Contributor

@georgesittas georgesittas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A couple more comments, but LGTM otherwise.

@themisvaltinos themisvaltinos merged commit 2540204 into main Feb 27, 2025
22 checks passed
@themisvaltinos themisvaltinos deleted the themis/bq_descriptions branch February 27, 2025 13:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support nested column descriptions in BigQuery
2 participants