Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fact daily trips incremental #1642

Merged
merged 4 commits into from
Jul 19, 2022
Merged

Conversation

lauriemerrell
Copy link
Contributor

@lauriemerrell lauriemerrell commented Jul 19, 2022

Description

Makes a test version of the gtfs_schedule_fact_daily_trips table that is incrementally materialized. We will let this run for about a week in test to confirm it is accurately replicating the current behavior and, if so, replace the old table (which currently takes 20+ minutes to run every day and causes our dbt run DAG task to fail.)

This also updates the documentation & tests on the main gtfs_schedule_fact_daily_trips table to:

  1. Fix the failing uniqueness test -- this partially addresses Deal with two-row feed_info.txt #1471, though not entirely.
  2. Correctly identify the (composite) primary key

Type of change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation
  • agencies.yml

How has this been tested?

dbt run and dbt test both pass locally for this new table

Screenshots (optional)

@lauriemerrell lauriemerrell marked this pull request as ready for review July 19, 2022 19:57
@lauriemerrell lauriemerrell merged commit 9f46986 into main Jul 19, 2022
@lauriemerrell lauriemerrell deleted the fact_daily_trips_incremental branch July 19, 2022 20:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants