Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(ingestion/bigquery): Add ability to filter GCP project ingestion based on project labels #11169

Merged
merged 31 commits into from
Aug 20, 2024
Merged

feat(ingestion/bigquery): Add ability to filter GCP project ingestion based on project labels #11169

merged 31 commits into from
Aug 20, 2024

Commits on Apr 7, 2024

  1. feat: First pass at enabling getting projects based on folder ids

    Alice Naghshineh committed Apr 7, 2024
    Configuration menu
    Copy the full SHA
    0c050be View commit details
    Browse the repository at this point in the history

Commits on Apr 8, 2024

  1. chore: Add google-cloud-resource-manager dependency

    Alice Naghshineh committed Apr 8, 2024
    Configuration menu
    Copy the full SHA
    69cf8e0 View commit details
    Browse the repository at this point in the history

  2. chore: Update some methods to account for new projects_client

    Alice Naghshineh committed Apr 8, 2024
    Configuration menu
    Copy the full SHA
    df9698c View commit details
    Browse the repository at this point in the history

  3. chore: Add a few tests, more to come

    Alice Naghshineh committed Apr 8, 2024
    Configuration menu
    Copy the full SHA
    6c77b1e View commit details
    Browse the repository at this point in the history

  4. Merge branch 'master' into add-support-for-ingestion-based-on-folder-ids

    Alice Naghshineh committed Apr 8, 2024
    Configuration menu
    Copy the full SHA
    7b1efab View commit details
    Browse the repository at this point in the history

  5. chore: Add another unit test

    Alice Naghshineh committed Apr 8, 2024
    Configuration menu
    Copy the full SHA
    7e3f829 View commit details
    Browse the repository at this point in the history

  6. feat: Switch to using labels instead of folders, more flexible

    Alice Naghshineh committed Apr 8, 2024
    Configuration menu
    Copy the full SHA
    463a18a View commit details
    Browse the repository at this point in the history

  7. chore: Remove line deletions

    Alice Naghshineh committed Apr 8, 2024
    Configuration menu
    Copy the full SHA
    bd1560d View commit details
    Browse the repository at this point in the history

  8. chore: Update failure report message to be accurate

    Alice Naghshineh committed Apr 8, 2024
    Configuration menu
    Copy the full SHA
    0aea92b View commit details
    Browse the repository at this point in the history

Commits on Apr 17, 2024

  1. Update metadata-ingestion/src/datahub/ingestion/source/bigquery_v2/bi… 

    …gquery_config.py

Co-authored-by: Tamas Nemeth <[email protected]>
    @anaghshineh @treff7es
    anaghshineh and treff7es authored Apr 17, 2024
    Configuration menu
    Copy the full SHA
    82147f3 View commit details
    Browse the repository at this point in the history

  2. docs: Update inline docs for new BQ config var

    Alice Naghshineh committed Apr 17, 2024
    Configuration menu
    Copy the full SHA
    d00d4db View commit details
    Browse the repository at this point in the history

  3. Merge branch 'bq-filter-ingestion-based-on-proj-labels' of github.com… 

    …:anaghshineh/datahub into bq-filter-ingestion-based-on-proj-labels
    Alice Naghshineh committed Apr 17, 2024
    Configuration menu
    Copy the full SHA
    110bb7e View commit details
    Browse the repository at this point in the history

Commits on Aug 13, 2024

  1. resolve conflict

    @sid-acryl
    sid-acryl committed Aug 13, 2024
    Configuration menu
    Copy the full SHA
    b5a932b View commit details
    Browse the repository at this point in the history

  2. Merge branch 'master' into ing-690-bq-filter-ingestion-based-on-proj-… 

    …labels
    @sid-acryl
    sid-acryl authored Aug 13, 2024
    Configuration menu
    Copy the full SHA
    2f51f8b View commit details
    Browse the repository at this point in the history

Commits on Aug 14, 2024

  1. test case

    @sid-acryl
    sid-acryl committed Aug 14, 2024
    Configuration menu
    Copy the full SHA
    f6af306 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'ing-690-bq-filter-ingestion-based-on-proj-labels' of gi… 

    …thub.com:sid-acryl/datahub-fork into ing-690-bq-filter-ingestion-based-on-proj-labels
    @sid-acryl
    sid-acryl committed Aug 14, 2024
    Configuration menu
    Copy the full SHA
    6379304 View commit details
    Browse the repository at this point in the history

  3. revert the base-requirements

    @sid-acryl
    sid-acryl committed Aug 14, 2024
    Configuration menu
    Copy the full SHA
    24204f0 View commit details
    Browse the repository at this point in the history

  4. fix mock

    @sid-acryl
    sid-acryl committed Aug 14, 2024
    Configuration menu
    Copy the full SHA
    68915a7 View commit details
    Browse the repository at this point in the history

  5. test case

    @sid-acryl
    sid-acryl committed Aug 14, 2024
    Configuration menu
    Copy the full SHA
    8525355 View commit details
    Browse the repository at this point in the history

  6. Merge branch 'master' into ing-690-bq-filter-ingestion-based-on-proj-… 

    …labels
    @sid-acryl
    sid-acryl authored Aug 14, 2024
    Configuration menu
    Copy the full SHA
    46a2447 View commit details
    Browse the repository at this point in the history

  7. Merge branch 'master' into ing-690-bq-filter-ingestion-based-on-proj-… 

    …labels
    @david-leifker
    david-leifker authored Aug 14, 2024
    Configuration menu
    Copy the full SHA
    3149ae7 View commit details
    Browse the repository at this point in the history

Commits on Aug 16, 2024

  1. Merge branch 'master' into ing-690-bq-filter-ingestion-based-on-proj-… 

    …labels
    @sid-acryl
    sid-acryl authored Aug 16, 2024
    Configuration menu
    Copy the full SHA
    a4d235a View commit details
    Browse the repository at this point in the history

Commits on Aug 19, 2024

  1. Merge branch 'master' into ing-690-bq-filter-ingestion-based-on-proj-… 

    …labels
    @sid-acryl
    sid-acryl authored Aug 19, 2024
    Configuration menu
    Copy the full SHA
    f049769 View commit details
    Browse the repository at this point in the history

  2. address review comments

    @sid-acryl
    sid-acryl committed Aug 19, 2024
    Configuration menu
    Copy the full SHA
    3584298 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'master' into ing-690-bq-filter-ingestion-based-on-proj-… 

    …labels
    @sid-acryl
    sid-acryl authored Aug 19, 2024
    Configuration menu
    Copy the full SHA
    f71f40f View commit details
    Browse the repository at this point in the history

Commits on Aug 20, 2024

  1. working with portal

    @sid-acryl
    sid-acryl committed Aug 20, 2024
    Configuration menu
    Copy the full SHA
    56cafba View commit details
    Browse the repository at this point in the history

  2. Merge branch 'ing-690-bq-filter-ingestion-based-on-proj-labels' of gi… 

    …thub.com:sid-acryl/datahub-fork into ing-690-bq-filter-ingestion-based-on-proj-labels
    @sid-acryl
    sid-acryl committed Aug 20, 2024
    Configuration menu
    Copy the full SHA
    624288c View commit details
    Browse the repository at this point in the history

  3. Merge branch 'master' into ing-690-bq-filter-ingestion-based-on-proj-… 

    …labels
    @sid-acryl
    sid-acryl authored Aug 20, 2024
    Configuration menu
    Copy the full SHA
    38d3dfd View commit details
    Browse the repository at this point in the history

  4. update quick start

    @sid-acryl
    sid-acryl committed Aug 20, 2024
    Configuration menu
    Copy the full SHA
    d38dd9e View commit details
    Browse the repository at this point in the history

  5. Merge branch 'ing-690-bq-filter-ingestion-based-on-proj-labels' of gi… 

    …thub.com:sid-acryl/datahub-fork into ing-690-bq-filter-ingestion-based-on-proj-labels
    @sid-acryl
    sid-acryl committed Aug 20, 2024
    Configuration menu
    Copy the full SHA
    ffa2dd4 View commit details
    Browse the repository at this point in the history

  6. lint fix

    @sid-acryl
    sid-acryl committed Aug 20, 2024
    Configuration menu
    Copy the full SHA
    38c3496 View commit details
    Browse the repository at this point in the history