-
Notifications
You must be signed in to change notification settings - Fork 14.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Introduces separate runtime provider schema #13488
Introduces separate runtime provider schema #13488
Conversation
cc: @mik-laj - for some reason I cannot add you as reviewer? |
ec40e72
to
426d584
Compare
426d584
to
e34db89
Compare
ed2a7ba
to
ab48fc5
Compare
All comments addressed. I also added more colors in the output with 'rich' to see more clearly errors/OK status. |
The Workflow run is cancelling this PR. It has some failed jobs matching ^Pylint$,^Static checks,^Build docs$,^Spell check docs$,^Backport packages$,^Provider packages,^Checks: Helm tests$,^Test OpenAPI*. |
973bffb
to
904f712
Compare
904f712
to
7a9e590
Compare
Looks like it's going to be green (except quarantined build) looking forward to approval! |
152fb49
to
30088c8
Compare
The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and take the risk. If they don't merge it quickly - please rebase it to the latest master at your convenience, or amend the last commit of the PR, and push it with --force-with-lease. |
The Workflow run is cancelling this PR. Building images for the PR has failed. Follow the the workflow link to check the reason. |
d371eab
to
e8e6ac4
Compare
The Workflow run is cancelling this PR. Building images for the PR has failed. Follow the the workflow link to check the reason. |
The Workflow run is cancelling this PR. It has some failed jobs matching ^Pylint$,^Static checks,^Build docs$,^Spell check docs$,^Backport packages$,^Provider packages,^Checks: Helm tests$,^Test OpenAPI*. |
e8e6ac4
to
057c397
Compare
@kaxil -> all fixed and only quarantined tests failing |
The provider.yaml contains more information that required at runtime (specifically about documentation building). Those fields are not needed at runtime and their presence is optional. Also the runtime check for provider information should be more relexed and allow for future compatibility (with additional properties set to false). This way we can add new, optional fields to provider.yaml without worrying about breaking future-compatibility of providers with future airflow versions. This changei restores 'additionalProperties': false in the main, development-focused provider.yaml schema and introduced new runtime schema that is used to verify the provider info when providers are discovered by airflow. This 'runtime' version should change very rarely as change to add a new required property in it breaks compatibility of providers with already released versions of Airflow. We also trim-down the provider.yaml file when preparing provider packages to only contain those fields that are required in the runtime schema.
057c397
to
fed91c6
Compare
The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and take the risk. If they don't merge it quickly - please rebase it to the latest master at your convenience, or amend the last commit of the PR, and push it with --force-with-lease. |
The provider.yaml contains more information that required at runtime (specifically about documentation building). Those fields are not needed at runtime and their presence is optional. Also the runtime check for provider information should be more relexed and allow for future compatibility (with additional properties set to false). This way we can add new, optional fields to provider.yaml without worrying about breaking future-compatibility of providers with future airflow versions. This changei restores 'additionalProperties': false in the main, development-focused provider.yaml schema and introduced new runtime schema that is used to verify the provider info when providers are discovered by airflow. This 'runtime' version should change very rarely as change to add a new required property in it breaks compatibility of providers with already released versions of Airflow. We also trim-down the provider.yaml file when preparing provider packages to only contain those fields that are required in the runtime schema. (cherry picked from commit ad2a030)
Based on ##13556
The provider.yaml contains more information that required at
runtime (specifically about documentation building). Those
fields are not needed at runtime and their presence is optional.
Also the runtime check for provider information should be more
relexed and allow for future compatibility (with
additional properties set to false). This way we can add new,
optional fields to provider.yaml without worrying about breaking
future-compatibility of providers with future airflow versions.
This changei restores 'additionalProperties': false in the
main, development-focused provider.yaml schema and introduced
new runtime schema that is used to verify the provider info when
providers are discovered by airflow.
This 'runtime' version should change very rarely as change to
add a new required property in it breaks compatibility of
providers with already released versions of Airflow.
We also trim-down the provider.yaml file when preparing provider
packages to only contain those fields that are required in the
runtime schema.
^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code change, Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in UPDATING.md.