Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[AIRFLOW-3540] Respect environment config when looking up config file. #4340

Merged
merged 1 commit into from
Dec 19, 2018

Conversation

jmcarp
Copy link
Contributor

@jmcarp jmcarp commented Dec 18, 2018

Make sure you have checked all steps below.

Jira

  • My PR addresses the following Airflow Jira issues and references them in the PR title. For example, "[AIRFLOW-XXX] My Airflow PR"

Description

  • Here are some details about my PR, including screenshots of any UI changes:

If ~/airflow/airflow.cfg exists, airflow uses that file instead of $AIRFLOW_HOME/airflow.cfg. This behavior is confusing--airflow shouldn't look for its config file in one location if it happens to exist there, else in another location. If $AIRFLOW_HOME is set and $AIRFLOW_CONFIG is not, we should only check $AIRFLOW_HOME/airflow.cfg.

Tests

  • My PR adds the following unit tests OR does not need testing for this extremely good reason:

Commits

  • My commits all reference Jira issues in their subject lines, and I have squashed multiple commits if they address the same issue. In addition, my commits follow the guidelines from "How to write a good git commit message":
    1. Subject is separated from body by a blank line
    2. Subject is limited to 50 characters (not including Jira issue reference)
    3. Subject does not end with a period
    4. Subject uses the imperative mood ("add", not "adding")
    5. Body wraps at 72 characters
    6. Body explains "what" and "why", not "how"

Documentation

  • In case of new functionality, my PR adds documentation that describes how to use it.
    • When adding new operators/hooks/sensors, the autoclass documentation generation needs to be added.
    • All the public functions and the classes in the PR contain docstrings that explain what it does

Code Quality

  • Passes flake8

@jmcarp jmcarp force-pushed the fix-default-config branch from 5f023e9 to 2d66bf9 Compare December 18, 2018 17:34
@codecov-io
Copy link

codecov-io commented Dec 18, 2018

Codecov Report

Merging #4340 into master will increase coverage by 0.01%.
The diff coverage is 100%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #4340      +/-   ##
==========================================
+ Coverage    78.1%   78.11%   +0.01%     
==========================================
  Files         201      201              
  Lines       16471    16471              
==========================================
+ Hits        12864    12866       +2     
+ Misses       3607     3605       -2
Impacted Files Coverage Δ
airflow/configuration.py 89.96% <100%> (+0.71%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f0bb3be...1fa4005. Read the comment docs.

@feng-tao
Copy link
Member

lgtm, could you add a unit test for this change?

@ashb
Copy link
Member

ashb commented Dec 19, 2018

:D This confused me just yesterday!


mkdir_p(AIRFLOW_HOME)

AIRFLOW_CONFIG = expand_env_var(os.environ)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks suspect - we're passing a dict to expand_env_var(), not a single value.

@ashb
Copy link
Member

ashb commented Dec 19, 2018

This probably deserves a note in UPDATING.md as it could in theory mean someone's install won't pick up an airflow.cfg that they were relying on before.

@jmcarp jmcarp force-pushed the fix-default-config branch 2 times, most recently from 665d257 to 9f130ac Compare December 19, 2018 17:11
@jmcarp jmcarp force-pushed the fix-default-config branch from 9f130ac to 1fa4005 Compare December 19, 2018 17:29
@jmcarp
Copy link
Contributor Author

jmcarp commented Dec 19, 2018

Thanks for reviewing @feng-tao and @ashb. I refactored the code to make it more straightforward to test, added tests, and dropped the typo that @ashb pointed out.

@feng-tao
Copy link
Member

lgtm

@feng-tao feng-tao merged commit 71129cd into apache:master Dec 19, 2018
aliceabe pushed a commit to aliceabe/incubator-airflow that referenced this pull request Jan 3, 2019
ashb pushed a commit to ashb/airflow that referenced this pull request Mar 6, 2019
wmorris75 pushed a commit to modmed/incubator-airflow that referenced this pull request Jul 29, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants