[AIRFLOW-5152]Change autodetect default value to be True

[ewljmgzbyydh123]

Make sure you have checked all steps below.

Jira

• My PR addresses the following [Airflow Jira] issues and references them in the PR title.
  • https://issues.apache.org/jira/browse/AIRFLOW-5152

Description

• Here are some details about my PR, including screenshots of any UI changes:

Set autodetect default value from false to be true to avoid breaking downstream services using GoogleCloudStorageToBigQueryOperator but not aware of the newly added autodetect field. This PR is to fix the current regression introduced by #3880

Tests

• My PR should pass the existing unit tests in [AIRFLOW-461] Support autodetected schemas in BigQuery run_load #3880

Commits

• My commits all reference Jira issues in their subject lines, and I have squashed multiple commits if they address the same issue. In addition, my commits follow the guidelines from "How to write a good git commit message":
  1. Subject is separated from body by a blank line
  2. Subject is limited to 50 characters (not including Jira issue reference)
  3. Subject does not end with a period
  4. Subject uses the imperative mood ("add", not "adding")
  5. Body wraps at 72 characters
  6. Body explains "what" and "why", not "how"

Documentation

• In case of new functionality, my PR adds documentation that describes how to use it.
  • All the public functions and the classes in the PR contain docstrings that explain what it does
  • If you implement backwards incompatible changes, please leave a note in the Updating.md so we can assign it to a appropriate release

Code Quality

• Passes flake8

[turbaszek]

This seems to be reasonable change but is it backward compatible? If user passes a schema definition and autodetect is set to True will the schema be use instead of autodetection?

Also, please adjust commit and PR message to reference the JIRA issue ✅

[criccomini]

I would argue that the change, itself, was backwards incompatible, and this is restoring the compatibility. We had DAGs that were running fine. After the patch referenced in the JIRA, they broke. This restores proper behavior.

[turbaszek]

@criccomini thank you for the insight!

[ewljmgzbyydh123]

This seems to be reasonable change but is it backward compatible? If user passes a schema definition and autodetect is set to True will the schema be use instead of autodetection?

That's a good catch. Not sure how this autodetect config will ultimately be handled by gcloud, but I think for this case, schema should still be used instead of autodetection. Please refer to codes below:

• If schema_fields is not None, it will use schema_fields.
  (https://github.com/apache/airflow/blob/master/airflow/contrib/operators/gcs_to_bq.py#L242-L243)
• If schema_fields is None but schema_objects is not None, it will use schema_objects.
  (https://github.com/apache/airflow/blob/master/airflow/contrib/operators/gcs_to_bq.py#L229-L235)
• If schema_fields is None, schema_objects is None and autodetect is True, it will use autodection.
  (https://github.com/apache/airflow/blob/master/airflow/contrib/operators/gcs_to_bq.py#L239-L240)
• If schema_fields is None, schema_object is None, and autodetect is False, it will throw Exception.
  (https://github.com/apache/airflow/blob/master/airflow/contrib/operators/gcs_to_bq.py#L236-L238)

Also, please adjust commit and PR message to reference the JIRA issue ✅

[ewljmgzbyydh123]

Declining this PR and created #5771 since I messed up the commit messages.

[turbaszek]

In such cases git commit --amend may help, it's quite useful:
https://help.github.com/en/articles/changing-a-commit-message

[ashb]

@criccomini Should we pull this (well the re-opened one) in as a bugfix for 1.10.5 then?

[criccomini]

@ashb that would be ideal, if possible. :)