CHANGELOG.md

Changelog

Unreleased

0.3.1 - 2021-10-21

Fixed

• fix import in spark3 visitor @mobuchowski

0.3.0 - 2021-10-21

Added

• Spark3 support @OleksandrDvornik/@collado-mike
• LineageBackend for Airflow 2 @mobuchowski
• Adding custom spark version facet to spark integration @OleksandrDvornik
• Adding dbt version facet @mobuchowski
• Added support for Redshift profile @AlessandroLollo

Fixed

• Sanitize JDBC URLs @OleksandrDvornik
• strip openlineage url in python client @OleksandrDvornik
• deploy spec if spec file changes @mobuchowski

0.2.3 - 2021-10-07

Fixed

• Add dbt v3 manifest support @mobuchowski

0.2.2 - 2021-09-08

Added

• Implement OpenLineageValidationAction for Great Expectations @collado-mike
• facet: add expectations assertions facet @mobuchowski

Fixed

• airflow: pendulum formatting fix, add tests @mobuchowski
• dbt: do not emit events if run_result file was not updated @mobuchowski

0.2.1 - 2021-08-27

Fixed

• Default --project-dir argument to current directory in dbt-ol script @mobuchowski

0.2.0 - 2021-08-23

Added

• Parse dbt command line arguments when invoking dbt-ol @mobuchowski. For example:

  $ dbt-ol run --project-dir path/to/dir
  

• Set UnknownFacet for spark (captures metadata about unvisited nodes from spark plan not yet supported) @OleksandrDvornik

Changed

• Remove model from dbt job name @mobuchowski
• Default dbt job namespace to output dataset namespace @mobuchowski
• Rename openlineage.spark.* to io.openlineage.spark.* @OleksandrDvornik

Fixed

• Remove instance references to extractors from DAG and avoid copying log property for serializability @collado-mike

0.1.0 - 2021-08-12

OpenLineage is an Open Standard for lineage metadata collection designed to record metadata for a job in execution. The initial public release includes:

• An inital specification. The the inital version 1-0-0 of the OpenLineage specification defines the core model and facets.
• Integrations that collect lineage metadata as OpenLineage events:
  • Apache Airflow with support for BigQuery, Great Expectations, Postgres, Redshift, Snowflake
  • Apache Spark
  • dbt
• Clients that send OpenLineage events to an HTTP backend. Both java and python are initially supported.