Skip to content

Commit

Permalink
Read API Source v2 (apache#25392)
Browse files Browse the repository at this point in the history 
* ReadAPI Source v2

* Renamed the Source V2. Also added tests for the same.

* v2 using OffsetBasedSource and OffsetBasedReader

* Updating tests to have more sensible mock values.

* Updated BqOption flag.

* Simplifying `fractionConsumed` calculation.

* Better variable names.

* Minor refactoring.

* Added a synchronized block in readNextRecord(). Also added comments to explain how OffsetBasedSource + RangeTracker is used.

* Removed unnecessary synchronized block, added Javadoc and improved unit test coverage.

* Consolidated code paths in `BigQueryIO` for Bundled and regular ReadAPI sources.

* Lint fixes.

* Minor Javadoc fix.

* Fix StreamBundle creation logic and some minor code comment updates.

* Updated logging.

* Lint fixes.

* ReadAPI Source v2

* Renamed the Source V2. Also added tests for the same.

* v2 using OffsetBasedSource and OffsetBasedReader

* Updating tests to have more sensible mock values.

* Updated BqOption flag.

* Simplifying `fractionConsumed` calculation.

* Better variable names.

* Minor refactoring.

* Added a synchronized block in readNextRecord(). Also added comments to explain how OffsetBasedSource + RangeTracker is used.

* Removed unnecessary synchronized block, added Javadoc and improved unit test coverage.

* Consolidated code paths in `BigQueryIO` for Bundled and regular ReadAPI sources.

* Lint fixes.

* Minor Javadoc fix.

* Fix StreamBundle creation logic and some minor code comment updates.

* Updated logging.

* Lint fixes.
  • Loading branch information
@vachan-shetty
vachan-shetty authored Feb 22, 2023
1 parent 93c1e92 commit 4ce8eed
Show file tree
Hide file tree
Showing 5 changed files with 2,867 additions and 105 deletions.
Loading

0 comments on commit 4ce8eed

Please sign in to comment.