Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[AVRO] Prioritise Avro providers from "extensions/core" #25611

Merged
merged 1 commit into from
Feb 24, 2023
Merged

[AVRO] Prioritise Avro providers from "extensions/core" #25611

merged 1 commit into from
Feb 24, 2023

Conversation

aromanenko-dev
Copy link
Contributor

@aromanenko-dev aromanenko-dev commented Feb 23, 2023 •
edited
Loading

Fixes #25601

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

  • Mention the appropriate issue in your description (for example: addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment fixes #<ISSUE NUMBER> instead.
  • Update CHANGES.md with noteworthy changes.
  • If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.

To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md

GitHub Actions Tests Status (on master branch)

Build python source distribution and wheels
Python tests
Java tests
Go tests

See CI.md for more information about GitHub Actions CI.

@Abacn
Copy link
Contributor

Abacn commented Feb 23, 2023

Run Python3.10 PostCommit

@aromanenko-dev
Copy link
Contributor Author

CC: @mosche @Abacn @sclukas77

mosche
mosche requested changes Feb 23, 2023
View reviewed changes
Copy link
Member

@mosche mosche left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@aromanenko-dev pls have a look, this changed the general behavior in unintended ways.

...va/org/apache/beam/sdk/extensions/schemaio/expansion/ExternalSchemaIOTransformRegistrar.java Outdated Show resolved Hide resolved
...va/org/apache/beam/sdk/extensions/schemaio/expansion/ExternalSchemaIOTransformRegistrar.java Show resolved Hide resolved
...va/org/apache/beam/sdk/extensions/schemaio/expansion/ExternalSchemaIOTransformRegistrar.java Outdated Show resolved Hide resolved
@github-actions
Copy link
Contributor

Assigning reviewers. If you would like to opt out of this review, comment assign to next reviewer:

R: @kennknowles for label java.

Available commands:

  • stop reviewer notifications - opt out of the automated review tooling
  • remind me after tests pass - tag the comment author after tests pass
  • waiting on author - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)

The PR bot will only process comments in the main thread (not review comments).

@aromanenko-dev aromanenko-dev force-pushed the avro_25601_python_postcommit branch from 50a7e1b to d54ac21 Compare February 23, 2023 17:11
@aromanenko-dev
Copy link
Contributor Author

Run Python 3.10 PostCommit

@aromanenko-dev aromanenko-dev force-pushed the avro_25601_python_postcommit branch from d54ac21 to 2e72129 Compare February 23, 2023 17:24
@aromanenko-dev
Copy link
Contributor Author

Run Python 3.10 PostCommit

@aromanenko-dev aromanenko-dev force-pushed the avro_25601_python_postcommit branch from 2e72129 to c2de6fb Compare February 23, 2023 17:44
@aromanenko-dev
Copy link
Contributor Author

Run Python 3.10 PostCommit

@aromanenko-dev
Copy link
Contributor Author

retest this please

1 similar comment
@aromanenko-dev
Copy link
Contributor Author

retest this please

@aromanenko-dev
Copy link
Contributor Author

Run Python 3.10 PostCommit

@aromanenko-dev aromanenko-dev force-pushed the avro_25601_python_postcommit branch from c2de6fb to f541656 Compare February 23, 2023 18:51
@mosche
Copy link
Member

mosche commented Feb 23, 2023

Run Python 3.10 PostCommit

mosche
mosche approved these changes Feb 23, 2023
View reviewed changes
Copy link
Member

@mosche mosche left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 This should fix the issue ...

While this adds a bit of code duplication, the actual smell isn't related to this PR in my opinion. Generating (versioned) red / write urns in the registrar seems wrong, these should be defined by the respective provider.

@damccorm
Copy link
Contributor

Run Java_Examples_Dataflow PreCommit

@damccorm
Copy link
Contributor

Run Java_Hadoop_IO_Direct PreCommit

@damccorm
Copy link
Contributor

Rerunning flakes impacted by jenkins flakiness (underlying errors were Jenkins errors, not test issues)

@aromanenko-dev aromanenko-dev mentioned this pull request Feb 23, 2023
Closed
15 tasks
@damccorm
Copy link
Contributor

Run Java_Examples_Dataflow PreCommit

@damccorm
Copy link
Contributor

Run Java_GCP_IO_Direct PreCommit

@Abacn
Copy link
Contributor

Abacn commented Feb 23, 2023

the affected test passed (seen from log of Python3.10 postcommit):

14:30:45 PASSED                                                                   [ 75%]
14:30:45 apache_beam/io/external/xlang_jdbcio_it_test.py::CrossLanguageJdbcIOTest::test_xlang_jdbc_write_read_1_mysql 
14:30:45 -------------------------------- live log call ---------------------------------
14:30:45 INFO     testcontainers.core.container:container.py:53 Pulling image mysql:latest
14:30:47 INFO     testcontainers.core.container:container.py:64 Container started: 22da3809393d
14:30:47 INFO     testcontainers.core.waiting_utils:waiting_utils.py:46 Waiting to be ready...
14:30:47 INFO     testcontainers.core.waiting_utils:waiting_utils.py:46 Waiting to be ready...
14:30:48 INFO     testcontainers.core.waiting_utils:waiting_utils.py:46 Waiting to be ready...
14:30:49 INFO     testcontainers.core.waiting_utils:waiting_utils.py:46 Waiting to be ready...
14:30:50 INFO     testcontainers.core.waiting_utils:waiting_utils.py:46 Waiting to be ready...
14:30:51 INFO     testcontainers.core.waiting_utils:waiting_utils.py:46 Waiting to be ready...
14:30:52 INFO     testcontainers.core.waiting_utils:waiting_utils.py:46 Waiting to be ready...
14:30:53 INFO     testcontainers.core.waiting_utils:waiting_utils.py:46 Waiting to be ready...
14:30:54 INFO     testcontainers.core.waiting_utils:waiting_utils.py:46 Waiting to be ready...
14:30:55 INFO     testcontainers.core.waiting_utils:waiting_utils.py:46 Waiting to be ready...
14:30:56 INFO     testcontainers.core.waiting_utils:waiting_utils.py:46 Waiting to be ready...
14:30:57 INFO     testcontainers.core.waiting_utils:waiting_utils.py:46 Waiting to be ready...
14:30:58

@damccorm damccorm merged commit dcfaddd into apache:master Feb 24, 2023
damccorm pushed a commit that referenced this pull request Feb 24, 2023
@aromanenko-dev @damccorm
Prioritise Avro providers from "extensions/core" (#25611)
5f87067
@damccorm damccorm mentioned this pull request Feb 24, 2023
Merged
3 tasks
aromanenko-dev added a commit that referenced this pull request Feb 24, 2023
@aromanenko-dev
Merge pull request #25618: Cherry Pick: Prioritise Avro providers fro…
8414cad 
…m "extensions/core" (#25611)
ruslan-ikhsan pushed a commit to akvelon/beam that referenced this pull request Mar 10, 2023
@aromanenko-dev @ruslan-ikhsan
Prioritise Avro providers from "extensions/core" (apache#25611)
137b53e
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@mosche mosche mosche approved these changes

Assignees
No one assigned
Labels
extensions java Next Action: Reviewers
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[Failing Test]: Python PostCommit failing due to duplicate AvroSchemaIO autoservice
4 participants
@aromanenko-dev @Abacn @mosche @damccorm