Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Transform] Workflow testing of continuous/batch data frame transforms #45369

Closed
hendrikmuhs opened this issue Aug 9, 2019 · 2 comments
Closed

[Transform] Workflow testing of continuous/batch data frame transforms #45369

hendrikmuhs opened this issue Aug 9, 2019 · 2 comments
Labels
:ml/Transform Transform >test Issues or PRs that are addressing/adding tests

Comments

@hendrikmuhs
Copy link

hendrikmuhs commented Aug 9, 2019 •
edited
Loading

In #45339 we discovered a bug in the workflow of data frame transforms which should have been covered by tests. The tricky part here is that we can not simulate this in an integration tests. In order to cover this we need a mocked test environment where we can set breakpoints to test for certain conditions, similar to https://github.com/elastic/elasticsearch/blob/master/x-pack/plugin/core/src/test/java/org/elasticsearch/xpack/core/indexing/AsyncTwoPhaseIndexerTests.java
@hendrikmuhs hendrikmuhs added >test Issues or PRs that are addressing/adding tests :ml/Transform Transform labels Aug 9, 2019
@elasticmachine
Copy link
Collaborator

Pinging @elastic/ml-core

@hendrikmuhs hendrikmuhs changed the title [DataFrame] Workflow testing of continuous/batch data frame transforms [Transform] Workflow testing of continuous/batch data frame transforms Oct 25, 2019
@hendrikmuhs hendrikmuhs mentioned this issue Oct 28, 2019
Merged
hendrikmuhs pushed a commit that referenced this issue Nov 1, 2019
[Transform] decouple task and indexer (#48567)
56731d8 
decouple TransformTask and ClientTransformIndexer. Interaction between the 2 classes are now moved into a context class which holds shared information.

relates #45369
hendrikmuhs pushed a commit to hendrikmuhs/elasticsearch that referenced this issue Nov 1, 2019
[Transform] decouple task and indexer (elastic#48567)
f56aad4 
decouple TransformTask and ClientTransformIndexer. Interaction between the 2 classes are now moved into a context class which holds shared information.

relates elastic#45369
@hendrikmuhs hendrikmuhs mentioned this issue Nov 1, 2019
Merged
hendrikmuhs pushed a commit that referenced this issue Nov 1, 2019
[7.x][Transform] decouple task and indexer (#48812)
5ecde37 
decouple TransformTask and ClientTransformIndexer. Interaction between the 2 classes are
now moved into a context class which holds shared information.

relates #45369
@hendrikmuhs
Copy link
Author

Improvement have been made with #62086 and #60469, closing this issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
:ml/Transform Transform >test Issues or PRs that are addressing/adding tests
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@hendrikmuhs @elasticmachine