Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Restrict which tasks can use testclusters #45198

Merged
merged 11 commits into from
Aug 9, 2019

Conversation

alpar-t
Copy link
Contributor

@alpar-t alpar-t commented Aug 5, 2019

This PR fixes a problem between the interaction of test-clusters and
build cache.
Before this any task could have used a cluster without tracking it as
input.
With this change a new interface is introduced to track the tasks that
can use clusters and we do consider the cluster as input for all of
them.

This PR fixes a problem between the interaction of test-clusters and
build cache.
Before this any task could have used a cluster without tracking it as
input.
With this change a new interface is introduced to track the tasks that
can use clusters and we do consider the cluster as input for all of
them.
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-core-infra

@mark-vieira
Copy link
Contributor

mark-vieira commented Aug 5, 2019

Before this any task could have used a cluster without tracking it as input.

Question: Do we have any examples of this actually happening? Also, keep in mind, it only actually matters if that task was also being cached.

Copy link
Contributor

@mark-vieira mark-vieira left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a couple comments, otherwise 👍

Good catch on fixing all those dependencies mistakenly put on the rest test task instead of the runner. I'm surprised this hasn't caused more issues with incorrect task dependencies.

@@ -123,7 +120,7 @@ private void createListClustersTask(Project project, NamedDomainObjectContainer<
private void createUseClusterTaskExtension(Project project, NamedDomainObjectContainer<ElasticsearchCluster> container) {
// register an extension for all current and future tasks, so that any task can declare that it wants to use a
// specific cluster.
project.getTasks().all((Task task) ->
project.getTasks().withType(TestClustersTask.class, (Task task) ->
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should try and align these two methods somehow at some point. That is, calling testCluster or useCluster would be all that is needed to register all the necessary wiring here. We could something like have the testCluster method on TestClustersTask reach out and grab the test clusters plugin and perform the registration.

We don't have to do this as part of this PR though.

qa/wildfly/build.gradle Show resolved Hide resolved
@alpar-t
Copy link
Contributor Author

alpar-t commented Aug 6, 2019

@elasticmachine run elasticsearch-ci/default-distro

@alpar-t
Copy link
Contributor Author

alpar-t commented Aug 8, 2019

@elasticmachine run elasticsearch-ci/2

Copy link
Contributor

@mark-vieira mark-vieira left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall this looks good and separating the registry actually makes this a lot easier to understand. Just one comment regarding method parameter naming.

}

public void maybeStartCluster(ElasticsearchCluster cluster) {
// we only start the cluster before the actions, so we'll not start it if the task is up-to-date
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This comment doesn't make much sense in this context. We should move it to where we define the TaskActionListener.

}

public void stopCluster(ElasticsearchCluster cluster, boolean taskFailed) {
if (taskFailed) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This condition is backwards, it just so happens to work because we are also inverting the taskFailed predicate at the call site.

});
}
((TestClustersAware) task).getClusters()
.forEach(cluster -> registry.stopCluster(cluster, state.getFailure() == null));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are passing the wrong value for taskFailed here. When getFailure() == null it means the task was successful. We need to either invert all this or rename the taskFailed param on stopCluster() to taskSucceeded or something so this all makes sense.

@alpar-t alpar-t merged commit 7397734 into elastic:master Aug 9, 2019
alpar-t added a commit that referenced this pull request Aug 9, 2019
* Restrict which tasks can use testclusters

This PR fixes a problem between the interaction of test-clusters and
build cache.
Before this any task could have used a cluster without tracking it as
input.
With this change a new interface is introduced to track the tasks that
can use clusters and we do consider the cluster as input for all of
them.
@alpar-t alpar-t deleted the custom-type-test-clusters branch November 11, 2019 09:41
@mark-vieira mark-vieira added the Team:Delivery Meta label for Delivery team label Nov 11, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Delivery/Build Build or test infrastructure >non-issue Team:Delivery Meta label for Delivery team v7.4.0 v8.0.0-alpha1
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants