JUnit testing support Bug fixes and Enhancements - Step 1 #4303

davidfestal · 2017-03-03T12:00:56Z

What does this PR do?

This PR provides a number of bug fixes / enhancements to the current limited JUnit testing support in Che.

More precisely:

it provides a new Tests text console view to follow in real-time the tests being run
it fixes problems encountered when trying to run tests on vertx projects:
- during the test session the working directory should be set to the project base directory, to be ensure that write access in the current directory is provided to the tested code
- during the test session the thread context class-loader should be set to the class-loader used to load / run the tested classes (which includes the tested project classpath).
it reduces the test execution time, by removing the need of running a "maven dependencies" command to build the project test classpath. Instead of that it uses the ClasspathService to retrieve the already-built classpath.

What issues does this PR fix or reference?

https://issues.jboss.org/browse/CHE-111
https://issues.jboss.org/browse/CHE-121

Changelog

JUnit testing support Bug fixes and Enhancements

Provided a new Tests text console view.
Fixed problems encountered when trying to run tests that:
- Require write access in the current directory.
- Use the thread context class-loader during test execution.
Reduced the JUnit test execution time by removing an unnecessary step.

Release Notes

(Include in release notes only if other PRs are merged in 5.5)

Important notice: This PR is the first step of series of PR that will all contribute to enhance the testing support. Fro all these steps there will probably be a single release note that would look like this:

Testing support enhancements:

Real-time notification of test execution,
Better view of the successful tests,
Navigation from the test view to the source code,
various bug fixes

Signed-off-by: davidfestal <[email protected]>

(JUnit 4 `RunListener` is a class, not an interface, so that and simple `Proxy` cannot be used. - Set the thread context classloader to the project classloader during the time of the test (required for VertX support for example) - Add test session start and stop events in the `AbstractTestListener` - Clear the `Tests` output view when a new test is started Signed-off-by: David Festal <[email protected]>

... to retrieve the Java project classpath. Signed-off-by: David Festal <[email protected]>

This avoids blinking each time a line is added. Signed-off-by: David Festal <[email protected]>

This is necessary for all the tests that, usually started with `mvn`, assume they are started in the project root directory, and have all write permissions on the current directory. Without this commit Vertx tests fail with an `IllegateStateException` because Vertx cannot create its cache directory in the current working directory (see https://github.com/mlabuda/vertx-with-che for a vertX test project). Signed-off-by: David Festal <[email protected]>

in order to build the classpath required to run tests. This is much faster. Signed-off-by: David Festal <[email protected]>

Signed-off-by: David Festal <[email protected]>

codenvy-ci · 2017-03-03T12:04:00Z

Can one of the admins verify this patch?

skabashnyuk · 2017-03-03T12:09:46Z

plugins/plugin-testing-java/plugin-testing-junit/che-plugin-testing-junit-server/pom.xml

@@ -33,6 +33,10 @@
            <groupId>javax.inject</groupId>
            <artifactId>javax.inject</artifactId>
        </dependency>
+		<dependency>


skabashnyuk · 2017-03-03T12:09:49Z

plugins/plugin-testing-java/plugin-testing-junit/che-plugin-testing-junit-server/pom.xml

@@ -57,5 +61,23 @@
            <groupId>org.eclipse.che.plugin</groupId>
            <artifactId>che-plugin-testing-classpath-server</artifactId>
        </dependency>
+		<dependency>


skabashnyuk · 2017-03-03T12:10:13Z

...-junit-server/src/main/java/org/eclipse/che/plugin/testing/junit/server/JUnitTestRunner.java

+    	        return false;
+    	    }
+    	});
+		Class<?> c = f.createClass();


format class

skabashnyuk · 2017-03-03T12:10:27Z

...-junit-server/src/main/java/org/eclipse/che/plugin/testing/junit/server/JUnitTestRunner.java

+    	        return false;
+    	    }
+    	});
+		Class<?> c = f.createClass();


format class

skabashnyuk · 2017-03-03T12:10:36Z

...src/main/java/org/eclipse/che/plugin/testing/junit/server/listener/AbstractTestListener.java

+		getOrCreateTestSummary(testKey).addError();
+		addedError(testKey, throwable);
+	}
+}


skabashnyuk · 2017-03-03T12:10:57Z

...r/src/main/java/org/eclipse/che/plugin/testing/junit/server/listener/OutputTestListener.java

+			consumer.consume(DtoFactory.cloneDto(new TestingOutputImpl(line, lineType)));
+		} catch (IOException e) {
+			// TODO Auto-generated catch block
+			e.printStackTrace();


skabashnyuk · 2017-03-03T12:11:07Z

plugins/plugin-testing/che-plugin-testing-ide/pom.xml

+		<dependency>
+		  <groupId>org.eclipse.che.plugin</groupId>
+		  <artifactId>che-plugin-machine-ext-client</artifactId>
+		</dependency>


skabashnyuk · 2017-03-03T12:11:19Z

wsagent/che-core-api-testing-shared/pom.xml

@@ -37,5 +37,17 @@
                <directory>src/main/java</directory>
            </resource>
        </resources>
+		<plugins>


skabashnyuk · 2017-03-03T12:11:29Z

wsagent/che-core-api-testing/pom.xml

@@ -111,6 +111,15 @@
            </resource>
        </resources>
        <plugins>
+		    <plugin>


benoitf · 2017-03-03T12:58:45Z

ci-build

benoitf · 2017-03-03T13:00:10Z

...n/java/org/eclipse/che/plugin/testing/classpath/maven/server/MavenTestClasspathProvider.java

+    	return toResolvedClassPath(classpathService.getClasspath(relativeProject).stream())
+		.map(dto -> {
+			try {
+				String dtoPath = dto.getPath();


it seems there is usage of tab characters and indent is not correct

benoitf · 2017-03-03T13:01:11Z

...-api-testing/src/main/java/org/eclipse/che/api/testing/server/handler/TestingOutputImpl.java

+		this.lineType = lineType;
+	}
+
+	private String output;


formatting looks odd

davidfestal · 2017-03-03T13:23:13Z

Thanks for these comments @benoitf @skabashnyuk. And sorry for all the formatting problems, that probably mostly come from some bad tab/space eclipse settings. I'll fix all this quickly.

benoitf · 2017-03-03T13:23:41Z

@davidfestal thx

codenvy-ci · 2017-03-03T14:24:40Z

Build finished.
Build success. $BUILD_URL

codenvy-ci · 2017-03-03T14:24:40Z

Build success. https://ci.codenvycorp.com/job/che-pullrequests-build/2096/

Signed-off-by: David Festal <[email protected]>

davidfestal · 2017-03-03T15:28:07Z

@benoitf @skabashnyuk I fixed the requested changes.

benoitf · 2017-03-03T15:29:21Z

...n/java/org/eclipse/che/plugin/testing/classpath/maven/server/MavenTestClasspathProvider.java

+
+	@Inject
+	public MavenTestClasspathProvider(ClasspathService classpathService) {
+		this.classpathService = classpathService;


@davidfestal I still see here a tab character ?

benoitf · 2017-03-03T15:29:52Z

...n/java/org/eclipse/che/plugin/testing/classpath/maven/server/MavenTestClasspathProvider.java

+    private Stream<ClasspathEntryDto> toResolvedClassPath(Stream<ClasspathEntryDto> rawClasspath) {
+    	return rawClasspath.flatMap(dto -> {
+    		if (dto.getEntryKind() == IClasspathEntry.CPE_CONTAINER) {
+    			return toResolvedClassPath(dto.getExpandedEntries().stream());


and there. so I'm not sure it's up-to-date ?

Signed-off-by: David Festal <[email protected]>

davidfestal · 2017-03-03T16:34:19Z

@benoitf : this time there shouldn't be any tab any more in all the projects I touched !

benoitf · 2017-03-03T16:37:35Z

...src/main/java/org/eclipse/che/plugin/testing/junit/ide/action/RunClassContextTestAction.java

+    private final TestServiceClient   service;
+    private final AppContext          appContext;
+    private final SelectionAgent      selectionAgent;
+    private RunTestActionDelegate     delegate;


is there a reason why this delegate is not a final variable ?

benoitf · 2017-03-03T16:38:18Z

...it-ide/src/main/java/org/eclipse/che/plugin/testing/junit/ide/action/RunClassTestAction.java

    private final TestResultPresenter presenter;
-    private final TestServiceClient service;
+    private final TestServiceClient   service;
+    private RunTestActionDelegate     delegate;


same question there

benoitf · 2017-03-03T16:39:09Z

...ide/src/main/java/org/eclipse/che/plugin/testing/junit/ide/action/RunTestActionDelegate.java

+    }
+
+    public RunTestActionDelegate(
+                                 Source source) {


it's the formatter that has done this newline break ?

benoitf · 2017-03-03T16:40:05Z

plugins/plugin-testing-java/plugin-testing-junit/che-plugin-testing-junit-server/pom.xml

+                <artifactId>license-maven-plugin</artifactId>
+                <configuration>
+                    <excludes>
+                        <exclude>**/**/AbstractTestListener.java</exclude>


why do we skip these files from the license plugin ?

In the new files I created, I added the copyright in the name of RedHat, Inc, and the license-checking maven plugin doesn't like that.

Signed-off-by: David Festal <[email protected]>

skabashnyuk · 2017-03-04T10:58:17Z

I don't know. May be @davidfestal can explain.

bmicklea · 2017-03-04T18:12:04Z

@davidfestal do you expect the other PRs will be ready in the next week? Or will you need more time?

davidfestal · 2017-03-04T18:28:20Z

The idea is to have most of the other PRs ready at the end of next week yes.

TylerJewell · 2017-03-04T18:49:20Z

@davidfestal - it would be nice to be merge-ready by Thursday COB. We try to close the release every Wednesday, but over the past two releases we have slipped 1-2 days as there are last minute PRs that merge on Monday or Tuesday, and then break the builds, forcing another QA cycle.

A Thursday night merge gives plenty of time to resolve any outstanding issues in time for the following Wednesday release.

Signed-off-by: David Festal <[email protected]>

davidfestal · 2017-03-06T20:04:10Z

I understand the idea @TylerJewell. So in this case, I assume I could simply add my next commits to this PR until Thursday night, so that they can be reviewed as soon as they are added, and we will be ready to merge on Friday ?

TylerJewell · 2017-03-06T20:08:03Z

Yes. Or if you miss the 5.5 release window, you just merge into the next cycle and it will be part of 5.6 two weeks later.

Signed-off-by: David Festal <[email protected]>

... sending the build output to the a command view, and execute the tests only if compilation succeeded. Signed-off-by: David Festal <[email protected]>

Signed-off-by: David Festal <[email protected]>

This includes some utility mocking classes that could be useful in other contexts. Signed-off-by: David Festal <[email protected]>

skabashnyuk

@davidfestal any plans to finalize this PR so we will be able to made final review?

skabashnyuk · 2017-03-09T20:01:58Z

...-ext-lang-server/src/main/java/org/eclipse/che/plugin/java/server/rest/ClasspathService.java

 /**
 * Service for getting information about classpath.
 *
 * @author Valeriy Svydenko
 */
 @Path("java/classpath/")
-public class ClasspathService {
+public class ClasspathService implements ClasspathServiceInterface {



What is the purpose of interface for rest service with such weird name?

The ClasspathService component is now also used from the server-side class org.eclipse.che.plugin.testing.classpath.maven.server.MavenTestClasspathProvider.

And when implementing the test for MavenTestClasspathProvider, mocking ClasspathService directly brought classpath problems, while mocking the implemented interface works perfectly.

see https://github.com/davidfestal/che/blob/5a698ac9aed88f54c6eb98372db493ef39b3ec8b/plugins/plugin-testing-java/plugin-testing-classpath/che-plugin-testing-classpath-maven-server/src/test/java/org/eclipse/che/plugin/testing/classpath/maven/server/MavenTestClasspathProviderTest.java#L46

I assume you've added this interface for testing purposes.
From our experience this approche has some disadvantages.

Inheritance in Rest services is quite unusual and we are trying to avoid it in our code.

Name of interface is syntetic, it show that it was made for some reason, but not for real inheritance.

Mockitng of rest service is quite rare too.

If you do realy want to test Rest service I recomend you to take a look on @listeners(value = {EverrestJetty.class, MockitoTestNGListener.class})
For example
https://github.com/eclipse/che/blob/master/wsmaster/che-core-api-user/src/test/java/org/eclipse/che/api/user/server/UserServiceTest.java
https://github.com/eclipse/che/blob/master/wsmaster/che-core-api-workspace/src/test/java/org/eclipse/che/api/workspace/server/WorkspaceServiceTest.java

Yes, I added this for testing purposes. The fact is that this rest service was not used by sever-side components until now. It was only used through ClasspathServiceClient which has not exactly the same method signature (it returns a client promise).

So probably there should be a singleton class, say ClasspathProvider available for injection in server-side components, and used by the ClasspathService rest service.

So do you want me to work on removing ClasspathServiceInterface now in this PR, or can it wait for the next PR ?
In other words does it prevent you to approve the merge of this PR or not ?

Yes, I do want to remove ClasspathServiceInterface. But if you think it'll take significant time you can do that in next PR.

skabashnyuk · 2017-03-09T20:03:49Z

.../che-plugin-testing-ide/src/test/java/org/eclipse/che/plugin/testing/ide/FunctionAnswer.java

+        ArgumentType arg = (ArgumentType)invocation.getArguments()[0];
+        return apply.apply(arg);
+    }
+}


at the end of the file ?

yes, Github has nice icon to show that,

new line added

skabashnyuk · 2017-03-09T20:03:59Z

.../che-plugin-testing-ide/src/test/java/org/eclipse/che/plugin/testing/ide/MockitoPrinter.java

+            }
+        }).filter(field -> field != null).toArray(Object[]::new));
+    }
+}


at the end of the file ?

new line added

skabashnyuk · 2017-03-09T20:04:35Z

.../che-plugin-testing-ide/src/test/java/org/eclipse/che/plugin/testing/ide/MockitoPrinter.java

+ * @author David Festal
+ */
+public interface MockitoPrinter {
+    default void printInvocationsOnAllMockedFields() {


this interface look wired. How it will be used?

This interface is used this way:

public class TestClass implements MockitoPrinter { @Mock private aMock1; @Mock private aMock2; ... @Test public void testFunction() { .... // print all mock invocations called during the test on // mocked fieldsfor debugging and test design purposes printInvocationsOnAllMockedFields(); } }

Ok. I've missed the fact that is part of the tests

skabashnyuk · 2017-03-09T20:04:53Z

...g/che-plugin-testing-ide/src/test/java/org/eclipse/che/plugin/testing/ide/PromiseMocker.java

+        }));
+        return this;
+    }
+}


at the end of the file ?

new line added

davidfestal · 2017-03-09T22:18:00Z

@davidfestal any plans to finalize this PR so we will be able to made final review?

@skabashnyuk: I plan to further push additional commits very soon, but since this PR is already a consistent whole, if you want I can also push next commits on a separate branch to produce a separate PR.
This way you would all be able to review this PR first.

Signed-off-by: David Festal <[email protected]>

skabashnyuk

I would like to make a review before merge.
If you are going to add more and more changes I think it would be reasonable to merge Step1 to master, and after create new pr for Step 2 . WDYT?

davidfestal · 2017-03-10T09:10:03Z

@skabashnyuk I'm OK for merging the PR now and pushing next commits in PR Step 2, sure.
Is there anything you want me to change before merging this PR (see this comment)?

skabashnyuk

We good to go.

benoitf · 2017-03-10T09:49:31Z

For merge, please use squash option

l0rd · 2017-03-10T10:34:46Z

@davidfestal build failed please have a look https://ci.codenvycorp.com/job/che-ci-master/1069/

davidfestal · 2017-03-10T11:11:49Z

Yes, it's a problem of undeclared POM transitive dependencies, that make the maven dependency analysis fail.
I already have the fix ready here but I'm waiting for the end of a local full build before submitting it in a new PR.

davidfestal · 2017-03-10T12:55:23Z

The PR is submitted: #4391

…e#4303) Provide JUnit test real-time output in a `Tests` output view

davidfestal added 7 commits March 1, 2017 19:05

Provide JUnit test real-time output in a Tests output view.

02ae5f2

Signed-off-by: davidfestal <[email protected]>

Try using the ClasspathService instead of Maven ...

a2316bf

... to retrieve the Java project classpath. Signed-off-by: David Festal <[email protected]>

Only add the Command output at test session start.

6cdbd06

This avoids blinking each time a line is added. Signed-off-by: David Festal <[email protected]>

Use ClasspathService instead of running the maven dependencies plugin

b36c5ec

in order to build the classpath required to run tests. This is much faster. Signed-off-by: David Festal <[email protected]>

Abstract the real test run in a single class used by all action classes

c7a7fc8

Signed-off-by: David Festal <[email protected]>

skabashnyuk requested changes Mar 3, 2017

View reviewed changes

benoitf reviewed Mar 3, 2017

View reviewed changes

dfestal added 3 commits March 3, 2017 16:19

Fix format

0f430e4

Signed-off-by: David Festal <[email protected]>

Call LOG instead of System.println.

239c490

Signed-off-by: David Festal <[email protected]>

Sort POMs

8780bfb

Signed-off-by: David Festal <[email protected]>

davidfestal force-pushed the master branch from 8eae786 to 8780bfb Compare March 3, 2017 15:25

benoitf reviewed Mar 3, 2017

View reviewed changes

Further fix format

3eda7ef

Signed-off-by: David Festal <[email protected]>

benoitf reviewed Mar 3, 2017

View reviewed changes

Some fields should have been made final

022a430

Signed-off-by: David Festal <[email protected]>

bmicklea added the kind/enhancement A feature request - must adhere to the feature request template. label Mar 4, 2017

bmicklea approved these changes Mar 4, 2017

View reviewed changes

JamesDrummond approved these changes Mar 5, 2017

View reviewed changes

Also pass the workspace-relative project path to the classpath provider

f87c3c7

Signed-off-by: David Festal <[email protected]>

davidfestal added 4 commits March 7, 2017 18:27

Test for the maven classpath provider

8c00de8

Signed-off-by: David Festal <[email protected]>

test-compile the project using maven before executing the tests...

d6900e6

... sending the build output to the a command view, and execute the tests only if compilation succeeded. Signed-off-by: David Festal <[email protected]>

Refactor a bit to ease the build of the tests

ba8a293

Signed-off-by: David Festal <[email protected]>

Tests for the TestServiceClient

5a698ac

This includes some utility mocking classes that could be useful in other contexts. Signed-off-by: David Festal <[email protected]>

skabashnyuk reviewed Mar 9, 2017

View reviewed changes

Add new line at the end of the file as requested by @skabashnyuk

0bf39a0

Signed-off-by: David Festal <[email protected]>

skabashnyuk requested changes Mar 10, 2017

View reviewed changes

skabashnyuk approved these changes Mar 10, 2017

View reviewed changes

l0rd merged commit 440f92d into eclipse-che:master Mar 10, 2017

davidfestal mentioned this pull request Mar 10, 2017

Add missing pom dependencies that were producing a build failure #4391

Merged

dmytro-ndp mentioned this pull request Mar 13, 2017

TestRunner plugin use wrong URL for launcning tests (regression) #4406

Closed

JamesDrummond mentioned this pull request Mar 17, 2017

5.5.0 Milestone Overview #4358

Closed

9 tasks

davidfestal mentioned this pull request Mar 20, 2017

Java Testing support Bug fixes and Enhancements - Step 2 #4481

Merged

JPinkney pushed a commit to JPinkney/che that referenced this pull request Aug 17, 2017

JUnit testing support Bug fixes and Enhancements - Step 1 (eclipse-ch…

5c15a84

…e#4303) Provide JUnit test real-time output in a `Tests` output view

JUnit testing support Bug fixes and Enhancements - Step 1 #4303

JUnit testing support Bug fixes and Enhancements - Step 1 #4303

Conversation

davidfestal commented Mar 3, 2017 • edited by JamesDrummond Loading

What does this PR do?

What issues does this PR fix or reference?

Changelog

Release Notes

codenvy-ci commented Mar 3, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

benoitf commented Mar 3, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

davidfestal commented Mar 3, 2017

benoitf commented Mar 3, 2017

codenvy-ci commented Mar 3, 2017

codenvy-ci commented Mar 3, 2017

davidfestal commented Mar 3, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

davidfestal commented Mar 3, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

skabashnyuk commented Mar 4, 2017

bmicklea commented Mar 4, 2017

davidfestal commented Mar 4, 2017

TylerJewell commented Mar 4, 2017

davidfestal commented Mar 6, 2017

TylerJewell commented Mar 6, 2017

skabashnyuk left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

davidfestal commented Mar 9, 2017

skabashnyuk left a comment

Choose a reason for hiding this comment

davidfestal commented Mar 10, 2017

skabashnyuk left a comment

Choose a reason for hiding this comment

benoitf commented Mar 10, 2017

l0rd commented Mar 10, 2017

davidfestal commented Mar 10, 2017 • edited Loading

davidfestal commented Mar 10, 2017

davidfestal commented Mar 3, 2017 •

edited by JamesDrummond

Loading

davidfestal commented Mar 10, 2017 •

edited

Loading