Print done message from test framework to stdout
#2993
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
We are returning the doneMessage from the test framework as value of the test task, but we are not printing it. Moreover, the output is shortcircuited when a task fails, so it's not passed to the `test` command. This PR uses `ctx.log.outputStream.println(doneMessage)` to print the message before returning it so users can clearly see it in case of both success and failure. This was discovered because of the way weaver works on Scala.js Since the `TestRunnerTests` are on `scalalib` and can't use Scala.js code, I implemented two dummy `sbt.testing.Framework`s which just return a fixed `done` message.
lolgab
force-pushed
the
print-doneMessage-tests
branch
from
ceade99 to
0da3f05
Compare
lolgab
force-pushed
the
print-doneMessage-tests
branch
from
3e6aea0 to
b3c0473
Compare
lefou
approved these changes
Jan 28, 2024
|
Nice fix @lolgab! I've been wondering for years where the lihaoyi mill$ ./mill -i main.util.test
...
-------------------------------- Running Tests --------------------------------
+ mill.util.VersionTests.ordering.ignoreQualifier 9ms
+ mill.util.VersionTests.ordering.maven 2ms
+ mill.util.VersionTests.ordering.osgi 1ms
+ mill.util.VersionTests.Version.isAtLeast(String, String).ignoreQualifier 0ms
-------------------------------- Running Tests --------------------------------
+ mill.util.JvmTests.createClasspathPassingJar 50ms
-------------------------------- Running Tests --------------------------------
+ mill.util.TarjanTests.empty 2ms
+ mill.util.TarjanTests.singleton 1ms
+ mill.util.TarjanTests.selfCycle 0ms
+ mill.util.TarjanTests.simpleCycle 0ms
+ mill.util.TarjanTests.multipleSingletons 0ms
+ mill.util.TarjanTests.straightLineNoCycles 0ms
+ mill.util.TarjanTests.straightLineNoCyclesReversed 0ms
+ mill.util.TarjanTests.independentSimpleCycles 0ms
+ mill.util.TarjanTests.independentLinkedCycles 0ms
+ mill.util.TarjanTests.independentLinkedCycles2 0ms
+ mill.util.TarjanTests.combinedCycles 0ms
+ mill.util.TarjanTests.combinedCycles 0ms
lihaoyi mill$ /Users/lihaoyi/.cache/mill/download/0.11.6-68-3036bd -i main.util.test
...
-------------------------------- Running Tests --------------------------------
+ mill.util.VersionTests.ordering.ignoreQualifier 9ms
+ mill.util.VersionTests.ordering.maven 2ms
+ mill.util.VersionTests.ordering.osgi 1ms
+ mill.util.VersionTests.Version.isAtLeast(String, String).ignoreQualifier 0ms
-------------------------------- Running Tests --------------------------------
+ mill.util.JvmTests.createClasspathPassingJar 51ms
-------------------------------- Running Tests --------------------------------
+ mill.util.TarjanTests.empty 2ms
+ mill.util.TarjanTests.singleton 1ms
+ mill.util.TarjanTests.selfCycle 0ms
+ mill.util.TarjanTests.simpleCycle 0ms
+ mill.util.TarjanTests.multipleSingletons 0ms
+ mill.util.TarjanTests.straightLineNoCycles 0ms
+ mill.util.TarjanTests.straightLineNoCyclesReversed 0ms
+ mill.util.TarjanTests.independentSimpleCycles 0ms
+ mill.util.TarjanTests.independentLinkedCycles 0ms
+ mill.util.TarjanTests.independentLinkedCycles2 0ms
+ mill.util.TarjanTests.combinedCycles 0ms
+ mill.util.TarjanTests.combinedCycles 0ms
Tests: 17, Passed: 17, Failed: 0 |
|
Also the nice test summaries when you run a target with a large number of tests are back! /Users/lihaoyi/.cache/mill/download/0.11.6-68-3036bd -i scalalib.test
...
----------------------------------- Results -----------------------------------
- mill.scalalib.HelloWorldTests 137453ms
- scalaVersion 15ms
+ fromBuild 10ms
+ override 5ms
- scalacOptions 8ms
+ emptyByDefault 3ms
+ override 5ms
- scalaDocOptions 5297ms
+ emptyByDefault 10ms
+ override 5ms
+ extend 11ms
+ docJarWithTitle 2627ms
+ docJarWithVersion 1321ms
+ docJarOnlyVersion 1323ms
- scalacPluginClasspath 14ms
+ withMacroParadise 14ms
- scalaDocPluginClasspath 13ms
+ extend 13ms
- compile 18040ms
+ fromScratch 2163ms
+ nonPreCompiledBridge 8864ms
+ recompileOnChange 3003ms
+ failOnError 2609ms
+ passScalacOptions 1401ms
- semanticDbData 1891ms
+ fromScratch 1891ms
+ artifactNameCross 10ms
+ scala-33-depend-on-scala-32-works 1ms List(stable[3.2.0])
+ scala-213-depend-on-scala-212-fails 0ms
- runMain 15116ms
+ runMainObject 4644ms
- runCross 7020ms
+ v210 0ms *** Disabled in Java 9+ - Reason: Scala 2.10 tests don't work with Java 9+ ***
+ v211 0ms *** Disabled in Java 9+ - Reason: Scala 2.11 tests don't work with Java 9+ ***
+ v2123 2577ms
+ v2124 2497ms
+ v2131 1946ms
+ notRunInvalidMainObject 2450ms
+ notRunWhenCompileFailed 1002ms
- forkRun 10749ms
+ runIfMainClassProvided 4625ms
+ notRunWithoutMainClass 1377ms
+ runDiscoverMainClass 4747ms
- run 7195ms
+ runIfMainClassProvided 2928ms
+ runWithDefaultMain 2796ms
+ notRunWithoutMainClass 1471ms
- jar 5584ms
+ nonEmpty 2760ms
+ logOutputToFile 2824ms
- assembly 53901ms
+ assembly 3312ms
- assemblyRules 45295ms
+ appendWithDeps 4288ms
+ appendMultiModule 3284ms
+ appendPatternWithDeps 3716ms
+ appendPatternMultiModule 2580ms
+ appendPatternMultiModuleWithSeparator 2653ms
+ excludeWithDeps 3435ms
+ excludeMultiModule 2623ms
+ excludePatternWithDeps 3652ms
+ excludePatternMultiModule 2802ms
- relocate 9313ms
+ withDeps 6679ms
+ run 2634ms
- writeDownstreamWhenNoRule 6949ms
+ withDeps 3845ms
+ multiModule 3104ms
+ run 5294ms
+ ivyDeps 500ms
+ typeLevel 772ms
- macros 4350ms
- scala-2.12 2ms
+ runMain 1ms skipped on Java 17+
+ docJar 1ms skipped on Java 17+
- scala-2.13 4348ms
+ runMain 1906ms
+ docJar 2442ms
- flags 5030ms
+ runMain 2341ms
+ docJar 2689ms
+ color-output 1026ms
+ scalacheck 3650ms
+ dotty213 1757ms
- pom 44ms
+ should include scala-library dependency 28ms
+ versionScheme 16ms
- publish 10ms
+ should retrieve credentials from environment variables if direct argument is empty 4ms
+ should prefer direct argument as credentials over environment variables 3ms
+ should throw exception if neither environment variables or direct argument were not passed 3ms
- ivy 15ms
+ should include scala-library dependency 15ms
+ replAmmoniteMainClass 8ms
- validated 51ms
- PathRef 10ms
+ unchecked 6ms
+ checked 4ms
- SeqPathRef 9ms
+ unchecked 5ms
+ checked 4ms
- AggPathRef 8ms
+ unchecked 4ms
+ checked 4ms
- other 24ms
+ unchecked 4ms
+ checked 20ms
- multiModuleClasspaths 2406ms
+ modMod 819ms
+ modCompile 799ms
+ compileMod 788ms
- mill.scalalib.TestClassLoaderTests 2455ms
+ com.sun classes exist in tests classpath (Java 8 only) 2455ms
- mill.scalalib.DottyDocTests 8074ms
+ static 2684ms
+ empty 2627ms
+ multiple 2763ms
- mill.scalalib.scalafmt.ScalafmtTests 2059ms
- scalafmt 2059ms
+ reformat 1159ms
+ reformatAll 900ms
- mill.scalalib.CrossVersionTests 1608ms
+ StandaloneScala213 64ms
+ JavaDependsOnScala213 399ms
+ Scala3DependsOnScala213 365ms
+ JavaDependsOnScala3 417ms
+ sndwitch3 363ms
- mill.scalalib.ResolveDepsTests 2410ms
+ resolveValidDeps 4ms
+ resolveValidDepsWithClassifier 1ms
+ resolveTransitiveRuntimeDeps 4ms
+ excludeTransitiveDeps 2ms
+ excludeTransitiveDepsByOrg 1ms
+ excludeTransitiveDepsByName 2ms
+ errOnInvalidOrgDeps 1366ms
+ errOnInvalidVersionDeps 518ms
+ errOnPartialSuccess 512ms
- mill.scalalib.TestRunnerTests 21744ms
- TestRunner 21744ms
- utest 10970ms
+ test case lookup 2179ms
- testOnly 8791ms
+ suffix 2167ms
+ prefix 2143ms
+ exactly 2299ms
+ multi 2182ms
- doneMessage 5834ms
+ failure 2051ms
+ success 1908ms
+ null 1875ms
- ScalaTest 2209ms
+ scalatest.test 2209ms
- ZioTest 2731ms
+ ziotest.test 2731ms
- mill.scalalib.ScalaVersionsRangesTests 21586ms
+ main with Scala 2.12- and 2.13+ specific code 10140ms List((), (), (), ())
+ test with Scala 2.12- and 2.13+ specific code 11446ms List((), (), (), ())
- mill.scalalib.VersionContolTests 0ms
+ github 0ms
+ git 0ms
+ svn 0ms
- mill.scalalib.api.ZincWorkerUtilTests 1ms
+ matchingVersions 0ms
+ versionRanges 1ms
- mill.scalalib.publish.IvyTests 9ms
- fullIvy 9ms
+ topLevel 4ms
+ publications 2ms
+ dependencies 3ms
- mill.scalalib.publish.LocalM2PublisherTests 7ms
+ Publish copies artifact files 5ms
+ Publish overwrites existing artifact files 2ms
- mill.scalalib.publish.PomTests 50ms
- fullPom 40ms
+ topLevel 27ms
+ licenses 3ms
+ scm 3ms
+ developers 2ms
+ dependencies 5ms
- pomEmptyScm 2ms
+ scm 2ms
- pomNoLicenses 1ms
+ licenses 1ms
- pomNoDeps 1ms
+ dependencies 1ms
- pomNoDevelopers 4ms
+ developers 4ms
+ pomProperties 2ms
<properties xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<myVersion>1.0</myVersion>
<scala.version>2.13.7</scala.version>
</properties>
- mill.scalalib.HelloJavaTests 6306ms
+ compile 774ms
+ semanticDbData 857ms
- docJar 1243ms
+ withoutArgsFile 428ms
+ withArgsFile 815ms
+ test 2157ms
+ failures 1275ms
- mill.scalalib.bsp.BspModuleTests 2815ms
- bspCompileClasspath 2815ms
- single module 2815ms
+ dependent module 72ms
- interdependencies are fast 2743ms
- reference (no BSP) 2318ms
+ index 1 (no deps) 641ms 612 msec
+ index 10 760ms 738 msec
+ index 15 917ms 875 msec
+ index 1 (no deps) 74ms 14 msec
+ index 10 142ms 124 msec
+ index 15 209ms 180 msec
- mill.scalalib.dependency.metadata.MetadataLoaderFactoryTests 11ms
+ mavenRepository 1ms
+ ivyRepository 10ms
+ otherRepository 0ms
- mill.scalalib.dependency.updates.UpdatesFinderTests 38ms
- snapshotArtifacts 11ms
+ noOldStableVersions 6ms
+ noOldMilestones 1ms
+ noOldSnapshots 1ms
+ noCurrentMilestones 1ms
+ noCurrentSnapshot 0ms
+ stableUpdates 0ms
+ milestoneUpdates 1ms
+ snapshotUpdates 0ms
+ noDifferencesRegardingOptionalPreReleases 1ms
- milestoneArtifacts 15ms
+ noOldStableVersions 1ms
+ noOldSnapshots 8ms
+ noOldMilestones 1ms
+ noCurrentSnapshot 1ms
+ currentMilestones 1ms
+ stableUpdates 1ms
+ noSnapshotUpdates 1ms
+ milestoneUpdates 0ms
+ noDifferencesRegardingOptionalPreReleases 1ms
- stableArtifacts 12ms
+ noOldStableVersions 1ms
+ noOldSnapshots 8ms
+ noOldMilestones 1ms
+ noCurrentSnapshot 1ms
+ noCurrentMilestones 1ms
+ stableUpdates 0ms
+ noSnapshotUpdates 0ms
+ noMilestoneUpdates 0ms
+ milestoneUpdatesWhenAllowingPreReleases 0ms
- mill.scalalib.dependency.versions.ScalaVersionTests 0ms
+ release 0ms
+ snapshot 0ms
+ nightly 0ms
+ dotty 0ms
- earlyscala3 0ms
+ RC 0ms
+ nightly 0ms
- scala3 0ms
+ release 0ms
+ RC 0ms
+ nightly 0ms
+ typelevel 0ms
- mill.scalalib.dependency.versions.VersionTests 11ms
- versionsClassification 5ms
+ ReleaseVersion 5ms
+ PreReleaseVersion 0ms
+ PreReleaseBuildVersion 0ms
+ BuildVersion 0ms
- short version defaults 0ms
+ 1 0ms 1
+ 1.0 0ms 1.0
+ 1.1 0ms 1.1
+ semverVersionsOrdering 6ms
- parser 0ms
+ parse 1.0.5 0ms
+ parse 1.0.M3 0ms
+ parse 1.0.3m 0ms
+ parse 1.0.3m.4 0ms
+ parse 9.1-901-1.jdbc4 0ms
+ parse 1.33.7+build/11.e0f985a 0ms
+ parse 9.1-901-1.jdbc4+build/11.e0f985a 0ms
- mill.scalalib.ScalaDoc3Tests 7311ms
+ static 2744ms
+ empty 2256ms
+ multiple 2311ms
- mill.scalalib.giter8.Giter8Tests 786ms
- init 786ms
+ hello 786ms
- mill.scalalib.CycleTests 7ms
- moduleDeps 7ms
+ self-reference 7ms
+ cycle-in-deps 0ms
- compileModuleDeps 0ms
+ self-reference 0ms
- mill.scalalib.CoursierMirrorTests 143ms
+ readMirror 143ms
Tests: 209, Passed: 209, Failed: 0 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes #2992
We are returning the doneMessage from the test framework as value of the test task, but we are not printing it. Moreover, the output is shortcircuited when a task fails, so it's not passed to the
testcommand.This PR uses
ctx.log.outputStream.println(doneMessage)to print the message before returning it so users can clearly see it in case of both success and failure.This was discovered because of the way weaver works on Scala.js Since the
TestRunnerTestsare onscalaliband can't use Scala.js code, I implemented some dummysbt.testing.Frameworks which just return a fixeddonemessage.Pull Request: #2993