Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix for 14106 cygwin failed tests #14113

Merged
merged 13 commits into from
Dec 16, 2021
Merged

Fix for 14106 cygwin failed tests #14113

merged 13 commits into from
Dec 16, 2021

Conversation

philwalk
Copy link
Contributor

[test_windows_full]
[test_non_bootstrapped]

Cygwin has an older version of coreutils and so is not compatible with /usr/bin/env -S bin/scala usage.
To fix the problem of legacy /usr/bin/env combining bin/scala with arguments passed to it, the hashbang is split apart in dist/bin/scala, similar to how it was done in scala3-3.0.2, although this variation is more flexible, as it's not limited to -classpath on the hashbang command line.

For reference, here are the hashbang lines affected by this change:

==> compiler/test-resources/scripting/argfileClasspath.sc <==
#!dist/target/pack/bin/scala @compiler/test-resources/scripting/cpArgumentsFile.txt

==> compiler/test-resources/scripting/classpathReport.sc <==
#!bin/scala -classpath 'dist/target/pack/lib/*'

==> compiler/test-resources/scripting/unglobClasspath.sc <==
#!bin/scala -classpath 'dist/target/pack/lib/*'

@philwalk philwalk mentioned this pull request Dec 14, 2021
@philwalk
Copy link
Contributor Author

philwalk commented Dec 14, 2021

The first problem in [test_windows_full] is this:

2021-12-14T19:21:44.1151367Z stderr[/cygdrive/c/Program Files (x86)/sbt/bin/sbt: line 2: $'\r': command not found]
2021-12-14T19:21:44.1152191Z stderr[/cygdrive/c/Program Files (x86)/sbt/bin/sbt: line 3: set: +]

The cause of this symptom is described here:
set-cygwin-to-ignore-crs

The fix (to be verified shortly) is to add SHELLOPTS with :igncr to the spawned bash process environment:

SHELLOPTS=igncr:braceexpand:hashall:histexpand:history:ignoreeof:interactive-comments:monitor

The next problem is a path that needs to be normalized:

stderr[/cygdrive/c/actions-runner/_work/dotty/dotty/bin/../bin/common: line 23: /cygdrive/c/actions-runner/_work/dotty/dotty/bin/../dist/target/pack/bin/scala: No such file or directory]

I'll test and push these fixes asap.

@philwalk philwalk requested a review from BarkingBad December 15, 2021 12:47
@philwalk
Copy link
Contributor Author

philwalk commented Dec 15, 2021

Some tests are failing because the test environment sometimes doesn't provide dist/target/pack/bin.
For example, line 735 of test_windows_full test logs

osname[windows 10]
uname[CYGWIN_NT-10.0]
using JAVA_HOME=C:/Program Files/AdoptOpenJDK/jdk-8.0.265.01-hotspot/
working directory is [C:/actions-runner/_work/dotty/dotty]
warning: not found below working directory: C:/actions-runner/_work/dotty/dotty/dist/target/pack/bin

@BarkingBad
Copy link
Contributor

BarkingBad commented Dec 15, 2021

Thanks @philwalk for your input. Actually, these logs are from the second attempt (which I killed manually), here are from the first one: https://github.com/lampepfl/dotty/runs/4527616578?check_suite_focus=true

Apparently, there is some deadlock/loop that blocks the tests execution (These were killed automatically after 6hours at the very same step, and these tasks takes usually 30-40 minutes to complete).

My rough guess is that it could be entering REPL? I don't know if it is possible, but maybe this is hanging the tests execution.

EDIT: I see that your latest commit got things working, though the tests are still failing, but now in a more controllable manner

@philwalk
Copy link
Contributor Author

Ah, the failure is caused by ansi colors polluting the strings returned by the scripts, causing the comparison to fail.
As a result, cwdline is incorrect.

Here's the relevant lines from the test log:

2021-12-15T12:59:51.4130671Z �[0Jcwd: /__w/dotty/dotty
2021-12-15T12:59:51.4155195Z classpath: /__w/dotty/dotty:/__w/dotty/dotty/dist/target/pack/lib/scala-library-2.13.6.jar:/__w/dotty/dotty/dist/target/pack/lib/scala3-library_3-3.1.2-RC1-bin-SNAPSHOT.jar:/__w/dotty/dotty/dist/target/pack/lib/scala-asm-9.1.0-scala-1.jar:/__w/dotty/dotty/dist/target/pack/lib/compiler-interface-1.3.5.jar:/__w/dotty/dotty/dist/target/pack/lib/scala3-interfaces-3.1.2-RC1-bin-SNAPSHOT.jar:/__w/dotty/dotty/dist/target/pack/lib/scala3-compiler_3-3.1.2-RC1-bin-SNAPSHOT.jar:/__w/dotty/dotty/dist/target/pack/lib/tasty-core_3-3.1.2-RC1-bin-SNAPSHOT.jar:/__w/dotty/dotty/dist/target/pack/lib/scala3-staging_3-3.1.2-RC1-bin-SNAPSHOT.jar:/__w/dotty/dotty/dist/target/pack/lib/scala3-tasty-inspector_3-3.1.2-RC1-bin-SNAPSHOT.jar:/__w/dotty/dotty/dist/target/pack/lib/jline-reader-3.19.0.jar:/__w/dotty/dotty/dist/target/pack/lib/jline-terminal-3.19.0.jar:/__w/dotty/dotty/dist/target/pack/lib/jline-terminal-jna-3.19.0.jar:/__w/dotty/dotty/dist/target/pack/lib/jna-5.3.1.jar:dist/target/pack/lib/ST4-4.0.7.jar:dist/target/pack/lib/antlr-3.5.1.jar:dist/target/pack/lib/antlr-runtime-3.5.1.jar:dist/target/pack/lib/autolink-0.6.0.jar:dist/target/pack/lib/compiler-interface-1.3.5.jar:dist/target/pack/lib/dist_3-3.1.2-RC1-bin-SNAPSHOT.jar:dist/target/pack/lib/flexmark-0.42.12.jar:dist/target/pack/lib/flexmark-ext-anchorlink-0.42.12.jar:dist/target/pack/lib/flexmark-ext-autolink-0.42.12.jar:dist/target/pack/lib/flexmark-ext-emoji-0.42.12.jar:dist/target/pack/lib/flexmark-ext-gfm-strikethrough-0.42.12.jar:dist/target/pack/lib/flexmark-ext-gfm-tables-0.42.12.jar:dist/target/pack/lib/flexmark-ext-gfm-tasklist-0.42.12.jar:dist/target/pack/lib/flexmark-ext-ins-0.42.12.jar:dist/target/pack/lib/flexmark-ext-superscript-0.42.12.jar:dist/target/pack/lib/flexmark-ext-tables-0.42.12.jar:dist/target/pack/lib/flexmark-ext-wikilink-0.42.12.jar:dist/target/pack/lib/flexmark-ext-yaml-front-matter-0.42.12.jar:dist/target/pack/lib/flexmark-formatter-0.42.12.jar:dist/target/pack/lib/flexmark-html-parser-0.42.12.jar:dist/target/pack/lib/flexmark-jira-converter-0.42.12.jar:dist/target/pack/lib/flexmark-util-0.42.12.jar:dist/target/pack/lib/jackson-annotations-2.2.3.jar:dist/target/pack/lib/jackson-core-2.9.8.jar:dist/target/pack/lib/jackson-databind-2.2.3.jar:dist/target/pack/lib/jackson-dataformat-yaml-2.9.8.jar:dist/target/pack/lib/jline-reader-3.19.0.jar:dist/target/pack/lib/jline-terminal-3.19.0.jar:dist/target/pack/lib/jline-terminal-jna-3.19.0.jar:dist/target/pack/lib/jna-5.3.1.jar:dist/target/pack/lib/jsoup-1.13.1.jar:dist/target/pack/lib/liqp-0.6.8.jar:dist/target/pack/lib/protobuf-java-3.7.0.jar:dist/target/pack/lib/scala-asm-9.1.0-scala-1.jar:dist/target/pack/lib/scala-library-2.13.6.jar:dist/target/pack/lib/scala3-compiler_3-3.1.2-RC1-bin-SNAPSHOT.jar:dist/target/pack/lib/scala3-interfaces-3.1.2-RC1-bin-SNAPSHOT.jar:dist/target/pack/lib/scala3-library_3-3.1.2-RC1-bin-SNAPSHOT.jar:dist/target/pack/lib/scala3-staging_3-3.1.2-RC1-bin-SNAPSHOT.jar:dist/target/pack/lib/scala3-tasty-inspector_3-3.1.2-RC1-bin-SNAPSHOT.jar:dist/target/pack/lib/scaladoc_3-3.1.2-RC1-bin-SNAPSHOT.jar:dist/target/pack/lib/snakeyaml-1.23.jar:dist/target/pack/lib/tasty-core_3-3.1.2-RC1-bin-SNAPSHOT.jar:dist/target/pack/lib/util-interface-1.3.0.jar
2021-12-15T12:59:51.4175470Z cwdline  []
2021-12-15T12:59:51.4176037Z expected[/__w/dotty/dotty]

@philwalk
Copy link
Contributor Author

I'm optimistic about #14113, it's testing now, but it turns out that in windows console bash processes, ANSI color codes were polluting STDOUT, causing string comparisons to fail.

@philwalk
Copy link
Contributor Author

philwalk commented Dec 15, 2021

The ANSI color problem seems to be fixed, so now the remaining issue has to do with system layout.

The problem with BashScriptsTests in [test_windows_full] is that scalac cannot be found. Or, stated differently, the current working directory does not have a dist/target/pack/bin directory below it.

2021-12-15T15:48:46.2090470Z stderr: [/usr/bin/bash: C:/actions-runner/_work/dotty/dotty/out/bootstrap/scala3-compiler-bootstrapped/scala-3.1.2-RC1-bin-SNAPSHOT-nonbootstrapped/test-classes/scripting/
scriptPath.sc: dist/target/pack/bin/scala: bad interpreter: No such file or directory]
2021-12-15T15:48:46.2112867Z [error] Test dotty.tools.scripting.BashScriptsTests.verifyScriptPathProperty failed: java.lang.AssertionError: assertion failed: script C:/actions-runner/_work/dotty/dotty
/out/bootstrap/scala3-compiler-bootstrapped/scala-3.1.2-RC1-bin-SNAPSHOT-nonbootstrapped/test-classes/scripting/scriptPath.sc did not report valid script.path value, took 0.062 sec
2021-12-15T15:48:46.2116080Z [error]     at scala.runtime.Scala3RunTime$.assertFailed(Scala3RunTime.scala:8)
2021-12-15T15:48:46.2118007Z [error]     at dotty.tools.scripting.BashScriptsTests.verifyScriptPathProperty(BashScriptsTests.scala:132)
2021-12-15T15:48:46.2119512Z [error]     ...
2021-12-15T15:48:48.5681039Z stderr[Picked up _JAVA_OPTIONS: -Xmx8192m -XX:ReservedCodeCacheSize=512m -XX:MaxMetaspaceSize=1024m]
2021-12-15T15:48:48.5682436Z stderr[/cygdrive/c/actions-runner/_work/dotty/dotty/bin/../bin/common: line 23: /cygdrive/c/actions-runner/_work/dotty/dotty/bin/../dist/target/pack/bin/scala: No such fil
e or directory]
2021-12-15T15:48:48.5685591Z [error] Test dotty.tools.scripting.BashScriptsTests.verifyScalaOpts failed: java.lang.AssertionError: assertion failed: script C:/actions-runner/_work/dotty/dotty/out/boot
strap/scala3-compiler-bootstrapped/scala-3.1.2-RC1-bin-SNAPSHOT-nonbootstrapped/test-classes/scripting/classpathReport.sc did not report valid java.class.path first entry, took 2.235 sec
2021-12-15T15:48:48.5688432Z [error]     at scala.runtime.Scala3RunTime$.assertFailed(Scala3RunTime.scala:8)
2021-12-15T15:48:48.5689954Z [error]     at dotty.tools.scripting.BashScriptsTests.verifyScalaOpts(BashScriptsTests.scala:154)
2021-12-15T15:48:48.5691071Z [error]     ...
2021-12-15T15:48:48.6611862Z [error] Test dotty.tools.scripting.BashScriptsTests.verifyScalaDProperty failed: org.junit.ComparisonFailure: expected:<[]> but was:<[Hello World4]>, took 0.047 sec
2021-12-15T15:48:48.6614689Z [error]     at dotty.tools.scripting.BashScriptsTests.verifyScalaDProperty(BashScriptsTests.scala:92)
2021-12-15T15:48:48.6616106Z [error]     ...
2021-12-15T15:48:48.7517124Z [error] Test dotty.tools.scripting.BashScriptsTests.verifyScJProperty failed: org.junit.ComparisonFailure: expected:<[]> but was:<[Hello World]>, took 0.047 sec
2021-12-15T15:48:48.7521549Z [error]     at dotty.tools.scripting.BashScriptsTests.verifyScJProperty(BashScriptsTests.scala:74)
2021-12-15T15:48:48.7523142Z [error]     ...
2021-12-15T15:48:48.8472486Z [error] Test dotty.tools.scripting.BashScriptsTests.saveAndRunWithDProperty failed: java.util.NoSuchElementException: None.get, took 0.046 sec
2021-12-15T15:48:48.8474144Z [error]     at scala.None$.get(Option.scala:627)
2021-12-15T15:48:48.8474497Z [error]     at scala.None$.get(Option.scala:626)
2021-12-15T15:48:48.8475389Z [error]     at dotty.tools.scripting.BashScriptsTests.saveAndRunWithDProperty(BashScriptsTests.scala:98)
2021-12-15T15:48:48.8476222Z [error]     ...
2021-12-15T15:48:49.0349709Z [error] Test dotty.tools.scripting.BashScriptsTests.verifyScalaJProperty failed: org.junit.ComparisonFailure: expected:<[]> but was:<[Hello World3]>, took 0.031 sec
2021-12-15T15:48:49.0353214Z [error]     at dotty.tools.scripting.BashScriptsTests.verifyScalaJProperty(BashScriptsTests.scala:80)
2021-12-15T15:48:49.0354315Z [error]     ...
2021-12-15T15:48:49.1226753Z [error] Test dotty.tools.scripting.BashScriptsTests.verifyScDProperty failed: org.junit.ComparisonFailure: expected:<[]> but was:<[Hello World3]>, took 0.047 sec
2021-12-15T15:48:49.1228740Z [error]     at dotty.tools.scripting.BashScriptsTests.verifyScDProperty(BashScriptsTests.scala:86)
2021-12-15T15:48:49.1229581Z [error]     ...
2021-12-15T15:59:58.4224171Z [error] Failed: Total 398, Failed 7, Errors 0, Passed 388, Skipped 3
2021-12-15T15:59:58.4228680Z [error] Failed tests:
2021-12-15T15:59:58.4229272Z [error]    dotty.tools.scripting.BashScriptsTests
2021-12-15T16:00:00.2649908Z [error] (scala3-compiler-bootstrapped / Test / testOnly) sbt.TestsFailedException: Tests unsuccessful
2021-12-15T16:00:00.2758412Z [error] Total time: 1322 s (22:02), completed Dec 15, 2021 5:00:00 PM

A similar problem exists with [test_non_bootstrapped]:

2021-12-15T16:15:37.9975163Z stderr: [/usr/bin/bash: /__w/dotty/dotty/compiler/target/scala-3.1.1-RC1/test-classes/scripting/scriptPath.sc: dist/target/pack/bin/scala: bad interpreter: No such file or
 directory]
2021-12-15T16:15:37.9996260Z [error] Test dotty.tools.scripting.BashScriptsTests.verifyScriptPathProperty failed: java.lang.AssertionError: assertion failed: script /__w/dotty/dotty/compiler/target/sc
ala-3.1.1-RC1/test-classes/scripting/scriptPath.sc did not report valid script.path value, took 0.011 sec
2021-12-15T16:15:38.0003498Z [error]     at scala.runtime.Scala3RunTime$.assertFailed(Scala3RunTime.scala:8)
2021-12-15T16:15:38.0010494Z [error]     at dotty.tools.scripting.BashScriptsTests.verifyScriptPathProperty(BashScriptsTests.scala:132)
2021-12-15T16:15:38.0018491Z [error]     at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
2021-12-15T16:15:38.0026221Z [error]     at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:78)
2021-12-15T16:15:38.0040397Z [error]     at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
2021-12-15T16:15:38.0050896Z [error]     at java.lang.reflect.Method.invoke(Method.java:567)
2021-12-15T16:15:38.0054738Z [error]     ...
2021-12-15T16:26:56.3444719Z [error] Failed: Total 386, Failed 1, Errors 0, Passed 382, Skipped 3
2021-12-15T16:26:56.3452468Z [error] Failed tests:
2021-12-15T16:26:56.3454585Z [error]    dotty.tools.scripting.BashScriptsTests
2021-12-15T16:26:57.5465622Z [error] (scala3-compiler / Test / testOnly) sbt.TestsFailedException: Tests unsuccessful
2021-12-15T16:26:57.5488394Z [error] Total time: 1321 s (22:01), completed Dec 15, 2021, 5:26:57 PM

Here's where the test thinks scalac should be:

[test_windows_full]:
2021-12-15T15:48:46.1477510Z scalac path: [/actions-runner/_work/dotty/dotty/dist/target/pack/bin/scalac]
2021-12-15T15:48:46.2525459Z scalac path: [/actions-runner/_work/dotty/dotty/dist/target/pack/bin/scalac]
2021-12-15T15:48:46.3402138Z scalac path: [/actions-runner/_work/dotty/dotty/dist/target/pack/bin/scalac]
2021-12-15T15:48:48.6106346Z scalac path: [/actions-runner/_work/dotty/dotty/dist/target/pack/bin/scalac]
2021-12-15T15:48:48.7037625Z scalac path: [/actions-runner/_work/dotty/dotty/dist/target/pack/bin/scalac]
2021-12-15T15:48:48.7953349Z scalac path: [/actions-runner/_work/dotty/dotty/dist/target/pack/bin/scalac]
2021-12-15T15:48:48.8918062Z scalac path: [/actions-runner/_work/dotty/dotty/dist/target/pack/bin/scalac]
2021-12-15T15:48:48.9894969Z scalac path: [/actions-runner/_work/dotty/dotty/dist/target/pack/bin/scalac]
2021-12-15T15:48:49.0728460Z scalac path: [/actions-runner/_work/dotty/dotty/dist/target/pack/bin/scalac]

[test_non_bootstrapped]:
2021-12-15T16:15:37.9870802Z scalacPath: /__w/dotty/dotty/dist/target/pack/bin/scalac
2021-12-15T16:15:37.9885430Z scalac path: [/__w/dotty/dotty/dist/target/pack/bin/scalac]
2021-12-15T16:15:38.0099085Z scalac path: [/__w/dotty/dotty/dist/target/pack/bin/scalac]
2021-12-15T16:15:38.0212757Z scalac path: [/__w/dotty/dotty/dist/target/pack/bin/scalac]
2021-12-15T16:16:32.6123738Z scalac path: [/__w/dotty/dotty/dist/target/pack/bin/scalac]
2021-12-15T16:16:35.6505736Z scalac path: [/__w/dotty/dotty/dist/target/pack/bin/scalac]
2021-12-15T16:16:38.4822172Z scalac path: [/__w/dotty/dotty/dist/target/pack/bin/scalac]
2021-12-15T16:16:42.2652562Z scalac path: [/__w/dotty/dotty/dist/target/pack/bin/scalac]
2021-12-15T16:16:45.6345101Z scalac path: [/__w/dotty/dotty/dist/target/pack/bin/scalac]
2021-12-15T16:16:46.2396795Z scalac path: [/__w/dotty/dotty/dist/target/pack/bin/scalac]

Where do I look to discover the system layout of these tests? Apparently, it's different than the normal git workspace.

@BarkingBad
Copy link
Contributor

BarkingBad commented Dec 15, 2021

Where do I look to discover the system layout of these tests? Apparently, it's different than the normal git workspace.

Isn't it because no dist/pack is run in a first place of these two jobs? That's why we have no dist/target/pack/bin directory. I mean, these tests were silently failing for a long time because there was no scalac found but there was no else branch to raise the problem. Now we would need it, am I correct?

@philwalk
Copy link
Contributor Author

these tests were silently failing for a long time because there was no scalac found but there was no else branch to raise the problem

Ahh, that make sense. So we just need to add dist/pack. I'll do that now.

Copy link
Contributor

@BarkingBad BarkingBad left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank @philwalk for your input and for resolving all the problems. Is it ready to be merged or do you want to change something?

@philwalk
Copy link
Contributor Author

Thank @philwalk for your input and for resolving all the problems. Is it ready to be merged or do you want to change something?

No changes needed, AFAIK

@BarkingBad BarkingBad merged commit 0857285 into scala:master Dec 16, 2021
@philwalk philwalk deleted the fix-for-14106-cygwin-failed-tests branch December 16, 2021 14:17
@@ -66,7 +66,7 @@ jobs:

- name: Test
run: |
./project/scripts/sbt ";compile ;test"
./project/scripts/sbt ";dist/pack; compile ;test"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This re-introduces bootstrapping the compiler in "test_non_bootstrapped", which is annoying (can't see the results of running the tests in CI while the compiler fails to bootstrap).

If these tests need a bootstrapped compiler, can't they be in bootstrap-only?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If these tests need a bootstrapped compiler, can't they be in bootstrap-only?

I don't know enough about the test architecture to offer a reply, perhaps this is a question for @BarkingBad ?

olsdavis pushed a commit to olsdavis/dotty that referenced this pull request Apr 4, 2022
* refactor common code from scripting tests; invalid tests fail by default

* dump stdout/stderr if BashScriptsTests.verifyScalaOpts fails

* fix for failing cygwin tests for scala#14106

* normalize scala and scalac paths; set proper SHELLOPTS in cygwin bashCommandline env

* improved detection of scalacPath and scalaPath; additional logging

* print warnings; remove unused code

* strip ansi colors from bash command line output, to fix windows tests

* dist/pack before sbt test in test_windows_full and test_non_bootstrapped

* squeeze redundancy from env-var-setting tests, add more log messages

* further reduced redundancy; additional log messages

* remove trailing java from JAVA_HOME value; shorten comand lines with relpath

* Update BashScriptsTests.scala

remove duplicate

* Update BashScriptsTests.scala

Fix not-updating property

Co-authored-by: Andrzej Ratajczak <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants