Add Support for Linux

[mattt]

Tests are currently failing on Linux, although it's unclear exactly why. Perhaps something to do with a behavioral difference in the Swift REPL between macOS and Linux.

[heckj]

👋 I just tried setting up a PR, and I think I'm running headfirst into this. The error I'm getting on swift test --enable-test-discovery (using the swift:5.2 ubuntu linux docker image) is:

Test Suite 'All tests' started at 2020-04-20 04:42:02.871
Test Suite 'debug.xctest' started at 2020-04-20 04:42:02.874
Test Suite 'DocTestTests' started at 2020-04-20 04:42:02.874
Test Case 'DocTestTests.testRunner' started at 2020-04-20 04:42:02.876
/DocTest/Tests/DocTestTests/DocTestTests.swift:20: error: DocTestTests.testRunner : XCTAssertEqual failed: ("0") is not equal to ("3") -
Fatal error: Index out of range: file /home/buildnode/jenkins/workspace/oss-swift-5.2-package-linux-ubuntu-18_04/swift/stdlib/public/core/ContiguousArrayBuffer.swift, line 444
Current stack trace:
0    libswiftCore.so                    0x00007fa668364f90 swift_reportError + 50
1    libswiftCore.so                    0x00007fa6683d6a60 _swift_stdlib_reportFatalErrorInFile + 115
2    libswiftCore.so                    0x00007fa6680eb735 <unavailable> + 1382197
3    libswiftCore.so                    0x00007fa6680eb257 <unavailable> + 1380951
4    libswiftCore.so                    0x00007fa6680eaf23 <unavailable> + 1380131
5    libswiftCore.so                    0x00007fa6680ea890 _assertionFailure(_:_:file:line:flags:) + 525
6    libswiftCore.so                    0x00007fa6680d0ed0 Array.subscript.getter + 123
7    DocTestPackageTests.xctest         0x0000563b71165efc <unavailable> + 761596
8    DocTestPackageTests.xctest         0x0000563b71166c9f <unavailable> + 765087
9    libXCTest.so                       0x00007fa66863a730 XCTAssertTrue(_:_:file:line:) + 38
10   DocTestPackageTests.xctest         0x0000563b71165ba5 <unavailable> + 760741
11   DocTestPackageTests.xctest         0x0000563b7115e6ff <unavailable> + 730879
12   DocTestPackageTests.xctest         0x0000563b711655b1 <unavailable> + 759217
13   DocTestPackageTests.xctest         0x0000563b71165229 <unavailable> + 758313
14   DocTestPackageTests.xctest         0x0000563b7116500f <unavailable> + 757775
15   DocTestPackageTests.xctest         0x0000563b71165214 <unavailable> + 758292
16   libXCTest.so                       0x00007fa668633fd1 <unavailable> + 200657
17   libXCTest.so                       0x00007fa668633dde <unavailable> + 200158
18   libXCTest.so                       0x00007fa668633d64 <unavailable> + 200036
19   libXCTest.so                       0x00007fa6686340a9 <unavailable> + 200873
20   libXCTest.so                       0x00007fa6686254e7 <unavailable> + 140519
21   libXCTest.so                       0x00007fa668631ef0 XCTestCase.invokeTest() + 55
22   libXCTest.so                       0x00007fa668631bc0 XCTestCase.perform(_:) + 157
23   libXCTest.so                       0x00007fa668636530 XCTest.run() + 150
24   libXCTest.so                       0x00007fa6686342e0 XCTestSuite.perform(_:) + 160
25   libXCTest.so                       0x00007fa668636530 XCTest.run() + 150
26   libXCTest.so                       0x00007fa6686342e0 XCTestSuite.perform(_:) + 160
27   libXCTest.so                       0x00007fa668636530 XCTest.run() + 150
28   libXCTest.so                       0x00007fa6686342e0 XCTestSuite.perform(_:) + 160
29   libXCTest.so                       0x00007fa668636530 XCTest.run() + 150
30   libXCTest.so                       0x00007fa66862fda0 XCTMain(_:) + 1524
31   DocTestPackageTests.xctest         0x0000563b71165301 <unavailable> + 758529
32   libc.so.6                          0x00007fa6659c1ab0 __libc_start_main + 231
33   DocTestPackageTests.xctest         0x0000563b71129a6a <unavailable> + 514666
Exited with signal code 4

I have a sneaky suspicion there's some sort of thread race scenario happening here, but I'm no expert.

[heckj]

I suspect the evaluationHandler on the REPL object isn't being called and finalized before you invoke the completion, even with repl.waitUntilExit(). Especially since it looks like that's a process fork/launch on Linux, I don't think there's anything holding it back.

I wonder if you don't want to explicitly force a "wait to submit the next evaluate" until the first has returned - or some other means of chaining these events explicitly - to force that evaluation ordering. (wishing for a little async/await here)

[mattt]

@heckj Yeah, I've found myself missing proper async / await a lot lately. Though in this case, I think the REPL has a reasonable guarantee about I/O ordering — the only trick is to buffer output to avoid incorrect UTF-8 text segmentation.

I think your assessment of why this isn't working on Linux is accurate; between the abstraction of NSTask API and the OS and API-level differences on Linux, it's hard to know what to do. Just something we'll have to figure out through trial and error.

[heckj]

I started poking at this over the weekend, and realized that my pattern for solving this would likely be using some sort of promise structure. Since there's not such a library in the standard library, and not one already in play here, do you have a preferred promise-like library that you'd be OK with using? I didn't want to propose adding a dependency without asking for preferences first.

I did some light research, but no deep comparisons, and found quite a number of options, including two that looked interesting to me:

• https://github.com/kean/Future
• https://github.com/AndrewBarba/Bluebird.swift (EDIT scratch this one as the implementation uses os_unfair_lock() and os_unfair_lock_unlock(), which isn't available on Linux/Foundation)

I also looked at PromiseKit, but since there was so a lot of ObjC pieces to it, I thought it might be best to leave it alone for such a swift-focused project.

My own inclination (due to past familiarity with the API style emulated) is to the use Bluebird.swift, but that's entirely because it feels the most comfortable to me.

[mattt]

@heckj Wow, thanks for looking into this! I haven't done much with promises libraries in Swift, but @kean's Future looks to be (unsurprisingly) excellent. If we were to take such an approach, that's the library I'd want to try first.

[heckj]

I'm poking more into kean/Future for it's linux support. There's some updates that would be good to have there structurally, just for testing and such, to allow it to easily support linux into the future. I'm going to do those first, then make a stab at the REPL as a Future in my fork, and I'll reference a WIP PR here.

[heckj]

I've started a branch in my repo fork, but didn't want to open a PR until it moves beyond my stupid-trying-things-out-phase. Still, its public if you're interested in watching: https://github.com/heckj/DocTest/tree/linux. If you'd like it opened as a draft PR just to keep an eye, I'd be happy to do that.

[mattt]

@heckj Awesome. Yeah, go ahead and submit that as a draft PR whenever you have a chance. That has a lot more visibility than a fork, which servers two purposes: 1) avoiding duplication of effort, and 2) providing an opportunity for others to share any useful insights they have.

[heckj]

@mattt After a few rounds of digging and debugging, the underlying issue isn't with anything in the code at all, but pertains to Linux security (and docker) and the use of the REPL, which this tool invokes down into with it's subprocess pieces.

Effectively using the REPL within a docker container requires loosening the permissions on the container while it's running, specifically with --security-opt seccomp=unconfined. Once I clued into that, running everything within a docker container is showing the whole system operating as expected.

The PR that I added (#12) ends up not being needed at all, but was an interesting debugging exercise.

So I think this might be resolved most simply by documenting that running this within a container requires looser security constraints - although how to wrap that into something like GitHub Action image and action.yml isn't entirely clear.

Short form that shows this working:

Dockerfile

FROM swift:5.2
# concept borrowed from https://www.objc.io/blog/2018/08/28/testing-swift-on-linux/
# use:
#     docker build . -f "Dockerfile_5.2" -t linuxbuild52
#     docker run --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --security-opt apparmor=unconfined -it linuxbuild52
# to run the build and test...
WORKDIR /lib
COPY Package.swift ./
COPY Sources ./Sources
COPY Tests ./Tests
CMD ["bash"]

The run it through:

docker build . -t linuxbuild52
docker run --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --security-opt apparmor=unconfined -it linuxbuild52

And within the docker container:

swift build # repeat until it completes 
# I had it failing sometimes during the process, but it picks back up where it left off

swift test --enable-test-discovery

And with the looser constraints on the container, it all works.

[mattt]

@heckj Thanks again for investigating this issue. I understand that this probably isn't the most satisfying resolution to the problem, but this is wonderfully useful information for us to share in the documentation for this project.

Searching around based on your comment, I found this issue, which provides some more context: swiftlang/swift-docker#9

Combining that with the information from this thread on the GitHub forums, I was able to configure the CI action to work on Linux: https://github.com/SwiftDocOrg/DocTest/runs/630643726?check_suite_focus=true

Edit: It looks like your action finished at the same time: https://github.com/SwiftDocOrg/DocTest/actions/runs/91427859

Comparing the logs for both, it looks like the workflow file in #17 is able to save a few minutes by avoiding the extra docker image build. Any objections to going with that solution instead?

[heckj]

All good - I'm glad we found a resolution! That's what matters, and I learned a few interesting things along the way (including what a PITA it is to get LLDB to invoke tests on Linux ;-) )

I've closed my other branch, as it was mostly investigative

[heckj]

(note - you'll notably speed up a local docker build if we have a .dockerignore file that excludes .git, .swiftpm, and .build if it exists - more related to anyone else coming along and debugging locally than a CI instance, which should be clean to start with)