Support Linux

[jpsim]

This could be built on top of jpsim/SourceKitten#223.

[koenpunt]

Now that jpsim/SourceKitten#268 is merged, would this be possible?

[jpsim]

Yes, and here's work in progress in #832

[brcolow]

Now that #832 has been merged, is it indeed possible to run SwiftLint on Linux (as this issue requests?) If so does any documentation/guide exist on how to get it working? Thanks!

[jpsim]

@brcolow as of #832, all of SwiftLint's dependencies support Linux, but SwiftLint itself still does not.

[marcelofabri]

Should we close this?

[jpsim]

Yes!

[ditansu]

So, now is it possible to install SwiftLint on linux?

[jpsim]

Yes.

[SDGGiesbrecht]

@ditansu

So, now is it possible to install SwiftLint on linux?

Only sort of. It can be installed on Linux, but not by the Swift Package Manager alone, and I cannot find clear instructions on getting it working.

When I run...

# Download package
git clone https://github.com/realm/SwiftLint
cd SwiftLint

# Prevent failure from swiftenv (Swift 3.2)–package manifest (Swift 4.0) version disagreement.
rm -f .swift‐version

swift run swiftlint

...I get...

fatal error: Loading libsourcekitdInProc.so failed: [...]

I assume this is what the read‐me was talking about:

On Linux, SourceKit is expected to be located in /usr/lib/libsourcekitdInProc.so or specified by the LINUX_SOURCEKIT_LIB_PATH environment variable.

Poking around inside Swift’s own install bundle, one finds usr/lib/libsourcekitdInProc.so, which is presumably what LINUX_SOUCREKIT_LIB_PATH needs to point at. But which version of Swift?

Aside: @jpsim, @norio-nomura, would it be possible, on Linux, for SwiftLint to link against the standard swiftenv install location(s)? These are both versioned and the most standard install of Swift on Linux (downloading from Swift.org does not put it anywhere in particular).

So to actually install, you need to, at least:

1. In the repository clone above, checkout the version you want. git checkout 0.23.1
2. Build for release swift build --configuration release
3. Move the contents of .build/release/ to some permanent location on the device.
4. Register the location in .bash_profile, etc. export PATH="wherever_I_just_put_it:$PATH"
5. Register SourceKit’s location in .bash_profile, etc. export LINUX_SOUCREKIT_LIB_PATH="wherever_I_have_swift_installed/usr/bin/libsourcekidInProc.so"
6. Restart the terminal and cross your fingers.

I got frustrated with the amount of work and gave up, so the above steps are just theoretical and still untested. Let me know if they work.

Aside: @jpsim, even with the above steps, SourceKit’s location is static, so it does not support side‐by‐side Swift versions. Will there be problems if the SourceKit pointed at does not match the version of Swift being fed to the rest of the system?

---

If you only want to be able to run SwiftLint in CI, then @norio-nomura’s docker image might be helpful. (See circle.yml in SwiftLint’s repository for how it runs its own tests on Linux.)

However, unless I misunderstand the way Docker works, it is not really helpful for a real user wanting to install and use SwiftLint on his own system.

[jpsim]

It's not just "sort of" possible to install SwiftLint on Linux. It has been supported for a long time now.

Although the exact steps aren't listed in the readme, so I admit doing so hasn't been very discoverable.

Here's one way to install SwiftLint on Ubunutu from scratch.

Let's create a brand new Linux machine on Digital Ocean and SSH into it:

$ doctl compute droplet create swiftlint --size 16gb \
    --image ubuntu-16-10-x64 --region sfo1
$ doctl compute ssh swiftlint

Once you're in, install Swift (curl here or your preferred installation method e.g. swiftenv):

$ apt-get update
$ apt-get install -y clang libblocksruntime0 libcurl4-openssl-dev
$ SWIFT_VERSION=swift-4.0-RELEASE
$ BASE_URL=https://swift.org/builds/swift-4.0-release/ubuntu1610
$ URL=$BASE_URL/$SWIFT_VERSION/$SWIFT_VERSION-ubuntu16.10.tar.gz
$ curl $URL | tar xz --directory $HOME --strip-components=1

Set $PATH to find the Swift binaries and $LINUX_SOURCEKIT_LIB_PATH to find
SourceKit:

$ export PATH=$HOME/usr/bin:$PATH
$ export LINUX_SOURCEKIT_LIB_PATH=$HOME/usr/lib

Clone, build & run SwiftLint on its own codebase:

$ git clone https://github.com/realm/SwiftLint.git
$ cd SwiftLint
$ swift run -c release swiftlint
Fetching https://github.com/Carthage/Commandant.git
Fetching https://github.com/jpsim/SourceKitten.git
Fetching https://github.com/jpsim/Yams.git
...
Compile Swift Module 'SwiftLintFramework' (208 sources)
Compile Swift Module 'swiftlint' (10 sources)
Linking ./.build/x86_64-unknown-linux/release/swiftlint
Loading configuration from '.swiftlint.yml'
Linting Swift files in current working directory
Linting 'UnusedClosureParameterRule.swift' (1/262)
Linting 'MultipleClosuresWithTrailingClosureRule.swift' (2/262)
Linting 'WeakDelegateRule.swift' (3/262)
Linting 'NimbleOperatorRule.swift' (4/262)
...
Linting 'TodoRuleTests.swift' (259/262)
Linting 'ConfigurationTests+ProjectMock.swift' (260/262)
Linting 'AttributesRuleTests.swift' (261/262)
Linting 'LinuxMain.swift' (262/262)
Done linting! Found 0 violations, 0 serious in 262 files.

To run SwiftLint using a different version of SourceKit, update $LINUX_SOURCEKIT_LIB_PATH.

[SDGGiesbrecht]

@jpsim that is bootstrapped but not really installed. Without exporting SwiftLint’s location, the swiftlint command is still unavailable to the terminal. And without adding entries to the shell login scripts (see 5. and 6. above), both the swiftlint command and SourceKit will be unavailable in future terminal sessions.

There is no easy equivalent that “just works”. Nothing comparable to macOS: brew install swiftlint, and voilà, done.

A user needs some level of understanding of the internals of both Swift and SwiftLint in order to puzzle through a manual install that needs adjusting according to the particular machine set‐up. That is why I still think “sort of” is the correct phrase.

I started to install it a year ago, but when I realized how much work it was, and that I would need to re‐do it each time an update was available for either Swift or SwiftLint... well I decided it was not worth it. And I have been waiting for a better way ever since. I imagine there are probably other like‐minded potential users out there.

Linux will only be truly supported when all a user needs to do is...

$ apt-get install swiftlint
$ swiftlint lint

...or at least...

eval "$(curl -sL https://gist.githubusercontent.com/realm/.../install-swiftlint.sh)"

[jpsim]

I'd love for it to be that easy too, of course.

Do you know of any open source Swift executables that are installable via apt-get that I could look at in order to make SwiftLint installable that way? Does Ubuntu APT even support Swift projects? I can't seem to find any.

It might be easiest to go the remote script execution route assuming that Swift is already installed, the current Swift version can build that version of SwiftLint, /usr/local/bin is already in the $PATH, and SourceKit is available at a reasonable location.

[ditansu]

@jpsim I've can't install SwiftLint by your instruction. What I've done wrong?

jenkins@cocoaheadsru-stage:~$ ll /usr/lib/libsourcekitdInProc.so
-rw-r--r-- 1 root root 99967840 Sep 20 00:52 /usr/lib/libsourcekitdInProc.so
jenkins@cocoaheadsru-stage:~$ env | grep LINUX_SOURCEKIT_LIB_PATH
LINUX_SOURCEKIT_LIB_PATH=/usr/lib
jenkins@cocoaheadsru-stage:~$ git clone https://github.com/realm/SwiftLint.git
Cloning into 'SwiftLint'...
remote: Counting objects: 22685, done.
remote: Compressing objects: 100% (55/55), done.
remote: Total 22685 (delta 39), reused 66 (delta 33), pack-reused 22597
Receiving objects: 100% (22685/22685), 5.67 MiB | 5.65 MiB/s, done.
Resolving deltas: 100% (15578/15578), done.
Checking connectivity... done.
jenkins@cocoaheadsru-stage:~$ cd SwiftLint/
jenkins@cocoaheadsru-stage:~/SwiftLint$ 
jenkins@cocoaheadsru-stage:~/SwiftLint$ swift run -c release swiftlint
Fetching https://github.com/Carthage/Commandant.git
Fetching https://github.com/jpsim/SourceKitten.git
Fetching https://github.com/jpsim/Yams.git
Fetching https://github.com/scottrhoyt/SwiftyTextTable.git
Fetching https://github.com/norio-nomura/SourceKit.git
Fetching https://github.com/norio-nomura/Clang_C.git
Fetching https://github.com/antitypical/Result.git
Fetching https://github.com/drmohundro/SWXMLHash.git
....
Compile CYaml src/api.c
Compile Swift Module 'SWXMLHash' (2 sources)
Compile Swift Module 'SwiftyTextTable' (1 sources)
Compile Swift Module 'Result' (2 sources)
Compile Swift Module 'Yams' (13 sources)
Compile Swift Module 'Commandant' (10 sources)
Compile Swift Module 'SourceKittenFramework' (34 sources)
Compile Swift Module 'SwiftLintFramework' (208 sources)
Compile Swift Module 'swiftlint' (10 sources)
Linking ./.build/x86_64-unknown-linux/release/swiftlint
Loading configuration from '.swiftlint.yml'
fatal error: Loading libsourcekitdInProc.so failed: file /home/jenkins/SwiftLint/.build/checkouts/SourceKitten.git--5847991402907209341/Source/SourceKittenFramework/library_wrapper.swift, line 61
Illegal instruction (core dumped)

Environment

jenkins@cocoaheadsru-stage:~$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 16.04.3 LTS
Release:        16.04
Codename:       xenial
jenkins@cocoaheadsru-stage:~$ swift --version 
Swift version 4.0 (swift-4.0-RELEASE)
Target: x86_64-unknown-linux-gnu

[jpsim]

@ditansu looks like a dlopen error to me. Could you modify /home/jenkins/SwiftLint/.build/checkouts/SourceKitten.git--5847991402907209341/Source/SourceKittenFramework/library_wrapper.swift with the following, then run swift run -c release swiftlint again please?

diff --git a/Source/SourceKittenFramework/library_wrapper.swift b/Source/SourceKittenFramework/library_wrapper.swift
index aabef71..305494a 100644
--- a/Source/SourceKittenFramework/library_wrapper.swift
+++ b/Source/SourceKittenFramework/library_wrapper.swift
@@ -56,6 +56,10 @@ struct Loader {
             if let handle = dlopen(fullPath, RTLD_LAZY) {
                 return DynamicLinkLibrary(path: path, handle: handle)
             }
+            print("Could not dlopen '\(fullPath)'")
+            if let errorMessage = String(cString: dlerror(), encoding: .utf8) {
+                print(errorMessage)
+            }
         }
 
         fatalError("Loading \(path) failed")

---

Also, I'd appreciate if you could file a new issue on GitHub to continue this discussion.

[jpsim]

Let me know if that error logging actually works and I'll make a PR towards SourceKitten to include that.

[ditansu]

@jpsim thanks a lot for fast reaction 👍
Sure , I'll open a new issue in near time.

[SDGGiesbrecht]

It might be easiest to go the remote script execution route.

Probably. Unfortunately “easiest” does not mean “easy”. I’m currently wrestling through the same thing for anther project. Ugh, Linux has too many different shells, too many different terminals, and too many places to put things... If I find an elegant solution I will report back.

Thanks for the reply, @jpsim.

[ditansu]

@jpsim, ooops, sorry, I did mistake. I forgot one step:
apt-get install -y clang libblocksruntime0 libcurl4-openssl-dev
now all it is OK!

...
Linting 'DiscouragedDirectInitRuleTests.swift' (253/262)
Linting 'IntegrationTests.swift' (254/262)
Linting 'LineLengthRuleTests.swift' (255/262)
Linting 'FunctionBodyLengthRuleTests.swift' (256/262)
Linting 'ConfigurationTests.swift' (257/262)
Linting 'ConfigurationTests+Nested.swift' (258/262)
Linting 'ExtendedNSStringTests.swift' (259/262)
Linting 'CustomRulesTests.swift' (260/262)
Linting 'FileHeaderRuleTests.swift' (261/262)
Linting 'LinuxMain.swift' (262/262)
Done linting! Found 0 violations, 0 serious in 262 files.

[jpsim]

Glad to hear it, @ditansu! Did you try with the logging I shared above? I'd love to know if that actually prints something useful. If it does, I'd include that in SourceKitten.

[ditansu]

Did you try with the logging I shared above?

sorry, but no
So @jpsim, how to use it, I'm not see swiftlint ? I would like use it for Vapor project. I have folder "server" and what I have to do to use SwiftLint ?

[ditansu]

Ok, I find it, so need one more step
cp ./.build/x86_64-unknown-linux/release/swiftlint /usr/bin/
Probably @SDGGiesbrecht in something right when he said "Only sort of" ;)

[SDGGiesbrecht]

@ditansu,

.build/x86_64-unknown-linux/release/ also contains dependency libraries. You have three options, ordered from easiest and worst to best and hardest:

A) Copy everything from the folder, not just the swiftlint executable. (Not recommended as it will make an unnecessary mess of /usr/bin/).
B) Create a symlink in /usr/bin/ called swiftlint that points to .../.build/x86_64-unknown-linux/release/swiftlint, and keep the repository around permanently.
C) Add an entry to path: export PATH=".../.build/x86_64-unknown-linux/release:$PATH". Again, keep the repository around. (If you want this to persist into future terminal sessions, you will need to register it for your shell. For example: echo 'export PATH=".../.build/x86_64-unknown-linux/release:$PATH"' >> ~/.bash_profile)

[SDGGiesbrecht]

@ditansu,

P.S. If you want, for B or C above, you could still copy out everything to a nicer location to point at than .../.build/x86_64-unknown-linux/release. Then you could delete the repository and save some space.

[jpsim]

It can be much simpler than all that.

swift build has a --static-swift-stdlib flag which creates a self-contained binary. You can also install Swift in /usr/bin and /usr/lib, meaning that there are no environment variables to mess with. If you install SwiftLint under /usr/local/bin, also no env vars to mess with.

From the top with a fresh Ubuntu machine:

Create a brand new Ubuntu machine

$ doctl compute droplet create swiftlint --size 16gb \
    --image ubuntu-16-10-x64 --region sfo1
$ doctl compute ssh swiftlint

Install Swift

$ apt-get update
$ apt-get install -y clang libblocksruntime0 libcurl4-openssl-dev
$ curl https://swift.org/builds/swift-4.0-release/ubuntu1610/swift-4.0-RELEASE/swift-4.0-RELEASE-ubuntu16.10.tar.gz \
    | tar xz --directory / --strip-components=1

Build & Install SwiftLint

$ git clone https://github.com/realm/SwiftLint.git
$ cd SwiftLint
$ swift build -c release --static-swift-stdlib
$ mv .build/x86_64-unknown-linux/release/swiftlint /usr/local/bin/

Clean up

$ cd ..
$ rm -rf SwiftLint

Use SwiftLint

$ swiftlint version
0.23.0
$ echo "let a = 0" | swiftlint lint --use-stdin
<nopath>:1:1: error: Identifier Name Violation: Variable name should be between 3 and 40 characters long: 'a' (identifier_name)
Done linting! Found 1 violation, 1 serious in 1 file.

[ditansu]

@jpsim thanks for clear steps! But last one don't work (I've get empty output) without .swiftlint.yml and theirs include files, I mean

included:
  - Source
  - Tests

so before remove you need to do

jenkins@cocoaheadsru-stage:~/SwiftLint$ cd ..
jenkins@cocoaheadsru-stage:~$ cp SwiftLint/.swiftlint.yml .
jenkins@cocoaheadsru-stage:~$ cp -r SwiftLint/Tests . 
jenkins@cocoaheadsru-stage:~$ cp -r SwiftLint/Source . 
jenkins@cocoaheadsru-stage:~$ rm -rf SwiftLint
jenkins@cocoaheadsru-stage:~$ echo "let a = 0" | swiftlint lint --use-stdin
Loading configuration from '.swiftlint.yml'
<nopath>:1:1: error: Identifier Name Violation: Variable name should be between 3 and 40 characters long: 'a' (identifier_name)
<nopath>:1:1: warning: File Header Violation: Header comments should be consistent with project patterns. (file_header)
Done linting! Found 2 violations, 1 serious in 1 file.
jenkins@cocoaheadsru-stage:~$

[jpsim]

I don't understand what the problem is. SwiftLint appears to be operating fine in your output from my understanding.

I'd like to keep discussing ways to improve the Linux experience, document it, help with issues etcetera, but this issue which has been closed for ~10 months is not the place to do it.

Please file new issues to continue these conversations. Thanks!