Swiftlint 0.39 library not loaded #3105

Oliver-Kirkland-Evoke · 2020-02-11T13:15:32Z

New Issue Checklist

Updated SwiftLint to the latest version
I searched for existing GitHub issues

Describe the bug

swiftlint always crashes after updating to version 0.39 via Homebrew. We've tried this on 3 systems and seen the same issue.

We have tried installing "Swift 5 Runtime Support for Command Line Tools" as per the known issues but the systems are too new to allow this to be installed.

We have downgraded to 0.38.2 which is working fine.

Complete output when running SwiftLint, including the stack trace and command used

$ swiftlint lint
dyld: Library not loaded: @rpath/lib_InternalSwiftSyntaxParser.dylib
  Referenced from: /usr/local/bin/swiftlint
  Reason: image not found
Abort trap: 6

Environment

SwiftLint version: 0.39
macOS versions: 10.15.3 && 10.14.6
Installation method used: Homebrew
Are you using nested configurations?
If so, paste their relative paths and respective contents.
Which Xcode version are you using: 11.3
Do you have a sample that shows the issue? Run `swiftlint"

marcelofabri · 2020-02-11T16:14:43Z

@Oliver-Kirkland-Evoke does it work if you try brew reinstall --build-from-source swiftlint?

Doxxer · 2020-02-11T16:27:32Z

Same issue, Installation method used: Cocoapods

marcelofabri · 2020-02-11T16:39:58Z

Same issue, Installation method used: Cocoapods

Can you post my details about your environment? I can image why using homebrew wouldn't work, but using CP should work fine 🤔

PatrickGaissert · 2020-02-11T16:48:30Z

We have the same problem. Integration is also via Cocoapods, Xcode 11.3.1.

marcelofabri · 2020-02-11T17:06:42Z

OK, so looks like the issue is that the binary is somehow hardcoding the path to the Xcode path from the machine that created the binary:

$ DYLD_PRINT_RPATHS=1 swiftlint help
RPATH failed expanding     @rpath/lib_InternalSwiftSyntaxParser.dylib to: /usr/lib/swift/lib_InternalSwiftSyntaxParser.dylib
RPATH failed expanding     @rpath/lib_InternalSwiftSyntaxParser.dylib to: /Applications/Xcode-11.3.1.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/swift/macosx/lib_InternalSwiftSyntaxParser.dylib
dyld: Library not loaded: @rpath/lib_InternalSwiftSyntaxParser.dylib
  Referenced from: /usr/local/bin/swiftlint

I'm not sure how to fix that and won't have time to look into this until later, so I recommend using 0.38.2 for now.

marcelofabri · 2020-02-11T17:14:17Z

@nathawes do you have any ideas on how to fix this? Ideally we'd statically link lib_InternalSwiftSyntaxParser, but maybe that's not possible?

RicardoKoch · 2020-02-11T17:51:49Z

This update broke our CI build system.
Currently it's pulling always the last version.
I'm testing a fix by copying the latest Xcode and renaming it to: Xcode-11.3.1.app

Oliver-Kirkland-Evoke · 2020-02-11T18:14:58Z

FYI
We use autoupdate with HomeBrew to keep our development systems and CI system using the same version of SwiftLint and other tools. If using HomeBrew I recommend using:

brew uninstall swiftlint
brew install https://raw.githubusercontent.com/Homebrew/homebrew-core/203436f8567a3b6455264a1d896da5b1df1e50e6/Formula/swiftlint.rb
brew pin swiftlint

This will keep SwiftLint at version 0.38.2 until brew unpin swiftlint is run.

rfree18 · 2020-02-11T18:17:13Z

My team is also experiencing this problem on Xcode 11.3.1. We're installing it with CocoaPods as well

serges147 · 2020-02-11T18:18:21Z

My team as well experiencing this problem on Xcode 11.3.1. We're installing it via CocoaPods.

nathawes · 2020-02-11T18:27:19Z

@nathawes do you have any ideas on how to fix this? Ideally we'd statically link lib_InternalSwiftSyntaxParser, but maybe that's not possible?

@akyrtzi what's the right thing to do here?

RicardoKoch · 2020-02-11T18:45:30Z

I was able to temporarily fix this issue by renaming Xcode.app to Xcode-11.3.1.app (inside Applications).

leifktaylor · 2020-02-11T18:47:35Z

This broke my team's CI build.
We resolved the issue for the time being by locking our Swiftlint version to 0.38.2 in our podfile.

bskendig · 2020-02-11T18:49:31Z

brew reinstall --build-from-source swiftlint built a copy of 0.39 for me that works properly, so that's a workaround.

akyrtzi · 2020-02-11T18:52:39Z

Ideally we'd statically link lib_InternalSwiftSyntaxParser, but maybe that's not possible?

Unfortunately not possible. In order to get a tool that is portable (working on a different machine than it was built with) you need to copy lib_InternalSwiftSyntaxParser.dylib and have it alongside the tool. And the tool's rpaths should be able to find it relative to itself.

marcelofabri · 2020-02-11T18:54:06Z

Ideally we'd statically link lib_InternalSwiftSyntaxParser, but maybe that's not possible?

Unfortunately not possible. In order to get a tool that is portable (working on a different machine than it was built with) you need to copy lib_InternalSwiftSyntaxParser.dylib and have it alongside the tool. And the tool's rpaths should be able to find it relative to itself.

Thanks! I'll look into that later 🙌

jpsim · 2020-02-11T20:46:55Z

@marcelofabri SourceKitten grabs clang and sourcekit dylibs dynamically here: https://github.com/jpsim/SourceKitten/blob/master/Source/SourceKittenFramework/library_wrapper.swift

We should do the same with lib_InternalSwiftSyntaxParser.dylib.

jpsim · 2020-02-11T20:47:47Z

@marcelofabri can we revert the SwiftSyntax inclusions in 0.39.0 and cut a 0.39.1 release to unblock folks who want to be on the latest bleeding edge SwiftLint version?

marcelofabri · 2020-02-11T20:49:50Z

@marcelofabri can we revert the SwiftSyntax inclusions in 0.39.0 and cut a 0.39.1 release to unblock folks who want to be on the latest bleeding edge SwiftLint version?

I was planning to look into fixing the issue in the evening, but if you (or any other contributor) have the time before that, feel free to revert the changes and cut a release.

jpsim · 2020-02-11T20:53:06Z

ok, I'll revert & cut a new release.

IMO we should avoid copying lib_InternalSwiftSyntaxParser.dylib and vending it with SwiftLint. We've been able to avoid doing that for SourceKit & libclang and we should be able to avoid it for SwiftSyntax too.

marcelofabri · 2020-02-11T20:54:57Z

Thanks - and sorry for the mess!

akyrtzi · 2020-02-11T21:03:40Z

IMO we should avoid copying lib_InternalSwiftSyntaxParser.dylib and vending it with SwiftLint. We've been able to avoid doing that for SourceKit & libclang and we should be able to avoid it for SwiftSyntax too.

There is a significant difference here in that sourcekitd.framework and libclang.dylib cannot be separated from the toolchain they came from, they will stop functioning if you try to copy them out of the toolchain.

lib_InternalSwiftSyntaxParser.dylib is completely self-contained so it can be copied out. We are even considering having it be a SwiftPM binary dependency, in which case it may not be in the toolchain anymore (or at least building SwiftSyntax from source will not require having that library in the toolchain).

jpsim · 2020-02-11T21:12:54Z

Thanks for that useful insight @akyrtzi! If we re-vend the binary as part of SwiftLint distributions, this also impacts our software license, so we'll need to carefully consider the implications there.

The new rules introduced in 0.39.0 that depend on SwiftSyntax have been temporarily removed as we work out release packaging issues. * `prohibited_nan_comparison` * `return_value_from_void_function` * `tuple_pattern` * `void_function_in_ternary` See #3105 for details.

norio-nomura · 2020-02-12T00:55:23Z

@marcelofabri SourceKitten grabs clang and sourcekit dylibs dynamically here:
https://github.com/jpsim/SourceKitten/blob/master/Source/SourceKittenFramework/library_wrapper.swift

We should do the same with lib_InternalSwiftSyntaxParser.dylib.

As far as I tested, lib_InternalSwiftSyntaxParser.dylib is not compatible between Xcode 11.3.1 and Xcode 11.4.

$ swift package reset
$ swift build --configuration release
…
[18/18] Linking swiftlint
$ install_name_tool -delete_rpath /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/swift/macosx .build/x86_64-apple-macosx/release/swiftlint
$ DYLD_LIBRARY_PATH=/Applications/Xcode-beta.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/swift/macosx .build/x86_64-apple-macosx/release/swiftlint lint --enable-all-rules --no-cache --quiet >&-
Never call this for file that sourcekitd fails.: file SwiftLintFile+Cache.swift, line 185
fish: 'DYLD_LIBRARY_PATH=/Applications…' terminated by signal SIGABRT (Abort)

So it needs to check if the version of lib_InternalSwiftSyntaxParser.dylib is compatible with swiftlint before dynamic loading.

jpsim · 2020-02-12T00:56:53Z

Yeah that's what @akyrtzi mentioned above in #3105 (comment)

marcelofabri · 2020-02-12T07:55:39Z

I was able to dynamically load it, but executing the binary still fails because of dyld: Library not loaded: @rpath/lib_InternalSwiftSyntaxParser.dylib (as @rpath/lib_InternalSwiftSyntaxParser.dylib is still mentioned in otool -L swiftlint) 🤔

norio-nomura · 2020-02-12T08:00:28Z

It needs to mark lib_InternalSwiftSyntaxParser.dylib as weak import like:

swift build --configuration release -Xlinker -weak-l_InternalSwiftSyntaxParser

kylebrowning · 2020-02-20T15:06:37Z

Confirmed, broken on CI.

jpsim · 2020-02-20T15:33:30Z

This issue shouldn’t happen with 0.39.1, which removed SwiftSyntax integration until we can improve the packaging situation.

fdiaz · 2020-04-08T19:04:12Z

I discovered a similar issue in a library I'm writing. A workaround was to rename Xcode-xyz.app to Xcode.app. It seems there's something in SwiftSyntax that assumes that swift toolchain will be under /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/swift.

I'm still investigating what's going on, but thought I'd share in case it's useful to anybody.

jpsim · 2020-04-08T20:44:04Z

Closing as we resolved this in 0.39.1 and 0.39.2.

Adding SwiftSyntax to the generator requires the dylib `_InternalSwiftSyntaxParser` which is normally bundled as part of the Xcode toolchain. Due to instability and crashing in SwiftSyntax for Swift 5.1, we are using SwiftSyntax for Swift 5.2 and thus requires the Xcode 11.4 toolchain. See also: realm/SwiftLint#3105 Several potential solutions I reviewed: 1. Only support running with Xcode 11.4 This is unacceptable because we want the CLI to run mostly independently of what Xcode toolchain is currently installed on the machine. 2. Provide the dylib when installing the CLI This is a straightforward option and works out-of-the-box if the dylib is in the same directory as the CLI (due to the order of rpath expansion by dyld). However, now the CLI is no longer easily portable and we'd need to create an installation method for all supported package managers. 3. Embed the dylib into the CLI This incurs a small complexity in the launch process but provides the most streamlined experience for developers. At a high level it involves embedding the dylib into the source and at runtime outputting it into a directory under a custom rpath. Once all dependent dylibs exist, it's not possible to just use runtime loading calls `dlopen` / `dlsym` unless we fork SwiftSyntax to pull the symbols dynamically. It's also not possible to call into dyld APIs to replace the loaded but unbound symbol references after the program has launched (`__dyld_start` has already completed). https://opensource.apple.com/source/dyld The best approach here seems to be to relaunch the current process as a subprocess with the same args and env.

marcelofabri added the bug Unexpected and reproducible misbehavior. label Feb 11, 2020

jpsim mentioned this issue Feb 11, 2020

Temporarily remove all SwiftSyntax rules and support #3107

Merged

jpsim mentioned this issue Feb 11, 2020

Re-add all SwiftSyntax rules and support #3108

Closed

kean mentioned this issue Apr 7, 2020

dyld: Library not loaded: @rpath/lib_InternalSwiftSyntaxParser.dylib SwiftDocOrg/swift-doc#60

Closed

jpsim closed this as completed Apr 8, 2020

elsh mentioned this issue Apr 24, 2020

lib_InternalSwiftSyntaxParser.dylib linking issue with Homebrew uber/mockolo#108

Closed

andrewchang-bird mentioned this issue Apr 25, 2020

Refactor import and compilation directive parsing to use SwiftSyntax typealiased/mockingbird#92

Merged

bonyadmitr mentioned this issue May 15, 2020

dyld: Library not loaded: @rpath/lib_InternalSwiftSyntaxParser.dylib NSHipster/SwiftSyntaxHighlighter#3

Closed

jpsim mentioned this issue Sep 22, 2020

Fix false positives for 'multiple_closures_with_trailing_closure' #3353

Merged

lechuckcaptain mentioned this issue Sep 23, 2020

macOS Big Sur beta - dyld: Library not loaded: @rpath/lib_InternalSwiftSyntaxParser.dylib FlineDev/BartyCrouch#200

Closed

jpsim mentioned this issue Mar 3, 2021

_InternalSwiftSyntaxParser error with 1.3.0 krzysztofzablocki/Sourcery#909

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Swiftlint 0.39 library not loaded #3105

Swiftlint 0.39 library not loaded #3105

Oliver-Kirkland-Evoke commented Feb 11, 2020

marcelofabri commented Feb 11, 2020

Doxxer commented Feb 11, 2020

marcelofabri commented Feb 11, 2020

PatrickGaissert commented Feb 11, 2020

marcelofabri commented Feb 11, 2020

marcelofabri commented Feb 11, 2020

RicardoKoch commented Feb 11, 2020 •

edited

Loading

Oliver-Kirkland-Evoke commented Feb 11, 2020 •

edited

Loading

rfree18 commented Feb 11, 2020

serges147 commented Feb 11, 2020

nathawes commented Feb 11, 2020

RicardoKoch commented Feb 11, 2020

leifktaylor commented Feb 11, 2020

bskendig commented Feb 11, 2020

akyrtzi commented Feb 11, 2020

marcelofabri commented Feb 11, 2020

jpsim commented Feb 11, 2020

jpsim commented Feb 11, 2020

marcelofabri commented Feb 11, 2020

jpsim commented Feb 11, 2020

marcelofabri commented Feb 11, 2020

akyrtzi commented Feb 11, 2020

jpsim commented Feb 11, 2020

norio-nomura commented Feb 12, 2020

jpsim commented Feb 12, 2020

marcelofabri commented Feb 12, 2020

norio-nomura commented Feb 12, 2020

kylebrowning commented Feb 20, 2020

jpsim commented Feb 20, 2020

fdiaz commented Apr 8, 2020 •

edited

Loading

jpsim commented Apr 8, 2020

Swiftlint 0.39 library not loaded #3105

Swiftlint 0.39 library not loaded #3105

Comments

Oliver-Kirkland-Evoke commented Feb 11, 2020

New Issue Checklist

Describe the bug

Complete output when running SwiftLint, including the stack trace and command used

Environment

marcelofabri commented Feb 11, 2020

Doxxer commented Feb 11, 2020

marcelofabri commented Feb 11, 2020

PatrickGaissert commented Feb 11, 2020

marcelofabri commented Feb 11, 2020

marcelofabri commented Feb 11, 2020

RicardoKoch commented Feb 11, 2020 • edited Loading

Oliver-Kirkland-Evoke commented Feb 11, 2020 • edited Loading

rfree18 commented Feb 11, 2020

serges147 commented Feb 11, 2020

nathawes commented Feb 11, 2020

RicardoKoch commented Feb 11, 2020

leifktaylor commented Feb 11, 2020

bskendig commented Feb 11, 2020

akyrtzi commented Feb 11, 2020

marcelofabri commented Feb 11, 2020

jpsim commented Feb 11, 2020

jpsim commented Feb 11, 2020

marcelofabri commented Feb 11, 2020

jpsim commented Feb 11, 2020

marcelofabri commented Feb 11, 2020

akyrtzi commented Feb 11, 2020

jpsim commented Feb 11, 2020

norio-nomura commented Feb 12, 2020

jpsim commented Feb 12, 2020

marcelofabri commented Feb 12, 2020

norio-nomura commented Feb 12, 2020

kylebrowning commented Feb 20, 2020

jpsim commented Feb 20, 2020

fdiaz commented Apr 8, 2020 • edited Loading

jpsim commented Apr 8, 2020

RicardoKoch commented Feb 11, 2020 •

edited

Loading

Oliver-Kirkland-Evoke commented Feb 11, 2020 •

edited

Loading

fdiaz commented Apr 8, 2020 •

edited

Loading