OpenBSD port tracking

[3405691582]

Description

Instead of using the forums, use this Github issue to track OpenBSD porting status and notes that are beneficial for other people trying to build Swift on this platform but not necessarily appropriate for putting into prs yet for official documentation.

Notes

Last edit: 2025-04-08.

Currently the project contains a percentage of C++ dependencies. However, Swift will eventually be written with critical components in Swift, requiring us to build a bootstrap toolchain. Therefore, there are three main different types of build output to think about, namely,

0. the legacy compiler toolchain containing just the legacy Swift libraries and compiler (i.o.w., no Dispatch, no Foundation)
1. the bootstrap toolchain containing the legacy compiler toolchain plus the minimal set of dependencies to build the standard toolchain components written in Swift (swift-driver, etc.)
2. the standard toolchain, including components written in Swift

You will most likely want to build the standard toolchain. You can do this in two phases: first building the bootstrap toolchain, then use that toolchain to build the standard toolchain.

First, save the following json file for the update-checkout script (compacted for simplicity):

{"ssh-clone-pattern":"git@github.com:%s.git","https-clone-pattern":"https://github.com/%s.git","repos":{"swift":{"remote":{"id":"swiftlang/swift"}},"llvm-project":{"remote":{"id":"swiftlang/llvm-project"}},"cmark":{"remote":{"id":"swiftlang/swift-cmark"}},"llbuild":{"remote":{"id":"swiftlang/swift-llbuild"}},"swift-build":{"remote":{"id":"swiftlang/swift-build"}},"swift-argument-parser":{"remote":{"id":"apple/swift-argument-parser"}},"swift-collections":{"remote":{"id":"apple/swift-collections"}},"swift-crypto":{"remote":{"id":"apple/swift-crypto"}},"swift-certificates":{"remote":{"id":"apple/swift-certificates"}},"swift-asn1":{"remote":{"id":"apple/swift-asn1"}},"swift-driver":{"remote":{"id":"swiftlang/swift-driver"}},"swift-toolchain-sqlite":{"remote":{"id":"swiftlang/swift-toolchain-sqlite"}},"swift-tools-support-core":{"remote":{"id":"swiftlang/swift-tools-support-core"}},"swiftpm":{"remote":{"id":"swiftlang/swift-package-manager"}},"swift-syntax":{"remote":{"id":"swiftlang/swift-syntax"}},"swift-system":{"remote":{"id":"apple/swift-system"}},"swift-corelibs-xctest":{"remote":{"id":"swiftlang/swift-corelibs-xctest"}},"swift-corelibs-foundation":{"remote":{"id":"swiftlang/swift-corelibs-foundation"}},"swift-foundation-icu":{"remote":{"id":"swiftlang/swift-foundation-icu"}},"swift-foundation":{"remote":{"id":"swiftlang/swift-foundation"}},"swift-corelibs-libdispatch":{"remote":{"id":"swiftlang/swift-corelibs-libdispatch"}},"swift-experimental-string-processing":{"remote":{"id":"swiftlang/swift-experimental-string-processing"}}},"default-branch-scheme":"main","branch-schemes":{"main":{"aliases":["swift/main","main","stable/20240723"],"repos":{"llvm-project":"stable/20240723","swift":"main","cmark":"gfm","llbuild":"main","swift-build":"main","swift-toolchain-sqlite":"1.0.1","swift-tools-support-core":"main","swiftpm":"main","swift-argument-parser":"1.4.0","swift-collections":"1.1.3","swift-crypto":"main","swift-certificates":"main","swift-asn1":"main","swift-driver":"main","swift-syntax":"main","swift-system":"main","swift-corelibs-xctest":"main","swift-corelibs-foundation":"main","swift-foundation-icu":"main","swift-foundation":"main","swift-corelibs-libdispatch":"main","swift-experimental-string-processing":"swift/main"}}}}

Save the following build-script invocation to a shell script to build the bootstrap toolchain:

./swift/utils/build-script \
  --release -A \
  --build-subdir=bootstrap \
  --llvm-targets-to-build=host \
  --skip-build-clang-tools-extra --skip-build-compiler-rt \
  --skip-early-swift-driver --skip-early-swiftsyntax \
  --enable-experimental-string-processing \
  --foundation --xctest \
  --install-swift \
  --install-foundation \
  --install-libdispatch \
  --install-xctest \
  --skip-clean-foundation \
  --skip-clean-libdispatch \
  --extra-cmake-options="\
        -DSWIFT_ENABLE_SWIFT_IN_SWIFT=OFF,\
        -DSWIFT_ENABLE_DISPATCH=OFF,\
        -DSWIFT_ENABLE_EXPERIMENTAL_CXX_INTEROP=OFF,\
        -DBRIDGING_MODE=PURE,\
        -DSWIFT_ENABLE_SYNCHRONIZATION=ON,\
        -DSWIFT_ENABLE_EXPERIMENTAL_CONCURRENCY=ON,\
        -DSWIFT_ENABLE_EXPERIMENTAL_STRING_PROCESSING=ON,\
        -DSWIFT_BUILD_SOURCEKIT=OFF,\
        -DSWIFT_USE_LINKER=lld,\
        -DCMAKE_INSTALL_DIR=/usr/local" 

Save the following build-script invocation to a shell script to build the standard toolchain:

env PATH=<path to your sources>/build/bootstrap/toolchain-openbsd-aarch64/usr/bin/:$PATH ./swift/utils/build-script \
  --release -A \
  --llvm-targets-to-build=host \
  --skip-build-clang-tools-extra --skip-build-compiler-rt \
  --enable-experimental-string-processing \
  --foundation --xctest \
  --swift-driver \
  --llbuild \
  --swiftpm \
  --install-swift \
  --install-foundation \
  --install-libdispatch \
  --install-xctest \
  --install-swift-driver \
  --install-swiftpm \
  --skip-clean-swiftpm \
  --skip-clean-libdispatch \
  --skip-clean-foundation \
  --extra-cmake-options="\
        -DLLVM_INSTALL_GTEST=ON,\
        -DSWIFT_ENABLE_EXPERIMENTAL_CXX_INTEROP=OFF,\
        -DBRIDGING_MODE=PURE,\
        -DSWIFT_ENABLE_SYNCHRONIZATION=ON,\
        -DSWIFT_ENABLE_EXPERIMENTAL_CONCURRENCY=ON,\
        -DSWIFT_ENABLE_EXPERIMENTAL_STRING_PROCESSING=ON,\
        -DSWIFT_BUILD_SOURCEKIT=OFF,\
        -DSWIFT_USE_LINKER=lld,\
        -DCMAKE_INSTALL_DIR=/usr/local" 

Check out swiftlang/swift.

Use the following update-checkout command:

./swift/utils/update-checkout --clone --config ./update-checkout-config.json --github-comment 'apple/swift-collections#456 apple/swift-certificates#239 swiftlang/swift-driver#1857 swiftlang/swift-corelibs-libdispatch#559 swiftlang/swift-corelibs-foundation#5196 apple/swift-crypto#349 swiftlang/swift-llbuild#979 swiftlang/swift-package-manager#8451'

Manual changes required:

• In llvm-project on arm64 builds, use Fix crash lowering stack guard on OpenBSD/aarch64. llvm/llvm-project#125416
• In swift,
  • change include/swift/AST/SILOptions.h so that EnableOSSAModules is false.
  • create an implementation for stdlib/public/Synchronization/Mutex.swift using pthread_mutex_t?, or use the 5.10 version of Foundation.
  • add the implementation and dependency in stdlib/public/Synchronization/CMakeLists.txt.
  • while this is not ideal to upstream, since pthread_mutex_t could be a value type with internal pointers, this should be safe enough for now since OpenBSD uses a pointer for pthread_mutex_t.
  • note the 5.10 version of swift-corelibs-foundation is the one prior to Mutex being adopted.
• In swift-foundation, add target_compile_options(FoundationEssentials PRIVATE "SHELL:$<$<COMPILE_LANGUAGE:Swift>:-Xfrontend -disable-large-loadable-types-reg2mem>") to Sources/FoundationEssentials/CMakeLists.txt or use a non-Release build type.

Other manual changes not described here may be required and will eventually be upstreamed or discussed further here.

• Needs investigation/known issues (roughly in priority order)
  • Blockers:
    • Currently the default configuration builds Swift with BTCFI disabled. This should be remedied.
    • Dispatch queues can possibly trigger concurrency problems in ld.so's allocator.
    • It's probably not going to be enough to disable C++ interop long term. We should possibly have a custom modulemap (much like the one for libstdc++), or figure out if the existing module-related errors can be worked around.
  • Not blockers/unknown
    • A durable solution to Synchronization should be found.
    • threading can be problematic: swiftc -num-threads can be memory-hungry and cause lockups, and ld.so warnings about recursive allocation errors occur sometimes
    • test suite discovery requires manual installation of libIndexStore.so, since the above scripts do not install the just-built llvm.

[finagolfin]

I've assigned you on this issue.

there are a number of issues with modules that prevent a successful build. Current suspicion is bad libc++ interaction with modules and/or existing upstream bugs with modules.

Have you tried turning off inline bridging mode, as currently done for Windows and as I used to do for Android?

I suggest you apply to be a committer if you plan to work on this port more. I can't say if you will get it, as I don't decide that, but based on your extensive track record, I suspect you will.

[finagolfin]

Note that yams has been removed in trunk, #78826.

[3405691582]

Have you tried turning off inline bridging mode, as currently done for Windows and as I used to do for Android?

I've experienced trouble with both PURE and INLINE bridging modes. Thankfully by turning interop off for now this sidesteps the issue and haven't been forced to deal with this problem further yet, though I likely will eventually.

[finagolfin]

I've experienced trouble with both PURE and INLINE bridging modes.

OK, pure bridging also stopped working for me natively on Android last March in trunk, but worked for building 5.10. You may want to try it with 5.10 on OpenBSD and see if that works. At the very least, that could give you a working "standard toolchain" that you could use to bootstrap everything else.

Thankfully by turning interop off for now this sidesteps the issue and haven't been forced to deal with this problem further yet, though I likely will eventually.

Yes, you will need it, as more and more of the compiler is written in Swift and directly calls LLVM/Swift C++ APIs inside the compiler.

I think Swift's C++ interop doesn't work with libc++ platforms and needs to be provided for C++ interop to work.

It does work, as most Swift platforms currently use libc++, such as all Darwin platforms and Android. Egor even got it working on linux, #75589.

That said, it can be a bit flaky and required some small changes to work on Android.

(I think this is why there are errors about a missing Clang module for CxxStdlib).

That module is added in this repo, you may just need to enable OpenBSD there, as Alex did for Android.

Moreover, OpenBSD's C++ includes are missing a modulemap.

That's surprising, as libc++ has shipped with a module map for years now. If OpenBSD is not including it for some reason, try copying it over and using it.

[3405691582]

I have what I think is a janky standard toolchain already atop HEAD/6.2 with C++ interop turned off, but granted, the next step is testing it to make sure it actually properly incorporates the Swift compiler sources. These are good breadcrumbs for when I get back to trying to reenable interop, thanks!

[jakepetroules]

@3405691582, thanks for your work on the OpenBSD port! We've recently open sourced Xcode's build system engine (Swift Build) which will eventually become part of the toolchain. I'm currently working on adding FreeBSD support there, and I expect OpenBSD support would be a similar change. Thought I'd mention it here so you can make sure you're aware of potential changes needed in this space. I filed an issue to track: swiftlang/swift-build#114