Enable SwiftCompilerSources on Windows #73247

eeckstein · 2024-04-25T09:17:51Z

With some hacks and workarounds I could get SwiftCompilerSources working on my local Windows machine.
To-do: also enable it in lldb (see my comment in the top-level CMakeLists file).

eeckstein · 2024-04-25T09:19:45Z

@swift-ci test windows

compnerd · 2024-04-25T14:11:50Z

CMakeLists.txt

+# A tempoarary hack: force enabling HOSTTOOLS mode on Windows.
+# Right now, SwiftCompilerSources cannot be enabled for lldb because the
+# Windows build script builds lldb before swift and that fails with a missing
+# dependency on swiftrt (which is built within swift).


lldb is not built before swift, it is built alongside as a single unified build. This implies that there is a missing dependency in the LLDB build definition.

Thanks for clarifying. I updated the comment (We can fix the underlying problem in a follow-up PR).

SwiftCompilerSources/CMakeLists.txt

compnerd · 2024-04-25T14:12:49Z

SwiftCompilerSources/CMakeLists.txt

@@ -106,6 +106,13 @@ function(add_swift_compiler_modules_library name)
      "-Xcc" "-std=c++17"
      "-Xcc" "-DCOMPILED_WITH_SWIFT" "-Xcc" "-DSWIFT_TARGET"
      "-Xcc" "-UIBOutlet" "-Xcc" "-UIBAction" "-Xcc" "-UIBInspectable")
+
+  if(CMAKE_SYSTEM_NAME STREQUAL "Windows")
+    list(APPEND swift_compile_options "-static")


Statically linking the standard library is not supported - how do we ensure that this is not applied to it?

Hm, I thought that -static means that the currently built module is statically linked. The option is applied to the SwiftCompilerSources modules (and not to the stdlib).
Anyway, without this option it doesn't work and with this option it works.

Ah, okay, this flag is only applied to SwiftCompilerSources? If so, that should be fine. I just wanted to ensure that it didn't accidentally leak. You are correct that it only applies to the module that you are building.

compnerd · 2024-04-25T14:13:44Z

SwiftCompilerSources/Sources/Basic/Utils.swift

+#if !os(Windows)
 public var standardError = CFileStream(fp: stderr)
+#endif


You should be able to implement this with the Win32 + ucrt APIs to convert the stderr handle to a FILE *.

compnerd · 2024-04-25T14:14:04Z

...lerSources/Sources/Optimizer/InstructionSimplification/SimplifyConvertEscapeToNoEscape.swift

@@ -28,6 +28,9 @@ private extension ConvertEscapeToNoEscapeInst {
  ///   %3 = thin_to_thick_function %1 to $@noescape () -> ()

  func tryCombineWithThinToThickOperand(_ context: SimplifyContext) {
+    // compiling bridged.getFunctionTypeWithNoEscape crashes the 5.10 Windows compiler


If this works on newer compilers, perhaps we should also include a version check?

I still need to check if it works with newer compilers

eeckstein · 2024-04-25T15:08:28Z

@swift-ci test Windows

eeckstein · 2024-04-25T16:46:13Z

@swift-ci test Windows

eeckstein · 2024-04-25T18:31:11Z

@swift-ci test Windows

eeckstein · 2024-04-25T20:08:43Z

@swift-ci test windows

eeckstein · 2024-04-26T07:55:41Z

The build is failing with

FAILED: tools/swift/SwiftCompilerSources/Basic.o T:/5/tools/swift/SwiftCompilerSources/Basic.o 
cmd.exe /C "cd /D C:\Users\swift-ci\jenkins\workspace\swift-PR-windows\swift\SwiftCompilerSources && T:\toolchains\swift-5.10-DEVELOPMENT-SNAPSHOT-2024-01-18-a-windows10\LocalApp\Programs\Swift\Toolchains\0.0.0+Asserts\usr\bin\swiftc.exe -c -o T:/5/tools/swift/SwiftCompilerSources/Basic.o -resource-dir T:/toolchains/swift-5.10-DEVELOPMENT-SNAPSHOT-2024-01-18-a-windows10/LocalApp/Programs/Swift/Platforms/0.0.0/Windows.platform/Developer/SDKs/Windows.sdk/usr/lib/swift -target x86_64-unknown-windows-msvc -module-name Basic -emit-module -emit-module-path T:/5/tools/swift/SwiftCompilerSources/Basic.swiftmodule -parse-as-library C:/Users/swift-ci/jenkins/workspace/swift-PR-windows/swift/SwiftCompilerSources/Sources/Basic/SourceLoc.swift C:/Users/swift-ci/jenkins/workspace/swift-PR-windows/swift/SwiftCompilerSources/Sources/Basic/StringParser.swift C:/Users/swift-ci/jenkins/workspace/swift-PR-windows/swift/SwiftCompilerSources/Sources/Basic/Utils.swift -wmo -color-diagnostics -Xfrontend -validate-tbd-against-ir=none -Xfrontend -disable-target-os-checking -Xcc -std=c++17 -Xcc -DCOMPILED_WITH_SWIFT -Xcc -DSWIFT_TARGET -Xcc -UIBOutlet -Xcc -UIBAction -Xcc -UIBInspectable -cxx-interoperability-mode=default -O -cross-module-optimization -Xcc -UNDEBUG -Xcc -DPURE_BRIDGING_MODE -Xfrontend -disable-legacy-type-info -static -Xllvm -sil-disable-pass=loadable-address -Xcc -I -Xcc C:/Users/swift-ci/jenkins/workspace/swift-PR-windows/llvm-project/llvm/include -Xcc -I -Xcc T:/5/include -Xcc -I -Xcc C:/Users/swift-ci/jenkins/workspace/swift-PR-windows/llvm-project/llvm/../clang/include -Xcc -I -Xcc T:/5/tools/clang/include -Xcc -I -Xcc C:/Users/swift-ci/jenkins/workspace/swift-PR-windows/swift/include -Xcc -I -Xcc T:/5/tools/swift/SwiftCompilerSources/../include -I T:/5/tools/swift/SwiftCompilerSources"
<unknown>:0: error: cannot load underlying module for 'CxxStdlib'

I didn't see this problem when building locally on my machine.

eeckstein · 2024-04-26T12:43:59Z

@swift-ci test Windows

eeckstein · 2024-04-26T17:03:14Z

@swift-ci test Windows

compnerd · 2024-04-26T19:21:42Z

The build is failing with

FAILED: tools/swift/SwiftCompilerSources/Basic.o T:/5/tools/swift/SwiftCompilerSources/Basic.o 
cmd.exe /C "cd /D C:\Users\swift-ci\jenkins\workspace\swift-PR-windows\swift\SwiftCompilerSources && T:\toolchains\swift-5.10-DEVELOPMENT-SNAPSHOT-2024-01-18-a-windows10\LocalApp\Programs\Swift\Toolchains\0.0.0+Asserts\usr\bin\swiftc.exe -c -o T:/5/tools/swift/SwiftCompilerSources/Basic.o -resource-dir T:/toolchains/swift-5.10-DEVELOPMENT-SNAPSHOT-2024-01-18-a-windows10/LocalApp/Programs/Swift/Platforms/0.0.0/Windows.platform/Developer/SDKs/Windows.sdk/usr/lib/swift -target x86_64-unknown-windows-msvc -module-name Basic -emit-module -emit-module-path T:/5/tools/swift/SwiftCompilerSources/Basic.swiftmodule -parse-as-library C:/Users/swift-ci/jenkins/workspace/swift-PR-windows/swift/SwiftCompilerSources/Sources/Basic/SourceLoc.swift C:/Users/swift-ci/jenkins/workspace/swift-PR-windows/swift/SwiftCompilerSources/Sources/Basic/StringParser.swift C:/Users/swift-ci/jenkins/workspace/swift-PR-windows/swift/SwiftCompilerSources/Sources/Basic/Utils.swift -wmo -color-diagnostics -Xfrontend -validate-tbd-against-ir=none -Xfrontend -disable-target-os-checking -Xcc -std=c++17 -Xcc -DCOMPILED_WITH_SWIFT -Xcc -DSWIFT_TARGET -Xcc -UIBOutlet -Xcc -UIBAction -Xcc -UIBInspectable -cxx-interoperability-mode=default -O -cross-module-optimization -Xcc -UNDEBUG -Xcc -DPURE_BRIDGING_MODE -Xfrontend -disable-legacy-type-info -static -Xllvm -sil-disable-pass=loadable-address -Xcc -I -Xcc C:/Users/swift-ci/jenkins/workspace/swift-PR-windows/llvm-project/llvm/include -Xcc -I -Xcc T:/5/include -Xcc -I -Xcc C:/Users/swift-ci/jenkins/workspace/swift-PR-windows/llvm-project/llvm/../clang/include -Xcc -I -Xcc T:/5/tools/clang/include -Xcc -I -Xcc C:/Users/swift-ci/jenkins/workspace/swift-PR-windows/swift/include -Xcc -I -Xcc T:/5/tools/swift/SwiftCompilerSources/../include -I T:/5/tools/swift/SwiftCompilerSources"
<unknown>:0: error: cannot load underlying module for 'CxxStdlib'

I didn't see this problem when building locally on my machine.

This is likely another missing dependency in the build graph (CC: @etcwilde). The build is shuffled on differently on different hosts. The dependencies should be accurate to avoid these types of spurious failures.

eeckstein · 2024-04-29T06:31:42Z

This is likely another missing dependency in the build graph

I already fixed it. I was using some wrong build options

It's not needed and doesn't work on Windows

…re not compilable on windows

…ling convert_function simplification

…s when building on Windows

swiftlang#73321

eeckstein · 2024-04-29T08:54:28Z

@swift-ci test

…ble-swift" This reverts commit 74ed041, reversing changes made to fca4bef.

eeckstein requested review from compnerd, bnbarham, etcwilde and DougGregor April 25, 2024 09:18

eeckstein marked this pull request as draft April 25, 2024 09:19

eeckstein requested a review from atrick April 25, 2024 09:21

compnerd reviewed Apr 25, 2024

View reviewed changes

eeckstein force-pushed the windows-enable-swift branch 2 times, most recently from 0fcfbbd to 6ae7846 Compare April 25, 2024 15:04

eeckstein force-pushed the windows-enable-swift branch from 6ae7846 to b5a6f37 Compare April 25, 2024 16:45

eeckstein force-pushed the windows-enable-swift branch from b5a6f37 to b569328 Compare April 25, 2024 18:26

eeckstein force-pushed the windows-enable-swift branch from b569328 to b5ab832 Compare April 25, 2024 20:08

eeckstein force-pushed the windows-enable-swift branch from b5ab832 to 8f447fb Compare April 26, 2024 12:43

eeckstein force-pushed the windows-enable-swift branch from 8f447fb to b5c4457 Compare April 26, 2024 17:02

This was referenced Apr 29, 2024

IRGen/linker_set_low_level_exec.swift is failing on Windows #73321

Open

Enable SwiftCompilerSources for lldb on Windows #73322

Closed

eeckstein added 6 commits April 29, 2024 10:52

cmake: enable SwiftCompilerSources on Windows

e9d6ba9

SwiftCompilerSources: remove a not needed import CxxStdlib

0c8c985

It's not needed and doesn't work on Windows

SwiftCompilerSources: exclude some low level file operations, which a…

87a520c

…re not compilable on windows

SwiftCompilerSources: workaround a compiler crash on windows by disab…

7bf193e

…ling convert_function simplification

SwiftCompilerSources: workaround a crash in the LoadableByAddress pas…

338dd18

…s when building on Windows

Disable the IRGen/linker_set_low_level_exec.swift on Windows

dcf27a1

swiftlang#73321

eeckstein force-pushed the windows-enable-swift branch from b5c4457 to dcf27a1 Compare April 29, 2024 08:52

eeckstein marked this pull request as ready for review April 29, 2024 08:54

eeckstein requested review from ahoppen, hamishknight and rintaro as code owners April 29, 2024 08:54

ahoppen removed their request for review April 29, 2024 16:49

eeckstein merged commit 74ed041 into swiftlang:main Apr 30, 2024
5 checks passed

eeckstein deleted the windows-enable-swift branch April 30, 2024 07:04

eeckstein mentioned this pull request May 3, 2024

Force enabling SwiftCompilerSources in the build #73417

Merged

hjyamauchi added a commit to hjyamauchi/swift that referenced this pull request Jun 14, 2024

Revert "Merge pull request swiftlang#73247 from eeckstein/windows-ena…

577e519

…ble-swift" This reverts commit 74ed041, reversing changes made to fca4bef.

Enable SwiftCompilerSources on Windows #73247

Enable SwiftCompilerSources on Windows #73247

Uh oh!

Conversation

eeckstein commented Apr 25, 2024

Uh oh!

eeckstein commented Apr 25, 2024

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

eeckstein commented Apr 25, 2024

Uh oh!

eeckstein commented Apr 25, 2024

Uh oh!

eeckstein commented Apr 25, 2024

Uh oh!

eeckstein commented Apr 25, 2024

Uh oh!

eeckstein commented Apr 26, 2024

Uh oh!

eeckstein commented Apr 26, 2024

Uh oh!

eeckstein commented Apr 26, 2024

Uh oh!

compnerd commented Apr 26, 2024

Uh oh!

eeckstein commented Apr 29, 2024

Uh oh!

eeckstein commented Apr 29, 2024

Uh oh!

Uh oh!

Uh oh!