[5.9] Enable macros on Windows #68743

bnbarham · 2023-09-25T18:48:03Z

Use WiX to extract a future toolchain to allow building the macros support when building the toolchain. We build swift-syntax and now wire that into the build itself.

try a more recent toolchain

bnbarham · 2023-09-25T18:49:50Z

swiftlang/swift-installer-scripts#246

@swift-ci please build toolchain windows platform

tristanlabelle · 2023-09-25T20:37:10Z

FileNotFoundError: [WinError 3] The system cannot find the path specified: 'C:\\Users\\swift-ci\\jenkins\\workspace\\swift-PR-build-toolchain-windows\\build\\1\\tools\\swift\\unittests\\SourceKit\\SwiftLang\\CMakeFiles\\SourceKitSwiftLangTests.dir\\C_\\Users\\swift-ci\\jenkins\\workspace\\swift-PR-build-toolchain-windows\\llvm-project\\llvm\\resources\\.'

MAX_PATH is 260 and this path is 256 chars, so it's not clear if this is a MAX_PATH issue. However it should never have used the path C:\Users\swift-ci\jenkins\workspace\swift-PR-build-toolchain\build since we map that to the T: drive. There's some realpath resolution that is still happening in here somewhere.

bnbarham · 2023-09-25T20:39:20Z

However it should never have used the path C:\Users\swift-ci\jenkins\workspace\swift-PR-build-toolchain\build since we map that to the T: drive. There's some realpath resolution that is still happening in here somewhere.

To be clear, I reverted all the substitution changes here. But given this is in lit.py and we already substitute T:, I could add back in the lit related ones?

EDIT: I've added 7d13da4 in and will run with the LLVM lit changes as well. The other Swift changes are related to the source directory substitution.

(cherry picked from commit 7d13da4)

bnbarham · 2023-09-25T20:46:26Z

swiftlang/llvm-project#7513
swiftlang/swift-installer-scripts#246

@swift-ci please build toolchain windows platform

tristanlabelle · 2023-09-25T20:57:18Z

@bnbarham I think we need this change from your previous PR:

This is what caused the expansion of the T:\ build directory I believe.

bnbarham · 2023-09-25T20:59:27Z

I don't think we can take that one, or at least if we did we'd have to split it to do only the swift_obj_root. Otherwise we'll get mismatches with SOURCE_DIR like we were seeing in the previous PR. Though I suspect we'd get issues with mismatching BUILD_DIR as well even if we only take that half 😅

EDIT: Oh, of course this isn't true because we're not using a substitute drive for SOURCE_DIR any more. And this must work because main has those changes already. So... picking up that change.

(cherry picked from commit 136642c)

bnbarham · 2023-09-25T23:37:25Z

swiftlang/llvm-project#7513
swiftlang/swift-installer-scripts#246

@swift-ci please build toolchain windows platform

tristanlabelle · 2023-09-26T13:57:09Z

@hyp Any idea about this test failure? I'm out of my depth. It seems like the C++ interop method is called with typed instead of untyped parameters? 🤔

FAIL: Swift(windows-x86_64) :: Interop/Cxx/class/method/msvc-abi-return-indirect-trivial-record.swift (3313 of 9544)
******************** TEST 'Swift(windows-x86_64) :: Interop/Cxx/class/method/msvc-abi-return-indirect-trivial-record.swift' FAILED ********************
Script:
--
: 'RUN: at line 1';   rm -rf T:\1\tools\swift\test-windows-x86_64\Interop\Cxx\class\method\Output\msvc-abi-return-indirect-trivial-record.swift.tmp
: 'RUN: at line 2';   split-file C:\Users\swift-ci\jenkins\workspace\swift-PR-build-toolchain-windows\swift\test\Interop\Cxx\class\method\msvc-abi-return-indirect-trivial-record.swift T:\1\tools\swift\test-windows-x86_64\Interop\Cxx\class\method\Output\msvc-abi-return-indirect-trivial-record.swift.tmp
: 'RUN: at line 3';   't:\\1\\bin\\swift-frontend.exe' -target x86_64-unknown-windows-msvc  -module-cache-path T:\1\swift-test-results\x86_64-unknown-windows-msvc\clang-module-cache -vfsoverlay T:/1/tools/swift\stdlib\windows-vfs-overlay.yaml -swift-version 4  -define-availability 'SwiftStdlib 9999:macOS 9999, iOS 9999, watchOS 9999, tvOS 9999' -define-availability 'SwiftStdlib 5.0:macOS 10.14.4, iOS 12.2, watchOS 5.2, tvOS 12.2' -define-availability 'SwiftStdlib 5.1:macOS 10.15, iOS 13.0, watchOS 6.0, tvOS 13.0' -define-availability 'SwiftStdlib 5.2:macOS 10.15.4, iOS 13.4, watchOS 6.2, tvOS 13.4' -define-availability 'SwiftStdlib 5.3:macOS 11.0, iOS 14.0, watchOS 7.0, tvOS 14.0' -define-availability 'SwiftStdlib 5.4:macOS 11.3, iOS 14.5, watchOS 7.4, tvOS 14.5' -define-availability 'SwiftStdlib 5.5:macOS 12.0, iOS 15.0, watchOS 8.0, tvOS 15.0' -define-availability 'SwiftStdlib 5.6:macOS 12.3, iOS 15.4, watchOS 8.5, tvOS 15.4' -define-availability 'SwiftStdlib 5.7:macOS 13.0, iOS 16.0, watchOS 9.0, tvOS 16.0' -define-availability 'SwiftStdlib 5.8:macOS 13.3, iOS 16.4, watchOS 9.4, tvOS 16.4' -define-availability 'SwiftStdlib 5.9:macOS 14.0, iOS 17.0, watchOS 10.0, tvOS 17.0' -define-availability 'SwiftStdlib 5.10:macOS 9999, iOS 9999, watchOS 9999, tvOS 9999' -typo-correction-limit 10  -emit-irgen -I T:\1\tools\swift\test-windows-x86_64\Interop\Cxx\class\method\Output\msvc-abi-return-indirect-trivial-record.swift.tmp/Inputs -enable-experimental-cxx-interop  -Xcc -std=c++17 T:\1\tools\swift\test-windows-x86_64\Interop\Cxx\class\method\Output\msvc-abi-return-indirect-trivial-record.swift.tmp/test.swift -module-name Test | "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python39_64\python.exe" C:\Users\swift-ci\jenkins\workspace\swift-PR-build-toolchain-windows\swift\utils\PathSanitizingFileCheck --allow-unused-prefixes --sanitize BUILD_DIR=T:/1/tools/swift --sanitize SOURCE_DIR=C:/Users/swift-ci/jenkins/workspace/swift-PR-build-toolchain-windows/swift --use-filecheck t:\1\bin\filecheck.exe --enable-windows-compatibility C:\Users\swift-ci\jenkins\workspace\swift-PR-build-toolchain-windows\swift\test\Interop\Cxx\class\method\msvc-abi-return-indirect-trivial-record.swift
--
Exit Code: 1

Command Output (stdout):
--
$ ":" "RUN: at line 1"
$ "rm" "-rf" "T:\1\tools\swift\test-windows-x86_64\Interop\Cxx\class\method\Output\msvc-abi-return-indirect-trivial-record.swift.tmp"
$ ":" "RUN: at line 2"
$ "split-file" "C:\Users\swift-ci\jenkins\workspace\swift-PR-build-toolchain-windows\swift\test\Interop\Cxx\class\method\msvc-abi-return-indirect-trivial-record.swift" "T:\1\tools\swift\test-windows-x86_64\Interop\Cxx\class\method\Output\msvc-abi-return-indirect-trivial-record.swift.tmp"
$ ":" "RUN: at line 3"
$ "t:\\1\\bin\\swift-frontend.exe" "-target" "x86_64-unknown-windows-msvc" "-module-cache-path" "T:\1\swift-test-results\x86_64-unknown-windows-msvc\clang-module-cache" "-vfsoverlay" "T:/1/tools/swift\stdlib\windows-vfs-overlay.yaml" "-swift-version" "4" "-define-availability" "SwiftStdlib 9999:macOS 9999, iOS 9999, watchOS 9999, tvOS 9999" "-define-availability" "SwiftStdlib 5.0:macOS 10.14.4, iOS 12.2, watchOS 5.2, tvOS 12.2" "-define-availability" "SwiftStdlib 5.1:macOS 10.15, iOS 13.0, watchOS 6.0, tvOS 13.0" "-define-availability" "SwiftStdlib 5.2:macOS 10.15.4, iOS 13.4, watchOS 6.2, tvOS 13.4" "-define-availability" "SwiftStdlib 5.3:macOS 11.0, iOS 14.0, watchOS 7.0, tvOS 14.0" "-define-availability" "SwiftStdlib 5.4:macOS 11.3, iOS 14.5, watchOS 7.4, tvOS 14.5" "-define-availability" "SwiftStdlib 5.5:macOS 12.0, iOS 15.0, watchOS 8.0, tvOS 15.0" "-define-availability" "SwiftStdlib 5.6:macOS 12.3, iOS 15.4, watchOS 8.5, tvOS 15.4" "-define-availability" "SwiftStdlib 5.7:macOS 13.0, iOS 16.0, watchOS 9.0, tvOS 16.0" "-define-availability" "SwiftStdlib 5.8:macOS 13.3, iOS 16.4, watchOS 9.4, tvOS 16.4" "-define-availability" "SwiftStdlib 5.9:macOS 14.0, iOS 17.0, watchOS 10.0, tvOS 17.0" "-define-availability" "SwiftStdlib 5.10:macOS 9999, iOS 9999, watchOS 9999, tvOS 9999" "-typo-correction-limit" "10" "-emit-irgen" "-I" "T:\1\tools\swift\test-windows-x86_64\Interop\Cxx\class\method\Output\msvc-abi-return-indirect-trivial-record.swift.tmp/Inputs" "-enable-experimental-cxx-interop" "-Xcc" "-std=c++17" "T:\1\tools\swift\test-windows-x86_64\Interop\Cxx\class\method\Output\msvc-abi-return-indirect-trivial-record.swift.tmp/test.swift" "-module-name" "Test"
# command stderr:
<unknown>:0: warning: the '-enable-experimental-cxx-interop' flag is deprecated; please pass '-cxx-interoperability-mode=' instead

<unknown>:0: note: Swift will maintain source compatibility for imported APIs based on the selected compatibility mode, so updating the Swift compiler will not change how APIs are imported


$ "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python39_64\python.exe" "C:\Users\swift-ci\jenkins\workspace\swift-PR-build-toolchain-windows\swift\utils\PathSanitizingFileCheck" "--allow-unused-prefixes" "--sanitize" "BUILD_DIR=T:/1/tools/swift" "--sanitize" "SOURCE_DIR=C:/Users/swift-ci/jenkins/workspace/swift-PR-build-toolchain-windows/swift" "--use-filecheck" "t:\1\bin\filecheck.exe" "--enable-windows-compatibility" "C:\Users\swift-ci\jenkins\workspace\swift-PR-build-toolchain-windows\swift\test\Interop\Cxx\class\method\msvc-abi-return-indirect-trivial-record.swift"
# command stderr:
C:\Users\swift-ci\jenkins\workspace\swift-PR-build-toolchain-windows\swift\test\Interop\Cxx\class\method\msvc-abi-return-indirect-trivial-record.swift:60:11: error: CHECK: expected string not found in input
// CHECK: call void @"?begin@?$vec@Vstr@@@@QEBA?AVIt@@XZ"(ptr %[[RESULT]], ptr sret{{.*}}
          ^
<stdin>:30:142: note: scanning from here
define dllexport swiftcc i32 @"$s4Test4testys5Int32VSo0014vecstr_yuJCataVF"(%TSo0014vecstr_yuJCataV* noalias nocapture dereferenceable(4) %0) #0 {
                                                                                                                                             ^
<stdin>:30:142: note: with "RESULT" equal to "0"
define dllexport swiftcc i32 @"$s4Test4testys5Int32VSo0014vecstr_yuJCataVF"(%TSo0014vecstr_yuJCataV* noalias nocapture dereferenceable(4) %0) #0 {
                                                                                                                                             ^
<stdin>:44:2: note: possible intended match here
 call void @"?begin@?$vec@Vstr@@@@QEBA?AVIt@@XZ"(%class.vec* %4, %class.It* sret(%class.It) %5)
 ^

Input file: <stdin>
Check file: C:\Users\swift-ci\jenkins\workspace\swift-PR-build-toolchain-windows\swift\test\Interop\Cxx\class\method\msvc-abi-return-indirect-trivial-record.swift

-dump-input=help explains the following input dump.

Input was:
<<<<<<
            .
            .
            .
           25: entry: 
           26:  %2 = bitcast i8** %1 to i8* 
           27:  ret i32 0 
           28: } 
           29:  
           30: define dllexport swiftcc i32 @"$s4Test4testys5Int32VSo0014vecstr_yuJCataVF"(%TSo0014vecstr_yuJCataV* noalias nocapture dereferenceable(4) %0) #0 { 
check:60'0                                                                                                                                                  X~~~~~ error: no match found
check:60'1                                                                                                                                                         with "RESULT" equal to "0"
           31: entry: 
check:60'0     ~~~~~~~
           32:  %result.debug = alloca %TSo0014vecstr_yuJCataV*, align 8 
check:60'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
           33:  %1 = bitcast %TSo0014vecstr_yuJCataV** %result.debug to i8* 
check:60'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
           34:  call void @llvm.memset.p0i8.i64(i8* align 8 %1, i8 0, i64 8, i1 false) 
check:60'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
           35:  %call.aggresult = alloca %TSo2ItV, align 4 
check:60'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
            .
            .
            .
           39:  store %TSo0014vecstr_yuJCataV* %0, %TSo0014vecstr_yuJCataV** %result.debug, align 8 
check:60'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
           40:  %3 = bitcast %TSo2ItV* %call.aggresult to i8* 
check:60'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
           41:  call void @llvm.lifetime.start.p0i8(i64 4, i8* %3) 
check:60'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
           42:  %4 = bitcast %TSo0014vecstr_yuJCataV* %0 to %class.vec* 
check:60'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
           43:  %5 = bitcast %TSo2ItV* %call.aggresult to %class.It* 
check:60'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
           44:  call void @"?begin@?$vec@Vstr@@@@QEBA?AVIt@@XZ"(%class.vec* %4, %class.It* sret(%class.It) %5) 
check:60'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
check:60'2      ?                                                                                               possible intended match
           45:  %call.aggresult.x = getelementptr inbounds %TSo2ItV, %TSo2ItV* %call.aggresult, i32 0, i32 0 
check:60'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
           46:  %call.aggresult.x._value = getelementptr inbounds %Ts5Int32V, %Ts5Int32V* %call.aggresult.x, i32 0, i32 0 
check:60'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
           47:  %6 = load i32, i32* %call.aggresult.x._value, align 4 
check:60'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
           48:  %7 = bitcast %TSo2ItV* %call.aggresult to i8* 
check:60'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
           49:  call void @llvm.lifetime.end.p0i8(i64 4, i8* %7) 
check:60'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
            .
            .
            .
>>>>>>

error: command failed with exit status: 1

hyp · 2023-09-26T22:32:29Z

@hyp Any idea about this test failure? I'm out of my depth. It seems like the C++ interop method is called with typed instead of untyped parameters? 🤔

I'm not sure why this test is failing on this run of building the toolchain, perhaps this test is somehow missed by regular windows PR CI? In any case, this failure is due to the test assuming opaque pointer types are enabled in LLVM, but the 5.9 LLVM branch assumes that opaque pointer types are not enabled. I think it's fine to fixup the test case's CHECK line in this patch.

bnbarham · 2023-09-26T23:13:27Z

swiftlang/llvm-project#7513
swiftlang/swift-installer-scripts#246

@swift-ci please build toolchain windows platform

bnbarham · 2023-09-26T23:14:01Z

Thanks @hyp! Re-running with those changes now.

5.9 has been running builds in CI without failing when tests failed, xfail the only failing test.

bnbarham · 2023-09-27T04:20:40Z

swiftlang/llvm-project#7513
swiftlang/swift-installer-scripts#246

@swift-ci please build toolchain windows platform

bnbarham · 2023-09-27T06:28:40Z

swiftlang/llvm-project#7513
swiftlang/swift-installer-scripts#246

@swift-ci please test

Squashed cherry-pick of swiftlang#68334 and swiftlang#68743.

compnerd and others added 9 commits September 25, 2023 11:46

build: enable macros during the build

4d64f27

Use WiX to extract a future toolchain to allow building the macros support when building the toolchain. We build swift-syntax and now wire that into the build itself.

Update build-windows-toolchain.bat

e201aa0

Update build-windows-toolchain.bat

422008e

Update build-windows-toolchain.bat

390f6e7

try a more recent toolchain

Update build-windows-toolchain.bat

3b76bda

Update build-windows-toolchain.bat

8fc3b7f

Update build-windows-toolchain.bat

c62d41d

Update lit.cfg

8c88f0f

Fix silent test failures in the Windows CI

254feac

bnbarham requested a review from a team as a code owner September 25, 2023 18:48

tristanlabelle mentioned this pull request Sep 25, 2023

[5.9] Enable macros on Windows #68692

Closed

Remove realpath in lit site config.

fea49fd

(cherry picked from commit 7d13da4)

Use llvm's abs_path_preserve_drive (swiftlang#68038)

2aeec43

(cherry picked from commit 136642c)

[Test] Add xfail for failing test

d1553c9

5.9 has been running builds in CI without failing when tests failed, xfail the only failing test.

bnbarham force-pushed the compnerd-59-windows-no-subst branch from 0b72f18 to d1553c9 Compare September 27, 2023 04:16

tristanlabelle approved these changes Sep 27, 2023

View reviewed changes

bnbarham changed the title ~~[5.9] Enable macros on Windows (without drive substitution)~~ [5.9] Enable macros on Windows Sep 27, 2023

DougGregor approved these changes Sep 28, 2023

View reviewed changes

bnbarham mentioned this pull request Sep 28, 2023

[5.9] Cherry-pick substitution drive fixes for Windows swiftlang/llvm-project#7513

Merged

bnbarham merged commit 102c60b into swiftlang:release/5.9 Sep 28, 2023

bnbarham deleted the compnerd-59-windows-no-subst branch September 28, 2023 16:53

bnbarham pushed a commit to bnbarham/swift that referenced this pull request Sep 29, 2023

Enable macros for Windows in 5.9.1

decb0fb

Squashed cherry-pick of swiftlang#68334 and swiftlang#68743.

bnbarham mentioned this pull request Sep 29, 2023

[5.9.1] Enable macros for Windows in 5.9.1 #68858

Merged

bnbarham pushed a commit to bnbarham/swift that referenced this pull request Sep 29, 2023

Enable macros for Windows in 5.9.1

eba2a02

Squashed cherry-pick of swiftlang#68334 and swiftlang#68743.

bnbarham pushed a commit to bnbarham/swift that referenced this pull request Oct 3, 2023

Enable macros for Windows in 5.9.1

95a66c5

Squashed cherry-pick of swiftlang#68334 and swiftlang#68743.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[5.9] Enable macros on Windows #68743

[5.9] Enable macros on Windows #68743

Uh oh!

bnbarham commented Sep 25, 2023 •

edited

Loading

Uh oh!

bnbarham commented Sep 25, 2023

Uh oh!

tristanlabelle commented Sep 25, 2023 •

edited

Loading

Uh oh!

bnbarham commented Sep 25, 2023 •

edited

Loading

Uh oh!

bnbarham commented Sep 25, 2023

Uh oh!

tristanlabelle commented Sep 25, 2023

Uh oh!

bnbarham commented Sep 25, 2023 •

edited

Loading

Uh oh!

bnbarham commented Sep 25, 2023

Uh oh!

tristanlabelle commented Sep 26, 2023

Uh oh!

hyp commented Sep 26, 2023

Uh oh!

bnbarham commented Sep 26, 2023

Uh oh!

bnbarham commented Sep 26, 2023

Uh oh!

bnbarham commented Sep 27, 2023

Uh oh!

bnbarham commented Sep 27, 2023

Uh oh!

Uh oh!

[5.9] Enable macros on Windows #68743

[5.9] Enable macros on Windows #68743

Uh oh!

Conversation

bnbarham commented Sep 25, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bnbarham commented Sep 25, 2023

Uh oh!

tristanlabelle commented Sep 25, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bnbarham commented Sep 25, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bnbarham commented Sep 25, 2023

Uh oh!

tristanlabelle commented Sep 25, 2023

Uh oh!

bnbarham commented Sep 25, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bnbarham commented Sep 25, 2023

Uh oh!

tristanlabelle commented Sep 26, 2023

Uh oh!

hyp commented Sep 26, 2023

Uh oh!

bnbarham commented Sep 26, 2023

Uh oh!

bnbarham commented Sep 26, 2023

Uh oh!

bnbarham commented Sep 27, 2023

Uh oh!

bnbarham commented Sep 27, 2023

Uh oh!

Uh oh!

bnbarham commented Sep 25, 2023 •

edited

Loading

tristanlabelle commented Sep 25, 2023 •

edited

Loading

bnbarham commented Sep 25, 2023 •

edited

Loading

bnbarham commented Sep 25, 2023 •

edited

Loading