[wasi] Bundle `runtimeconfig` in the single file bundle #95377

radical · 2023-11-29T03:19:21Z

This removes the need for having access to a local filesystem for reading the runtimeconfig . Thus the app can be run with wasmtime run foo.wasm.

[wasi] WasiAppBuilder: error out for single file bundles, if there ..
[wasi] driver.c: bundle runtimeconfig.bin, and use that for the single file bundle case
[wasi] Don't add '--dir' for single file bundles

Fixes #94407 .

.. are extra files to deploy, or files for VFS since that's not supported

…e file bundle case

ghost · 2023-11-29T03:19:25Z

Tagging subscribers to 'arch-wasm': @lewing
See info in area-owners.md if you want to be subscribed.

Issue Details

This removes the need for having access to a local filesystem for reading the runtimeconfig .

[wasi] WasiAppBuilder: error out for single file bundles, if there ..
[wasi] driver.c: bundle runtimeconfig.bin, and use that for the single file bundle case
[wasi] Don't add '--dir' for single file bundles

TODO: add test for invariantglobalization

Fixes #94407 .

Author:	radical
Assignees:	-
Labels:	`arch-wasm`
Milestone:	-

radical · 2023-11-29T04:03:51Z

Tests here would overlap with #95281 .

This fixes the case when `RunAOTCompilation=true` in the project, but we only build with relinking.

radical · 2023-11-29T05:36:24Z

/azp run runtime-wasm

azure-pipelines · 2023-11-29T05:36:34Z

Azure Pipelines successfully started running 1 pipeline(s).

radical · 2023-11-29T06:10:15Z

cc @ilonatommy

lewing

are we sure this is how we want to handle extrafilestodeploy in this case?

radical · 2023-11-29T06:37:55Z

are we sure this is how we want to handle extrafilestodeploy in this case?

Till we figure out how we want to handle it, I want to catch attempts to use this when it will just silently fail.

src/mono/wasi/Wasi.Build.Tests/WasiTemplateTests.cs

pavelsavara · 2023-11-29T09:37:14Z

interesting Log

#94820

[05:58:23] info: Discovering: System.Security.Cryptography.Tests.dll (method display = ClassAndMethod, method display options = None)
[05:58:23] warn: 
[05:58:23] warn: Unhandled Exception:
[05:58:23] warn: System.PlatformNotSupportedException: Operation is not supported on this platform.
[05:58:23] warn:    at System.Security.Cryptography.RSA.Create()
[05:58:23] warn:    at System.Security.Cryptography.X509Certificates.Tests.SignatureSupport..cctor()
[05:58:23] warn: 
[05:58:23] warn: Unhandled Exception:
[05:58:23] warn: System.PlatformNotSupportedException: System.Security.Cryptography.X509Certificates is not supported on this platform.
[05:58:23] warn:    at System.Security.Cryptography.X509Certificates.CertificatePal.FromBlob(ReadOnlySpan`1 rawData, SafePasswordHandle password, X509KeyStorageFlags keyStorageFlags)
[05:58:23] warn:    at System.Security.Cryptography.X509Certificates.X509Certificate..ctor(Byte[] rawData, String password, X509KeyStorageFlags keyStorageFlags)
[05:58:23] warn:    at System.Security.Cryptography.X509Certificates.X509Certificate..ctor(Byte[] rawData, String password)
[05:58:23] warn:    at System.Security.Cryptography.X509Certificates.X509Certificate2..ctor(Byte[] rawData, String password)
[05:58:23] warn:    at System.Security.Cryptography.X509Certificates.Tests.CertificateCreation.CertificateRequestChainTests.DetectPssSupport()
[05:58:23] warn:    at System.Security.Cryptography.X509Certificates.Tests.CertificateCreation.CertificateRequestChainTests..cctor()
[05:58:24] warn: 
[05:58:24] warn: Unhandled Exception:
[05:58:24] warn: System.TypeInitializationException: The type initializer for 'CryptoInitializer' threw an exception.
[05:58:24] warn:  ---> System.DllNotFoundException: libSystem.Security.Cryptography.Native.OpenSsl
[05:58:24] warn:    at Interop.CryptoInitializer..cctor()
[05:58:24] warn:    --- End of inner exception stack trace ---
[05:58:24] warn:    at Interop.Crypto..cctor()
[05:58:24] warn: 
[05:58:24] warn: Unhandled Exception:
[05:58:24] warn: System.PlatformNotSupportedException: Operation is not supported on this platform.
[05:58:24] warn:    at System.Security.Cryptography.DSA.CreateCore()
[05:58:24] warn:    at System.Security.Cryptography.DSA.Create()
[05:58:24] warn:    at System.Security.Cryptography.Dsa.Tests.DefaultDSAProvider.Create()
[05:58:24] warn:    at System.Security.Cryptography.Dsa.Tests.DSAFactory.Create()
[05:58:24] warn:    at System.Security.Cryptography.Dsa.Tests.DSAKeyGeneration.GetHasSecondMinSize()
[05:58:24] warn:    at System.Security.Cryptography.Dsa.Tests.DSAKeyGeneration..cctor()
[05:58:26] info: Discovered:  System.Security.Cryptography.Tests.dll (found 1558 of 4184 test cases)
[05:58:26] info: Using random seed for test cases: 1765878136
[05:58:26] info: Using random seed for collections: 1765878136
[05:58:26] info: Starting:    System.Security.Cryptography.Tests.dll
[05:58:26] info: [2023-11-29T05:58:26.769Z] [PASS] System.Security.Cryptography.Tests.Sha256ManagedTests.InvalidInput_NegativeOffset
[05:58:26] info: [2023-11-29T05:58:26.774Z] [PASS] System.Security.Cryptography.Tests.Sha256ManagedTests.HashData_BufferTooSmall

...

[05:58:57] info: [2023-11-29T05:58:57.013Z] [PASS] System.Security.Cryptography.Tests.Sha256Tests.InvalidInput_TooBigOffset
[05:59:10] info: Unable to evaluate script: tab crashed

maraf · 2023-11-29T10:11:47Z

src/mono/wasi/build/WasiApp.targets


    <_RuntimeConfigJsonPath>$([MSBuild]::NormalizePath($(_AppBundleDirForRunCommand), '$(AssemblyName).runtimeconfig.json'))</_RuntimeConfigJsonPath>
-    <RunArguments Condition="'$(RunArguments)' == ''">exec &quot;$([MSBuild]::NormalizePath($(WasmAppHostDir), 'WasmAppHost.dll'))&quot; --runtime-config &quot;$(_RuntimeConfigJsonPath)&quot; $(WasmHostArguments)</RunArguments>
+    <RunArguments Condition="'$(RunArguments)' == '' and '$(WasmSingleFileBundle)' != 'true'">exec &quot;$([MSBuild]::NormalizePath($(WasmAppHostDir), 'WasmAppHost.dll'))&quot; --runtime-config &quot;$(_RuntimeConfigJsonPath)&quot; $(WasmHostArguments)</RunArguments>


nit: I would prefer concatenating the arguments one by one and having a condition on the dir one rather than duplicating the whole line

It's simple enough right now, so I'll keep it as-is. But I wanted to take the same approach as you suggested, but it would make a little more complicated, tracking to make sure that we don't add the extra argument when RunArguments was set before these lines.

fanyang-mono

LGTM overall, except for one question.

src/mono/wasi/runtime/driver.c

radical · 2023-11-29T17:56:45Z

interesting Log

#94820

[05:58:23] info: Discovering: System.Security.Cryptography.Tests.dll (method display = ClassAndMethod, method display options = None)
[05:58:23] warn: 
[05:58:23] warn: Unhandled Exception:
[05:58:23] warn: System.PlatformNotSupportedException: Operation is not supported on this platform.
[05:58:23] warn:    at System.Security.Cryptography.RSA.Create()
[05:58:23] warn:    at System.Security.Cryptography.X509Certificates.Tests.SignatureSupport..cctor()
[05:58:23] warn: 
[05:58:23] warn: Unhandled Exception:
[05:58:23] warn: System.PlatformNotSupportedException: System.Security.Cryptography.X509Certificates is not supported on this platform.
[05:58:23] warn:    at System.Security.Cryptography.X509Certificates.CertificatePal.FromBlob(ReadOnlySpan`1 rawData, SafePasswordHandle password, X509KeyStorageFlags keyStorageFlags)
[05:58:23] warn:    at System.Security.Cryptography.X509Certificates.X509Certificate..ctor(Byte[] rawData, String password, X509KeyStorageFlags keyStorageFlags)
[05:58:23] warn:    at System.Security.Cryptography.X509Certificates.X509Certificate..ctor(Byte[] rawData, String password)
[05:58:23] warn:    at System.Security.Cryptography.X509Certificates.X509Certificate2..ctor(Byte[] rawData, String password)
[05:58:23] warn:    at System.Security.Cryptography.X509Certificates.Tests.CertificateCreation.CertificateRequestChainTests.DetectPssSupport()
[05:58:23] warn:    at System.Security.Cryptography.X509Certificates.Tests.CertificateCreation.CertificateRequestChainTests..cctor()
[05:58:24] warn: 
[05:58:24] warn: Unhandled Exception:
[05:58:24] warn: System.TypeInitializationException: The type initializer for 'CryptoInitializer' threw an exception.
[05:58:24] warn:  ---> System.DllNotFoundException: libSystem.Security.Cryptography.Native.OpenSsl
[05:58:24] warn:    at Interop.CryptoInitializer..cctor()
[05:58:24] warn:    --- End of inner exception stack trace ---
[05:58:24] warn:    at Interop.Crypto..cctor()
[05:58:24] warn: 
[05:58:24] warn: Unhandled Exception:
[05:58:24] warn: System.PlatformNotSupportedException: Operation is not supported on this platform.
[05:58:24] warn:    at System.Security.Cryptography.DSA.CreateCore()
[05:58:24] warn:    at System.Security.Cryptography.DSA.Create()
[05:58:24] warn:    at System.Security.Cryptography.Dsa.Tests.DefaultDSAProvider.Create()
[05:58:24] warn:    at System.Security.Cryptography.Dsa.Tests.DSAFactory.Create()
[05:58:24] warn:    at System.Security.Cryptography.Dsa.Tests.DSAKeyGeneration.GetHasSecondMinSize()
[05:58:24] warn:    at System.Security.Cryptography.Dsa.Tests.DSAKeyGeneration..cctor()
[05:58:26] info: Discovered:  System.Security.Cryptography.Tests.dll (found 1558 of 4184 test cases)
[05:58:26] info: Using random seed for test cases: 1765878136
[05:58:26] info: Using random seed for collections: 1765878136
[05:58:26] info: Starting:    System.Security.Cryptography.Tests.dll
[05:58:26] info: [2023-11-29T05:58:26.769Z] [PASS] System.Security.Cryptography.Tests.Sha256ManagedTests.InvalidInput_NegativeOffset
[05:58:26] info: [2023-11-29T05:58:26.774Z] [PASS] System.Security.Cryptography.Tests.Sha256ManagedTests.HashData_BufferTooSmall

...

[05:58:57] info: [2023-11-29T05:58:57.013Z] [PASS] System.Security.Cryptography.Tests.Sha256Tests.InvalidInput_TooBigOffset
[05:59:10] info: Unable to evaluate script: tab crashed

Yeah -- #94820 . I'll also update xharness to handle this tab crashed case a little better.

radical added 3 commits November 28, 2023 21:53

[wasi] WasiAppBuilder: error out for single file bundles, if there ..

179a7c4

.. are extra files to deploy, or files for VFS since that's not supported

[wasi] driver.c: bundle runtimeconfig.bin, and use that for the singl…

4c0a041

…e file bundle case

[wasi] Don't add '--dir' for single file bundles

b89f55d

radical added the arch-wasm WebAssembly architecture label Nov 29, 2023

ghost added the area-Build-mono label Nov 29, 2023

ghost assigned radical Nov 29, 2023

radical requested review from fanyang-mono, maraf and mdh1418 and removed request for fanyang-mono and mdh1418 November 29, 2023 03:29

radical added 2 commits November 28, 2023 23:37

[wasi] wbt: Add tests for invariantglobalization

67745cb

Use ENABLE_AOT only if we will be AOT'ing

e26cde3

This fixes the case when `RunAOTCompilation=true` in the project, but we only build with relinking.

radical marked this pull request as ready for review November 29, 2023 06:09

radical requested review from kg, lewing, marek-safar and pavelsavara as code owners November 29, 2023 06:09

lewing reviewed Nov 29, 2023

View reviewed changes

build-analysis bot mentioned this pull request Nov 29, 2023

[wasm] Unhandled System.PlatformNotSupportedException for crypto tests #94820

Closed

ilonatommy approved these changes Nov 29, 2023

View reviewed changes

src/mono/wasi/Wasi.Build.Tests/WasiTemplateTests.cs Show resolved Hide resolved

This was referenced Nov 29, 2023

[wasm][debugger] Debugger tests disabled #89409

Open

The active test run was aborted. Reason: Test host process crashed dotnet/dnceng#451

Closed

pavelsavara approved these changes Nov 29, 2023

View reviewed changes

maraf approved these changes Nov 29, 2023

View reviewed changes

fanyang-mono reviewed Nov 29, 2023

View reviewed changes

src/mono/wasi/runtime/driver.c Show resolved Hide resolved

fanyang-mono approved these changes Nov 29, 2023

View reviewed changes

radical merged commit 042c11d into dotnet:main Nov 29, 2023

radical deleted the wasi-fix-runtimeconfig branch November 29, 2023 18:51

github-actions bot locked and limited conversation to collaborators Dec 30, 2023

[wasi] Bundle runtimeconfig in the single file bundle #95377

[wasi] Bundle runtimeconfig in the single file bundle #95377

Uh oh!

Conversation

radical commented Nov 29, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ghost commented Nov 29, 2023

Uh oh!

radical commented Nov 29, 2023

Uh oh!

radical commented Nov 29, 2023

Uh oh!

azure-pipelines bot commented Nov 29, 2023

Uh oh!

radical commented Nov 29, 2023

Uh oh!

lewing left a comment

Choose a reason for hiding this comment

Uh oh!

radical commented Nov 29, 2023

Uh oh!

Uh oh!

pavelsavara commented Nov 29, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

maraf Nov 29, 2023

Choose a reason for hiding this comment

Uh oh!

radical Nov 29, 2023

Choose a reason for hiding this comment

Uh oh!

fanyang-mono left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

radical commented Nov 29, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

[wasi] Bundle `runtimeconfig` in the single file bundle #95377

[wasi] Bundle `runtimeconfig` in the single file bundle #95377

radical commented Nov 29, 2023 •

edited

Loading

pavelsavara commented Nov 29, 2023 •

edited

Loading