OpenSSL 3.1

[ManickaP]

Describe the bug

The build scripts don't work with newer OpenSSL 3 versions

-- Found OpenSSL: /usr/lib/libcrypto.so (found version "3.1.1")
CMake Error at submodules/CMakeLists.txt:349 (message):
  OpenSSL 3.0 not found, found 3.1.1

@wfurt

Affected OS

• Windows
• Linux
• macOS
• Other (specify below)

Additional OS information

No response

MsQuic version

main

Steps taken to reproduce bug

Build msquic on Linux with OpenSSL newer than 3.0.*.

Expected behavior

It will build.

Actual outcome

Doesn't build.

Additional details

Workaround is not to use system crypto.

[nibanks]

We don't explicitly specify a 3.0 dependency anywhere I know of. Our package just specifies 3 as the dependency. Also, our system crypto tests are testing exactly what you seem to be doing. We test Ubuntu 22.04 using our system crypto built against 3.1, which passes.

[ManickaP]

I think this is the culprit:

msquic/submodules/CMakeLists.txt

Lines 339 to 353 in 88f0269

	if (QUIC_USE_SYSTEM_LIBCRYPTO)
	include(FindOpenSSL)
	if (OPENSSL_FOUND)
	# Get Major.Minor so 3.0 can match 3.0.2.
	# We cannot use VERSION_GREATER as 3.0 would work for 1.1.1 but they are not compatible.
	string(FIND ${OPENSSL_VERSION} "." VERSIONLEN REVERSE)
	string(SUBSTRING ${OPENSSL_VERSION} 0 ${VERSIONLEN} OPENSSL_MAJORMINOR)
	if (OPENSSL_VERSION VERSION_EQUAL EXPECTED_OPENSSL_VERSION OR OPENSSL_MAJORMINOR VERSION_EQUAL EXPECTED_OPENSSL_VERSION)
	target_link_libraries(OpenSSLQuic INTERFACE OpenSSL::Crypto)
	else()
	message(FATAL_ERROR "OpenSSL ${EXPECTED_OPENSSL_VERSION} not found, found ${OPENSSL_VERSION}")
	endif()
	else()
	message(FATAL_ERROR "System OpenSSL not found when requested")
	endif()

EDIT: It works if I change this line to 3.1:

msquic/submodules/CMakeLists.txt

Line 24 in 88f0269

set(EXPECTED_OPENSSL_VERSION 3.0)

[nibanks]

Ah, you're probably right. Is there anyway to just link to the MAJOR version only, instead of 3.0? Does this work @ManickaP?

set(EXPECTED_OPENSSL_VERSION 3) 

[nibanks]

cc @wfurt & @csujedihy

[ManickaP]

Ah, you're probably right. Is there anyway to just link to the MAJOR version only, instead of 3.0? Does this work @ManickaP?

set(EXPECTED_OPENSSL_VERSION 3) 

Doesn't work.

[wfurt]

From what I remember the problem was comparisons e.g. 3.x >= 1.1.1. But since 1.x is done now we can flip it.

[vindicatorr]

I'm just going to subscribe to this issue since https://learn.microsoft.com/en-us/aspnet/core/fundamentals/servers/kestrel/http3 (which I see referenced here: #2291) doesn't seem to be enough:

Exception has occurred: CLR/System.InvalidOperationException
An unhandled exception of type 'System.InvalidOperationException' occurred in Microsoft.AspNetCore.Server.Kestrel.Core.dll: 'Unable to bind an HTTP/3 endpoint. This could be because QUIC has not been configured using UseQuic, or the platform doesn't support QUIC or HTTP/3.'
   at Microsoft.AspNetCore.Server.Kestrel.Core.KestrelServerImpl.<>c__DisplayClass28_0`1.<<StartAsync>g__OnBind|0>d.MoveNext()
--- End of stack trace from previous location ---
   at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.AddressBinder.BindEndpointAsync(ListenOptions endpoint, AddressBindContext context, CancellationToken cancellationToken)
   at Microsoft.AspNetCore.Server.Kestrel.Core.ListenOptions.BindAsync(AddressBindContext context, CancellationToken cancellationToken)
   at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.AddressBinder.EndpointsStrategy.BindAsync(AddressBindContext context, CancellationToken cancellationToken)
   at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.AddressBinder.BindAsync(ListenOptions[] listenOptions, AddressBindContext context, Func`2 useHttps, CancellationToken cancellationToken)
   at Microsoft.AspNetCore.Server.Kestrel.Core.KestrelServerImpl.BindAsync(CancellationToken cancellationToken)
   at Microsoft.AspNetCore.Server.Kestrel.Core.KestrelServerImpl.StartAsync[TContext](IHttpApplication`1 application, CancellationToken cancellationToken)
   at Microsoft.AspNetCore.Hosting.GenericWebHostService.StartAsync(CancellationToken cancellationToken)
   at Microsoft.Extensions.Hosting.Internal.Host.<StartAsync>b__14_1(IHostedService service, CancellationToken token)
   at Microsoft.Extensions.Hosting.Internal.Host.ForeachService[T](IEnumerable`1 services, CancellationToken token, Boolean concurrent, Boolean abortOnFirstException, List`1 exceptions, Func`3 operation)
   at Microsoft.Extensions.Hosting.Internal.Host.StartAsync(CancellationToken cancellationToken)
   at Microsoft.Extensions.Hosting.HostingAbstractionsHostExtensions.RunAsync(IHost host, CancellationToken token)
   at Microsoft.Extensions.Hosting.HostingAbstractionsHostExtensions.RunAsync(IHost host, CancellationToken token)
   at Microsoft.Extensions.Hosting.HostingAbstractionsHostExtensions.Run(IHost host)
   at Main(String[] pArgs)

I'm on Arch Linux and snagged https://packages.microsoft.com/ubuntu/23.04/prod/pool/main/libm/libmsquic/, then ran debtap to convert the format, and pacman to install it.
My "platform [DOES] support QUIC or HTTP/3", as I've enabled Quic in my browser via chrome://flags, and tested it from cloudflare while looking at protocol in chrome's dev-tools.

I saw mention that the openssl fork is required, but I'm just assuming that's only for building it since in the .deb manifest, I see mention of openssl, but no specified "fork".

I was going to run the repo test, but I guess it's required to build the repo first, even though I already installed libmsquic.

EDIT0: Seeing #3664, I went ahead and ran ldd on the 2 libs and saw:
"liblttng-ust.so.1 => not found" and "libcrypto.so.1.1 => not found" (odd given I have "core/openssl 1.1.1.k-1 [installed: 3.1.1-1]" containing "usr/lib/libcrypto.so.1.1".

EDIT1: Installed lttng-ust to take care of that one, and openssl-1.1 for the other.
Nope, same error message even though ldd gives me a free-and-clear:

$ ldd /usr/lib/x86_64-linux-gnu/libmsquic.lttng.so.2.2.2
        linux-vdso.so.1 (0x00007fff023ce000)
        liblttng-ust.so.1 => /usr/lib/liblttng-ust.so.1 (0x00007f4ae221a000)
        libdl.so.2 => /usr/lib/libdl.so.2 (0x00007f4ae2215000)
        libc.so.6 => /usr/lib/libc.so.6 (0x00007f4ae202b000)
        libnuma.so.1 => /usr/lib/libnuma.so.1 (0x00007f4ae201d000)
        liblttng-ust-common.so.1 => /usr/lib/liblttng-ust-common.so.1 (0x00007f4ae2010000)
        liblttng-ust-tracepoint.so.1 => /usr/lib/liblttng-ust-tracepoint.so.1 (0x00007f4ae1ff2000)
        /usr/lib64/ld-linux-x86-64.so.2 (0x00007f4ae245c000)

$ ldd /usr/lib/x86_64-linux-gnu/libmsquic.so.2.2.2
        linux-vdso.so.1 (0x00007fff4bb76000)
        libcrypto.so.1.1 => /usr/lib/libcrypto.so.1.1 (0x00007fb93c43a000)
        libdl.so.2 => /usr/lib/libdl.so.2 (0x00007fb93c435000)
        libnuma.so.1 => /usr/lib/libnuma.so.1 (0x00007fb93c427000)
        libpthread.so.0 => /usr/lib/libpthread.so.0 (0x00007fb93c422000)
        libc.so.6 => /usr/lib/libc.so.6 (0x00007fb93c238000)
        /usr/lib64/ld-linux-x86-64.so.2 (0x00007fb93c8af000)

EDIT2: Got it! ldconfig needed /usr/lib/x86_64-linux-gnu set in the path in /etc/ld.so.conf.d/<whatever>.conf

[wfurt]

It is strange that you need OpenSSL 1.1 Starting with Ubuntu 22.04 MsQuic should depend on OpenSSL 3. And that what Arch uses by default, right @vindicatorr?

And yes, ldd is good friend -> note that Quic is trying to load libmsquic.so.2. (you would see that with LD_DEBUG=libs) If in doubts you can always build simple app and check QuicListener.IsSupported` to avoid all the Kestrel layers.

[vindicatorr]

should depend on OpenSSL 3. And that what Arch uses by default

Correct:

$ pacman -Qi openssl
Name            : openssl
Version         : 3.1.2-1

[wfurt]

Could you grab binaries from https://packages.microsoft.com/ubuntu/22.04 and give it another try @vindicatorr?
Also note that there is Arch linux package and it seems to be updated to latest: https://aur.archlinux.org/packages/msquic

[vindicatorr]

Well that's interesting that 22 would be aligned with openssl-3 unlike ubuntu-v23 depending on 1.1...
Right off the bat, when running ldd on the lib before installing it (during debtap extraction), I see libcrypto.so.3 => /usr/lib/libcrypto.so.3 (0x00007f62e6164000).

The only thing I had to do was change the libssl3 dependency in the manifest (during debtap) to openssl (or I could have chosen to skip for Do you want to skip the above package for this upgrade).
I didn't even need lttng-ust, so I uninstalled that and openssl-1.1, and it still worked. Thanks.

I'm not sure I'd be want to go with the aur release given that it's v2.2 (same as the .deb), but also downloads other sources in order to build it like googletest. I figure I'll just stick with the pre-built .deb.