SamplingProfiler does not load on Alpine based linux image

[svenrienstra]

Describe the bug

Enabling the sampling profiler on an Alpine based linux image results in a UnsatisfiedLinkError because libasyncProfiler-linux-x64 is loaded instead of the MUSL version.

Steps to reproduce

1. Attach APM agent on Alpine based linux machine (in our case eclipse-temurin:17-alpine)
2. Enable profiling_inferred_spans_enabled

Results in:

java.lang.IllegalStateException: It is likely that /home/appuser/ is not an executable location. Consider setting the profiling_inferred_spans_lib_directory property to a directory on a partition that allows execution
	at co.elastic.apm.agent.profiler.asyncprofiler.AsyncProfiler.getInstance(AsyncProfiler.java:86) ~[elastic-apm-agent-acb9b5f90d88f71f7c4eae5b818f1082-e6256b9ad98d9992211305097d439d61.jar:1.39.0]
	at co.elastic.apm.agent.profiler.SamplingProfiler.profile(SamplingProfiler.java:395) ~[elastic-apm-agent-acb9b5f90d88f71f7c4eae5b818f1082-e6256b9ad98d9992211305097d439d61.jar:1.39.0]
	at co.elastic.apm.agent.profiler.SamplingProfiler.run(SamplingProfiler.java:376) [elastic-apm-agent-acb9b5f90d88f71f7c4eae5b818f1082-e6256b9ad98d9992211305097d439d61.jar:1.39.0]
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539) [?:?]
	at java.util.concurrent.FutureTask.run(FutureTask.java:264) [?:?]
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304) [?:?]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) [?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) [?:?]
	at co.elastic.apm.agent.util.ExecutorUtils$2.run(ExecutorUtils.java:99) [elastic-apm-agent-acb9b5f90d88f71f7c4eae5b818f1082-e6256b9ad98d9992211305097d439d61.jar:1.39.0]
	at java.lang.Thread.run(Thread.java:833) [?:?]
Caused by: java.lang.UnsatisfiedLinkError: /home/appuser/libasyncProfiler-linux-x64-acb9b5f90d88f71f7c4eae5b818f1082-c6d96bf140e0562f0ce2765a99ced1c2.so: Error loading shared library libstdc++.so.6: No such file or directory (needed by /home/appuser/libasyncProfiler-linux-x64-acb9b5f90d88f71f7c4eae5b818f1082-c6d96bf140e0562f0ce2765a99ced1c2.so)
	at jdk.internal.loader.NativeLibraries.load(Native Method) ~[?:?]
	at jdk.internal.loader.NativeLibraries$NativeLibraryImpl.open(NativeLibraries.java:388) ~[?:?]
	at jdk.internal.loader.NativeLibraries.loadLibrary(NativeLibraries.java:232) ~[?:?]
	at jdk.internal.loader.NativeLibraries.loadLibrary(NativeLibraries.java:174) ~[?:?]
	at java.lang.ClassLoader.loadLibrary(ClassLoader.java:2389) ~[?:?]
	at java.lang.Runtime.load0(Runtime.java:755) ~[?:?]
	at java.lang.System.load(System.java:1953) ~[?:?]
	at co.elastic.apm.agent.profiler.asyncprofiler.AsyncProfiler.loadNativeLibrary(AsyncProfiler.java:106) ~[elastic-apm-agent-acb9b5f90d88f71f7c4eae5b818f1082-e6256b9ad98d9992211305097d439d61.jar:1.39.0]
	at co.elastic.apm.agent.profiler.asyncprofiler.AsyncProfiler.getInstance(AsyncProfiler.java:84) ~[elastic-apm-agent-acb9b5f90d88f71f7c4eae5b818f1082-e6256b9ad98d9992211305097d439d61.jar:1.39.0]
	... 9 more

Note that we're running with ENV LD_PRELOAD=/lib/libgcompat.so.0:/usr/lib/libunwind.so.8 because of a Netty native issue, without that you might get another exception

Expected behavior

Load the MUSL library. Either by detecting Alpine environment somehow or by adding a configuration option to instruct to load MUSL library version.

Debug logs

2023-07-03 08:27:15,716 [Attach Listener] INFO  co.elastic.apm.agent.configuration.StartupInfo - Starting Elastic APM 1.39.0 as ** on Java 17.0.7 Runtime version: 17.0.7+7 VM version: 17.0.7+7 (Eclipse Adoptium) Linux 5.10.176+
2023-07-03 08:27:15,718 [Attach Listener] INFO  co.elastic.apm.agent.configuration.StartupInfo - environment: 'NL-DEV' (source: /configuration/elasticapm.properties)
2023-07-03 08:27:15,720 [Attach Listener] INFO  co.elastic.apm.agent.configuration.StartupInfo - config_file: '/configuration/elasticapm.properties' (source: Environment Variables)
2023-07-03 08:27:15,721 [Attach Listener] INFO  co.elastic.apm.agent.configuration.StartupInfo - plugins_dir: '/home/appuser/apm-plugins' (source: Environment Variables)
2023-07-03 08:27:15,721 [Attach Listener] INFO  co.elastic.apm.agent.configuration.StartupInfo - activation_method: 'PROGRAMMATIC_SELF_ATTACH' (source: Attachment configuration)
2023-07-03 08:27:15,722 [Attach Listener] INFO  co.elastic.apm.agent.configuration.StartupInfo - secret_token: 'XXXX' (source: /configuration/elasticapm.properties)
2023-07-03 08:27:15,723 [Attach Listener] INFO  co.elastic.apm.agent.configuration.StartupInfo - server_urls: '**' (source: /configuration/elasticapm.properties)
2023-07-03 08:27:15,725 [Attach Listener] INFO  co.elastic.apm.agent.configuration.StartupInfo - application_packages: 'io.h4h' (source: /configuration/elasticapm.properties)
2023-07-03 08:27:15,725 [Attach Listener] INFO  co.elastic.apm.agent.configuration.StartupInfo - profiling_inferred_spans_enabled: 'true' (source: Attachment configuration)
2023-07-03 08:27:15,726 [Attach Listener] INFO  co.elastic.apm.agent.configuration.StartupInfo - profiling_inferred_spans_lib_directory: '/home/appuser/' (source: Attachment configuration)

[jackshirazi]

Hi, thank you for the bug report. We'd love to add support for asyncprofiler to run under alpine, but unfortunately we don't have the resources to prioritize this any time soon, especially given the low demand for this combination. We'd be delighted for anyone to contribute a PR to enable this (probably easiest done by adding a config option to select the alpine library as we think it's difficult to auto-determine that the image is specifically alpine).