Limit external configuration sources parsing in native-mode #42140

zakkak · 2024-07-25T13:19:15Z

SmallryeConfigBuilder tries to access properties files from various sources, despite them not being available at run-time nor supported in native-mode, see #41994

This patch also avoids getting a MissingRegistrationError when using -H:+ThrowMissingRegistrationErrors or --exact-reachability-metadata.

Related to #41994 and #41995

Supersedes #42103

zakkak · 2024-07-25T13:25:04Z

@radcortez please take a look. I opened it as draft since there are way more reflective access being caught by -H:ThrowMissingRegistrationErrors=

The ones I am trying to understand now are ones generated because of the sources automatically added to the RunTimeConfigCustomizer. Do you think we could/should skip some of those when targeting native-mode, or are they all necessary?

Since I have your attention I would also like to ask why you are not generating ServiceProviderBuildItems instead of ReflectiveClassBuildItems in

quarkus/core/deployment/src/main/java/io/quarkus/deployment/steps/ConfigGenerationBuildStep.java

Line 688 in fc678ed

reflectiveClass.produce(ReflectiveClassBuildItem.builder(service).build());

Thanks

radcortez · 2024-07-25T23:25:18Z

The ones I am trying to understand now are ones generated because of the sources automatically added to the RunTimeConfigCustomizer. Do you think we could/should skip some of those when targeting native-mode, or are they all necessary?

The idea is to discover all the services during the build and generate a config customizer that programmatically registers all the discovered services. I think we cannot skip any of the things we discover. Are you seeing issues with this?

Since I have your attention I would also like to ask why you are not generating ServiceProviderBuildItems instead of ReflectiveClassBuildItems in

We probably don't need the reflection registration anymore. We can try to drop it and check :)

radcortez · 2024-07-25T23:28:28Z

core/deployment/src/main/java/io/quarkus/deployment/steps/ConfigGenerationBuildStep.java

@@ -391,6 +392,17 @@ public void setupConfigOverride(

    @BuildStep
    public void watchConfigFiles(BuildProducer<HotDeploymentWatchedFileBuildItem> watchedFiles) {


The watched files include external files like .env and {user.dir}/config/application{-profile}.properties. Should these be registered as a resource with the native image?

Anything accessed through java.lang.ClassLoader.getResources (invoked by https://github.com/smallrye/smallrye-common/blob/c29bb1c00f484f5fa86f183e75c37af462ac6546/classloader/src/main/java/io/smallrye/common/classloader/ClassPathUtils.java#L84) needs to be registered (at least in theory).

External files should be calling processAsPath https://github.com/smallrye/smallrye-common/blob/c29bb1c00f484f5fa86f183e75c37af462ac6546/classloader/src/main/java/io/smallrye/common/classloader/ClassPathUtils.java#L132-L160 directly, without using the classloader:

For .env:
https://github.com/smallrye/smallrye-config/blob/64295f6d4350d3b0922c69c093f1f106ba035b24/implementation/src/main/java/io/smallrye/config/DotEnvConfigSourceProvider.java#L54-L57

For {user.dir}/config/application.properties:
https://github.com/smallrye/smallrye-config/blob/e8d87ce1cfbf1e6f259a798c85498267dd12b32b/implementation/src/main/java/io/smallrye/config/AbstractLocationConfigSourceLoader.java#L118-L135

At least until now, we have never had issues loading these files and the native executable.

Yes, these "issues" are popping up when using -H:+ThrowMissingRegistrationErrors (see #41995) which aims to help developers detect unregistered accesses to elements. What I suspect is that there are many places where Quarkus tries to access unregistered elements and just fails without a warning, but since there are other places or ways to get the same data back it works as expected.

So in many of these cases it might be OK to not register the elements, but at the same time the best practice would be to ensure hat Quarkus also doesn't try to access them at run-time (e.g. through substitutions or changing configuration).

radcortez · 2024-07-25T23:29:24Z

...fig-yaml/deployment/src/main/java/io/quarkus/config/yaml/deployment/ConfigYamlProcessor.java

        List<String> configWatchedFiles = new ArrayList<>();
        String userDir = System.getProperty("user.dir");

        // Main files
+        configWatchedFiles.add("META-INF/microprofile-config.yaml");
+        configWatchedFiles.add("META-INF/microprofile-config.yml");
        configWatchedFiles.add("application.yaml");
        configWatchedFiles.add("application.yml");
        configWatchedFiles.add(Paths.get(userDir, "config", "application.yaml").toAbsolutePath().toString());


Check previous comments about external files.

radcortez · 2024-07-25T23:31:40Z

...ntime/src/main/java/io/quarkus/runtime/configuration/FileSystemOnlySourcesConfigBuilder.java

+public class FileSystemOnlySourcesConfigBuilder implements ConfigBuilder {
+    @Override
+    public SmallRyeConfigBuilder configBuilder(final SmallRyeConfigBuilder builder) {
+        return builder.withSources(


You want to disable the default sources which include the classpath lookup and then add the inFileSystem: builder.setAddDefaultSources(false).addSystemSources().withSources(inFileSystem...)

radcortez · 2024-07-25T23:32:30Z

...ntime/src/main/java/io/quarkus/runtime/configuration/FileSystemOnlySourcesConfigBuilder.java

+    public SmallRyeConfigBuilder configBuilder(final SmallRyeConfigBuilder builder) {
+        return builder.withSources(
+                inFileSystem(Paths.get(System.getProperty("user.dir"), "config", "application.properties").toUri().toString(),
+                        260, Thread.currentThread().getContextClassLoader()));


Use the classloader set in the builder: io.smallrye.config.SmallRyeConfigBuilder#getClassLoader.

zakkak · 2024-07-26T09:39:59Z

The idea is to discover all the services during the build and generate a config customizer that programmatically registers all the discovered services. I think we cannot skip any of the things we discover. Are you seeing issues with this?

Not necessarily, it kind of complicates the resource registration process so I am just trying to see if there is any opportunity for trimming down :)

zakkak · 2024-07-26T09:53:36Z

We probably don't need the reflection registration anymore. We can try to drop it and check :)

It looks like you are right.

`SmallryeConfigBuilder` tries to access properties files from various sources, despite them not being available at run-time nor supported in native-mode, see quarkusio#41994 This patch also avoids getting a `MissingRegistrationError` when using `-H:+ThrowMissingRegistrationErrors` or `--exact-reachability-metadata`. Related to quarkusio#41994 and quarkusio#41995

radcortez · 2024-07-26T11:19:43Z

Not necessarily, it kind of complicates the resource registration process so I am just trying to see if there is any opportunity for trimming down :)

Hum, the goal was to avoid as many registrations as possible. Since this has been an ongoing process, there are probably some code leftovers that are not required anymore (the reflection registration of services, for instance).

Let me know what you are seeing and we try to hax a few things.

radcortez · 2024-07-26T11:22:41Z

I think we can drop registrations of ServiceProvider in

quarkus/core/deployment/src/main/java/io/quarkus/deployment/steps/ConfigBuildSteps.java

Lines 31 to 51 in bf77189

    
           // XXX replace this with constant-folded service loader impl 
        
           @BuildStep 
        
           void nativeServiceProviders( 
        
                   final BuildProducer<ServiceProviderBuildItem> providerProducer) throws IOException { 
        
               providerProducer.produce(new ServiceProviderBuildItem(ConfigProviderResolver.class.getName(), 
        
                       SmallRyeConfigProviderResolver.class.getName())); 
        
               final ClassLoader classLoader = Thread.currentThread().getContextClassLoader(); 
        
               for (Class<?> serviceClass : Arrays.asList( 
        
                       Converter.class, 
        
                       ConfigSourceInterceptor.class, 
        
                       ConfigSourceInterceptorFactory.class, 
        
                       SecretKeysHandler.class, 
        
                       SecretKeysHandlerFactory.class, 
        
                       ConfigValidator.class)) { 
        
                   final String serviceName = serviceClass.getName(); 
        
                   final Set<String> names = ServiceUtil.classNamesNamedIn(classLoader, SERVICES_PREFIX + serviceName); 
        
                   if (!names.isEmpty()) { 
        
                       providerProducer.produce(new ServiceProviderBuildItem(serviceName, names)); 
        
                   } 
        
               } 
        
           }

for everything except ConfigValidator which we don't generate yet.

zakkak · 2024-07-26T12:22:07Z

Let me know what you are seeing and we try to hax a few things.

If you run:

./mvnw -Dnative -pl integration-tests/smallrye-config/ -Dnative.surefire.skip -Dformat.skip -Dno-descriptor-tests clean verify -Dquarkus.native.builder-image=quay.io/quarkus/ubi-quarkus-mandrel-builder-image:jdk-22 -Dquarkus.native.additional-build-args=-H:ThrowMissingRegistrationErrors=,-H:MissingRegistrationReportingMode=Warn

you will see a bunch of exceptions being thrown, indicating that the resulting native-image is trying to access elements that have not been registered at build time. Unfortunately the Warn mode limits the trace to just 4 lines, so you can't get enough details from it. FYI I am attaching the output of the above command when using a modified Mandrel build without the 4 lines limit.

exceptions.log

A couple of indicative examples:

com.oracle.svm.core.jdk.resources.MissingResourceRegistrationError: The program tried to access the resource at path 

   META-INF/microprofile-config.yaml

 without it being registered as reachable. Add it to the resource metadata to solve this problem. See https://www.graalvm.org/latest/reference-manual/native-image/metadata/#resources-and-resource-bundles for help
  java.base@22.0.1-beta/java.lang.ClassLoader.getResources(ClassLoader.java:99)
  io.smallrye.common.classloader.ClassPathUtils.consumeAsPaths(ClassPathUtils.java:84)
  io.smallrye.config.AbstractLocationConfigSourceLoader.tryClassPath(AbstractLocationConfigSourceLoader.java:141)
  io.smallrye.config.AbstractLocationConfigSourceLoader.loadConfigSources(AbstractLocationConfigSourceLoader.java:104)
  io.smallrye.config.AbstractLocationConfigSourceLoader.loadConfigSources(AbstractLocationConfigSourceLoader.java:87)
  io.smallrye.config.source.yaml.YamlConfigSourceLoader$InClassPath.getConfigSources(YamlConfigSourceLoader.java:37)
  io.smallrye.config.source.yaml.YamlConfigSourceLoader$InClassPath.getConfigSources(YamlConfigSourceLoader.java:31)
  io.smallrye.config.SmallRyeConfig$ConfigSources.buildSources(SmallRyeConfig.java:832)
  io.smallrye.config.SmallRyeConfig$ConfigSources.<init>(SmallRyeConfig.java:770)
  io.smallrye.config.SmallRyeConfig.<init>(SmallRyeConfig.java:85)
  io.smallrye.config.SmallRyeConfigBuilder.build(SmallRyeConfigBuilder.java:736)
  io.quarkus.runtime.generated.Config.readConfig(Unknown Source)
  io.quarkus.runtime.generated.Config.createRunTimeConfig(Unknown Source)
  io.quarkus.deployment.steps.RuntimeConfigSetup.deploy(Unknown Source)
  io.quarkus.runner.ApplicationImpl.doStart(Unknown Source)
  io.quarkus.runtime.Application.start(Application.java:101)
  io.quarkus.runtime.ApplicationLifecycleManager.run(ApplicationLifecycleManager.java:111)
  io.quarkus.runtime.Quarkus.run(Quarkus.java:71)
  io.quarkus.runtime.Quarkus.run(Quarkus.java:44)
  io.quarkus.runtime.Quarkus.run(Quarkus.java:124)
  io.quarkus.runner.GeneratedMain.main(Unknown Source)
com.oracle.svm.core.jdk.resources.MissingResourceRegistrationError: The program tried to access the resource at path

org.graalvm.nativeimage.MissingReflectionRegistrationError: The program tried to reflectively access

   io.quarkus.datasource.runtime.DataSourcesRuntimeConfig.getDeclaredMethods()

 without it being registered for runtime reflection. Add io.quarkus.datasource.runtime.DataSourcesRuntimeConfig.getDeclaredMethods() to the reflection metadata to solve this problem. See https://www.graalvm.org/latest/reference-manual/native-image/metadata/#reflection for help.
  java.base@22.0.1-beta/java.lang.Class.getDeclaredMethods(DynamicHub.java:1165)
  io.smallrye.config.ConfigMappingInterface.createConfigurationInterface(ConfigMappingInterface.java:757)
  io.smallrye.config.ConfigMappingInterface$1.computeValue(ConfigMappingInterface.java:37)
  io.smallrye.config.ConfigMappingInterface$1.computeValue(ConfigMappingInterface.java:35)
  java.base@22.0.1-beta/java.lang.ClassValue.get(JavaLangSubstitutions.java:531)
  io.smallrye.config.ConfigMappingInterface.getConfigurationInterface(ConfigMappingInterface.java:52)
  io.smallrye.config.validator.BeanValidationConfigValidator.validateMapping(BeanValidationConfigValidator.java:38)
  io.smallrye.config.SmallRyeConfig.getConfigMapping(SmallRyeConfig.java:604)
  io.quarkus.deployment.steps.DataSourcesExcludedFromHealthChecksProcessor$produceBean807665441.deploy_0(Unknown Source)
  io.quarkus.deployment.steps.DataSourcesExcludedFromHealthChecksProcessor$produceBean807665441.deploy(Unknown Source)
  io.quarkus.runner.ApplicationImpl.doStart(Unknown Source)
  io.quarkus.runtime.Application.start(Application.java:101)
  io.quarkus.runtime.ApplicationLifecycleManager.run(ApplicationLifecycleManager.java:111)
  io.quarkus.runtime.Quarkus.run(Quarkus.java:71)
  io.quarkus.runtime.Quarkus.run(Quarkus.java:44)
  io.quarkus.runtime.Quarkus.run(Quarkus.java:124)
  io.quarkus.runner.GeneratedMain.main(Unknown Source)
org.graalvm.nativeimage.MissingReflectionRegistrationError: The program tried to reflectively access

radcortez · 2024-07-26T13:23:18Z

If you run:
./mvnw -Dnative -pl integration-tests/smallrye-config/ -Dnative.surefire.skip -Dformat.skip -Dno-descriptor-tests clean verify -Dquarkus.native.builder-image=quay.io/quarkus/ubi-quarkus-mandrel-builder-image:jdk-22 -Dquarkus.native.additional-build-args=-H:ThrowMissingRegistrationErrors=,-H:MissingRegistrationReportingMode=Warn
you will see a bunch of exceptions being thrown, indicating that the resulting native-image is trying to access elements that have not been registered at build time. Unfortunately the Warn mode limits the trace to just 4 lines, so you can't get enough details from it. FYI I am attaching the output of the above command when using a modified Mandrel build without the 4 lines limit.

Great. Thanks.

Ok, let's try to trim these down. Did you run with these PR changes? It shouldn't be looking for *.properties files in the classpath.

zakkak · 2024-07-29T07:57:26Z

Did you run with these PR changes?

Yes, it removes some of these warnings but not all.

radcortez · 2024-07-29T10:55:46Z

Yes, it removes some of these warnings but not all.

Yes, some of the warnings related to the lookups to the config files should be gone now.

Looking into other warnings, I see many related to the Validator. The issue here is that we only register for reflection the config classes that have validation annotations. The config code still calls getDeclaredMethods to no effect because there are no properties to validate, so it doesn't really fail.

There is no quick fix for this. I guess the best option is to override the Validator and pass in a list of config classes that require validation to avoid that call.

quarkus-bot bot added area/config area/core labels Jul 25, 2024

radcortez requested changes Jul 25, 2024

View reviewed changes

zakkak force-pushed the 2024-07-24-smallryeconfigsources branch from fc678ed to 628ee63 Compare July 26, 2024 09:54

zakkak mentioned this pull request Jul 31, 2024

Extend reflection configuration for SmallRye FT #42229

Closed

linghengqian mentioned this pull request Oct 10, 2024

Enable Safe Composition of Metadata for ShardingSphere Proxy Native apache/shardingsphere#33179

Merged

6 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Limit external configuration sources parsing in native-mode #42140

Limit external configuration sources parsing in native-mode #42140

zakkak commented Jul 25, 2024

zakkak commented Jul 25, 2024

radcortez commented Jul 25, 2024

radcortez Jul 25, 2024 •

edited

Loading

zakkak Jul 26, 2024

radcortez Jul 26, 2024

zakkak Jul 26, 2024

radcortez Jul 25, 2024

radcortez Jul 25, 2024

radcortez Jul 25, 2024

zakkak commented Jul 26, 2024

zakkak commented Jul 26, 2024

radcortez commented Jul 26, 2024

radcortez commented Jul 26, 2024

zakkak commented Jul 26, 2024

radcortez commented Jul 26, 2024

zakkak commented Jul 29, 2024 •

edited

Loading

radcortez commented Jul 29, 2024

		@@ -391,6 +392,17 @@ public void setupConfigOverride(

		@BuildStep
		public void watchConfigFiles(BuildProducer<HotDeploymentWatchedFileBuildItem> watchedFiles) {

Limit external configuration sources parsing in native-mode #42140

Are you sure you want to change the base?

Limit external configuration sources parsing in native-mode #42140

Conversation

zakkak commented Jul 25, 2024

zakkak commented Jul 25, 2024

radcortez commented Jul 25, 2024

radcortez Jul 25, 2024 • edited Loading

Choose a reason for hiding this comment

zakkak Jul 26, 2024

Choose a reason for hiding this comment

radcortez Jul 26, 2024

Choose a reason for hiding this comment

zakkak Jul 26, 2024

Choose a reason for hiding this comment

radcortez Jul 25, 2024

Choose a reason for hiding this comment

radcortez Jul 25, 2024

Choose a reason for hiding this comment

radcortez Jul 25, 2024

Choose a reason for hiding this comment

zakkak commented Jul 26, 2024

zakkak commented Jul 26, 2024

radcortez commented Jul 26, 2024

radcortez commented Jul 26, 2024

zakkak commented Jul 26, 2024

radcortez commented Jul 26, 2024

zakkak commented Jul 29, 2024 • edited Loading

radcortez commented Jul 29, 2024

radcortez Jul 25, 2024 •

edited

Loading

zakkak commented Jul 29, 2024 •

edited

Loading