ServiceConfigurationError when calling async functions on Hazelcast IMap

### Describe the bug

We currently have a Quarkus application that uses RestEasy Reactive and integrates with Hazelcast for caching. The caching works initially. However, after a period of time (5-10 minutes) when the code is accessed, a SmallRye exception is thrown. Once thrown, the code path will always fail with the same exception. I've read up on [Context Propagation](https://quarkus.io/guides/context-propagation) and review [other tickets](https://github.com/quarkusio/quarkus/issues/24896), but can't seem to figure out why the correct classloader is not being used. We would appreciate any insight into this that you can provide.

### Expected behavior

I would expect the initial behavior of not throwing an exception to always be the case.

### Actual behavior

An exception is thrown:
```
ERROR [or.jb.re.re.co.co.AbstractResteasyReactiveContext] (vert.x-eventloop-thread-1) Request failed: java.util.ServiceConfigurationError: io.smallrye.config.SmallRyeConfigFactory: io.quarkus.runtime.configuration.QuarkusConfigFactory not a subtype
	at java.base/java.util.ServiceLoader.fail(ServiceLoader.java:593)
	at java.base/java.util.ServiceLoader$LazyClassPathLookupIterator.hasNextService(ServiceLoader.java:1244)
	at java.base/java.util.ServiceLoader$LazyClassPathLookupIterator.hasNext(ServiceLoader.java:1273)
	at java.base/java.util.ServiceLoader$2.hasNext(ServiceLoader.java:1309)
	at java.base/java.util.ServiceLoader$3.hasNext(ServiceLoader.java:1393)
	at io.smallrye.config.SmallRyeConfigProviderResolver.getFactoryFor(SmallRyeConfigProviderResolver.java:100)
	at io.smallrye.config.SmallRyeConfigProviderResolver.getConfig(SmallRyeConfigProviderResolver.java:76)
	at io.smallrye.config.SmallRyeConfigProviderResolver.getConfig(SmallRyeConfigProviderResolver.java:64)
	at org.eclipse.microprofile.config.ConfigProvider.getConfig(ConfigProvider.java:85)
	at io.smallrye.context.impl.DefaultValues.<init>(DefaultValues.java:41)
	at io.smallrye.context.SmallRyeContextManager.<init>(SmallRyeContextManager.java:63)
	at io.smallrye.context.SmallRyeContextManager$Builder.build(SmallRyeContextManager.java:337)
	at io.smallrye.context.SmallRyeContextManagerProvider.getContextManager(SmallRyeContextManagerProvider.java:48)
	at io.smallrye.context.SmallRyeContextManagerProvider.getContextManager(SmallRyeContextManagerProvider.java:37)
	at io.smallrye.context.SmallRyeContextManagerProvider.getManager(SmallRyeContextManagerProvider.java:97)
	at io.smallrye.context.SmallRyeThreadContext.getCurrentThreadContextOrDefaultContexts(SmallRyeThreadContext.java:160)
	at io.smallrye.mutiny.context.DefaultContextPropagationInterceptor.getThreadContext(DefaultContextPropagationInterceptor.java:12)
	at io.smallrye.mutiny.context.BaseContextPropagationInterceptor.decorate(BaseContextPropagationInterceptor.java:24)
	at io.smallrye.mutiny.infrastructure.Infrastructure.decorate(Infrastructure.java:145)
	at io.smallrye.mutiny.groups.UniCreate.completionStage(UniCreate.java:142)
	at io.smallrye.mutiny.groups.UniCreate.completionStage(UniCreate.java:78)
	at com.floify.cache.Cache.put(Cache.java:69)
	at com.floify.cache.Cache.lambda$computeIfAbsent$0(Cache.java:95)
	at io.smallrye.context.impl.wrappers.SlowContextualConsumer.accept(SlowContextualConsumer.java:21)
	at io.smallrye.mutiny.groups.UniOnNotNull.lambda$invoke$0(UniOnNotNull.java:40)
	at io.smallrye.context.impl.wrappers.SlowContextualConsumer.accept(SlowContextualConsumer.java:21)
	at io.smallrye.mutiny.operators.uni.UniOnItemConsume$UniOnItemComsumeProcessor.invokeEventHandler(UniOnItemConsume.java:77)
	at io.smallrye.mutiny.operators.uni.UniOnItemConsume$UniOnItemComsumeProcessor.onItem(UniOnItemConsume.java:42)
	at io.smallrye.mutiny.operators.uni.UniOnItemTransformToUni$UniOnItemTransformToUniProcessor.onItem(UniOnItemTransformToUni.java:60)
	at io.smallrye.mutiny.operators.uni.builders.UniCreateFromKnownItem$KnownItemSubscription.forward(UniCreateFromKnownItem.java:38)
	at io.smallrye.mutiny.operators.uni.builders.UniCreateFromKnownItem$KnownItemSubscription.access$100(UniCreateFromKnownItem.java:26)
	at io.smallrye.mutiny.operators.uni.builders.UniCreateFromKnownItem.subscribe(UniCreateFromKnownItem.java:23)
	at io.smallrye.mutiny.operators.AbstractUni.subscribe(AbstractUni.java:36)
	at io.smallrye.mutiny.operators.uni.UniOnItemTransformToUni$UniOnItemTransformToUniProcessor.performInnerSubscription(UniOnItemTransformToUni.java:81)
	at io.smallrye.mutiny.operators.uni.UniOnItemTransformToUni$UniOnItemTransformToUniProcessor.onItem(UniOnItemTransformToUni.java:57)
	at io.smallrye.mutiny.operators.uni.builders.UniCreateFromCompletionStage$CompletionStageUniSubscription.forwardResult(UniCreateFromCompletionStage.java:63)
	at io.smallrye.context.impl.wrappers.SlowContextualBiConsumer.accept(SlowContextualBiConsumer.java:21)
	at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863)
	at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841)
	at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483)
	at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373)
	at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182)
	at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655)
	at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622)
	at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165)
```

### How to Reproduce?

Here is some of the code:

Hazelcast Producer:
```java
public class HazelcastClientService {

  @Produces
  @ApplicationScoped
  @IfBuildProperty(name = "clustering.aws.enabled", stringValue = "false", enableIfMissing = true)
  HazelcastInstance getDefaultHazelcastClient(final ClusteringConfig config) {
    return HazelcastClient.newHazelcastClient(config.toClientConfig());
  }

  @Produces
  @ApplicationScoped
  @IfBuildProperty(name = "clustering.aws.enabled", stringValue = "true")
  HazelcastInstance getAwsHazelcastClient(
      final ClusteringConfig config, final AwsCredentials credentials) {
    return HazelcastClient.newHazelcastClient(config.toClientConfig(credentials));
  }
}
```

Cache manager:
```java
@ApplicationScoped
public class Caches {
  @Inject HazelcastInstance hazelcast;
  @Inject
  ThreadContext threadContext;
  @Inject
  ManagedExecutor managedExecutor;
  private final ConcurrentMap<String, Cache> backingCaches = new ConcurrentHashMap<>();

  public Cache<User> getUserByEmailCache() {
    return getCache("userByEmailCache", User.class);
  }

  public Cache<Team> getTeamByDomainCache() {
    return getCache("teamByDomainCache", Team.class);
  }

  public Cache<UserState> getUserStateByEmailCache() {
    return getCache("userStateByEmailCache", UserState.class);
  }

  @SuppressWarnings("unchecked")
  private <T> Cache<T> getCache(final String key, final Class<T> type) {
    Cache<T> cache = backingCaches.get(key);

    if (cache == null) {
      cache = new Cache<>(threadContext, managedExecutor, hazelcast.getMap(key), type);

      backingCaches.put(key, cache);
    }

    return cache;
  }
}
```

Cache:
```java
@Slf4j
public class Cache<T> {
  protected final IMap<String, String> backingCache;
  @Getter protected final ObjectMapper mapper;
  @Getter protected final Class<T> type;
  private final ThreadContext threadContext;
  private final ManagedExecutor managedExecutor;

  public Cache(final IMap<String, String> cache, final Class<T> type) {
    this.mapper = new ObjectMapper();
    this.mapper.findAndRegisterModules();
    this.mapper.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false);

    this.backingCache = cache;
    this.type = type;
    this.threadContext = null;
    this.managedExecutor = null;
  }

  public Cache(ThreadContext threadContext,
      ManagedExecutor managedExecutor, IMap<String, String> cache, Class<T> type) {
    this.mapper = new ObjectMapper();
    this.mapper.findAndRegisterModules();
    this.mapper.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false);

    this.backingCache = cache;
    this.type = type;
    this.threadContext = threadContext;
    this.managedExecutor = managedExecutor;
  }

  public Uni<T> get(final String key) {
    if (StringUtils.isBlank(key)) {
      return Uni.createFrom().nullItem();
    }

    CompletionStage<T> get = threadContext
        .withContextCapture(backingCache.getAsync(key))
        .thenApplyAsync(this::deserialize, managedExecutor);

    return Uni.createFrom().completionStage(get);
  }

  public Uni<T> put(final String key, final T value) {
    if (StringUtils.isBlank(key) || value == null) {
      return Uni.createFrom().nullItem();
    }

    try {
      CompletionStage<T> put = threadContext
          .withContextCapture(backingCache.putAsync(key, mapper.writeValueAsString(value)))
          .thenApplyAsync(this::deserialize, managedExecutor);

      return Uni.createFrom().completionStage(put);
    } catch (JsonProcessingException exception) {
      log.error("Could not serialize object for key {}", key, exception);

      return Uni.createFrom().failure(exception);
    }
  }

  public Uni<T> delete(final String key) {
    if (StringUtils.isBlank(key)) {
      return Uni.createFrom().nullItem();
    }

    CompletionStage<T> delete = threadContext
        .withContextCapture(backingCache.removeAsync(key))
        .thenApplyAsync(this::deserialize, managedExecutor);

    return Uni.createFrom().completionStage(delete);
  }

  public Uni<T> computeIfAbsent(final String key, final Supplier<Uni<T>> computer) {
    // spotless:off
    return get(key)
        .onItem()
        .ifNull().switchTo(computer::get)
        .onItem()
        .ifNotNull().invoke(value -> put(key, value));
    // spotless:on
  }

  private T deserialize(final String json) {
    try {
      if (StringUtils.isBlank(json)) {
        return null;
      }

      return mapper.readValue(json, type);
    } catch (JsonProcessingException exception) {
      log.error("Could not deserialize cached json {}", json, exception);

      return null;
    }
  }
}
```

### Output of `uname -a` or `ver`

Darwin MacBook-Pro.local 21.6.0 Darwin Kernel Version 21.6.0: Wed Aug 10 14:25:27 PDT 2022; root:xnu-8020.141.5~2/RELEASE_X86_64 x86_64

### Output of `java -version`

openjdk version "17" 2021-09-14 OpenJDK Runtime Environment (build 17+35-2724) OpenJDK 64-Bit Server VM (build 17+35-2724, mixed mode, sharing)

### GraalVM version (if different from Java)

_No response_

### Quarkus version or git rev

2.14.2.Final

### Build tool (ie. output of `mvnw --version` or `gradlew --version`)

Gradle 7.4.2

### Additional information

_No response_

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ServiceConfigurationError when calling async functions on Hazelcast IMap #30182

Describe the bug

Expected behavior

Actual behavior

How to Reproduce?

Output of `uname -a` or `ver`

Output of `java -version`

GraalVM version (if different from Java)

Quarkus version or git rev

Build tool (ie. output of `mvnw --version` or `gradlew --version`)

Additional information

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development