Describe the bug

The smallrye openapi (Swagger?) UI shows fields that aren't part of the RestEasy output. It seems like the Jackson ObjectMapperCustomizer only applies to RestEasy, not Swagger.

For example, OptaPlanner has a HardSoftScore class with 3 fields (initScore, hardScore and softScore) and helper methods like isFeasible() which returns (hardScore >= 0).
The REST API renders HardSoftScore as a String in Json, but the Swagger API description shows it as an object class with feasible etc:

It seems like RestEasty takes this into account, but Swagger does not:

    @BuildStep
    void registerOptaPlannerJacksonModule(BuildProducer<AdditionalBeanBuildItem> additionalBeans) {
        additionalBeans.produce(new AdditionalBeanBuildItem(OptaPlannerObjectMapperCustomizer.class));
    }

https://github.com/kiegroup/optaplanner/blob/main/optaplanner-quarkus-integration/optaplanner-quarkus-jackson/deployment/src/main/java/org/optaplanner/quarkus/jackson/deployment/OptaPlannerJacksonProcessor.java

https://github.com/kiegroup/optaplanner/blob/main/optaplanner-quarkus-integration/optaplanner-quarkus-jackson/runtime/src/main/java/org/optaplanner/quarkus/jackson/OptaPlannerObjectMapperCustomizer.java

Expected behavior

REST API:

myClass {
  score : "0hard/5soft"
}

Swagger API:

myClass {
  score : String
}

Actual behavior

REST API:

myClass {
  score : "0hard/5soft"
}

Swagger API:

myClass {
  score : HardSoftScore
}
HardSoftScore {
 initScore: number
 hardScore: number
 softScore: number
 feasible: boolean
 ...
}

How to Reproduce?

1. Add these dependencies:

   org.optaplanner optaplanner org.optaplanner optaplanner-quarkus-jackson io.quarkus quarkus-smallrye-openapi

2. Create a REST method that returns MyClass with a field of type HardSoftScore.
   Jackson will customize the json marshalling with OptaPlannerObjectMapperCustomizer.

3. Check the Swagger API screen, notice that there is a HardSoftScore model that includes "feasible", so OptaPlannerObjectMapperCustomizer is not active.

Output of uname -a or ver

No response

Output of java -version

No response

GraalVM version (if different from Java)

No response

Quarkus version or git rev

No response

Build tool (ie. output of mvnw --version or gradlew --version)

No response

Additional information

No response