ArrayIndexOutOfBoundsException in DoubleHistogram

[shelliglick]

Using micrometer + prometheus + reactor integration.

Getting this exception:

java.lang.ArrayIndexOutOfBoundsException: value outside of histogram covered range. Caused by: java.lang.ArrayIndexOutOfBoundsException: 64 at org.HdrHistogram.AbstractHistogram.handleRecordException(AbstractHistogram.java:428) at org.HdrHistogram.AbstractHistogram.recordCountAtValue(AbstractHistogram.java:405) at org.HdrHistogram.AbstractHistogram.recordValueWithCount(AbstractHistogram.java:344) at org.HdrHistogram.DoubleHistogram.recordCountAtValue(DoubleHistogram.java:317) at org.HdrHistogram.DoubleHistogram.recordValueWithCount(DoubleHistogram.java:280) at org.HdrHistogram.DoubleHistogram.add(DoubleHistogram.java:641) at org.HdrHistogram.DoubleHistogram.copyInto(DoubleHistogram.java:598) at org.HdrHistogram.DoubleRecorder.getIntervalHistogramInto(DoubleRecorder.java:176) at io.micrometer.core.instrument.distribution.TimeWindowPercentileHistogram.accumulate(TimeWindowPercentileHistogram.java:69) at io.micrometer.core.instrument.distribution.AbstractTimeWindowHistogram.accumulateIfStale(AbstractTimeWindowHistogram.java:173) at io.micrometer.core.instrument.distribution.AbstractTimeWindowHistogram.takeSnapshot(AbstractTimeWindowHistogram.java:163) at io.micrometer.core.instrument.AbstractTimer.takeSnapshot(AbstractTimer.java:188) at io.micrometer.prometheus.PrometheusTimer.takeSnapshot(PrometheusTimer.java:99) at io.micrometer.prometheus.PrometheusMeterRegistry.lambda$newTimer$2(PrometheusMeterRegistry.java:167) at io.micrometer.prometheus.MicrometerCollector.collect(MicrometerCollector.java:92) at io.prometheus.client.CollectorRegistry$MetricFamilySamplesEnumeration.findNextElement(CollectorRegistry.java:183) at io.prometheus.client.CollectorRegistry$MetricFamilySamplesEnumeration.nextElement(CollectorRegistry.java:216) at io.prometheus.client.CollectorRegistry$MetricFamilySamplesEnumeration.nextElement(CollectorRegistry.java:137) at io.prometheus.client.exporter.common.TextFormat.write004(TextFormat.java:22) at io.micrometer.spring.autoconfigure.export.prometheus.PrometheusScrapeEndpoint.invoke(PrometheusScrapeEndpoint.java:50) at io.micrometer.spring.autoconfigure.export.prometheus.PrometheusScrapeEndpoint.invoke(PrometheusScrapeEndpoint.java:36) at org.springframework.boot.actuate.endpoint.mvc.AbstractEndpointMvcAdapter.invoke(AbstractEndpointMvcAdapter.java:56) at org.springframework.boot.actuate.endpoint.mvc.EndpointMvcAdapter.invoke(EndpointMvcAdapter.java:42) at io.micrometer.spring.autoconfigure.export.prometheus.PrometheusScrapeMvcEndpoint.invoke(PrometheusScrapeMvcEndpoint.java:30) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205) at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:133) at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:97) at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:854) at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:765) at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:85) at org.springframework.boot.actuate.autoconfigure.EndpointWebMvcChildContextConfiguration$CompositeHandlerAdapter.handle(EndpointWebMvcChildContextConfiguration.java:288) at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:967) at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:901) at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:970) at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:861) at javax.servlet.http.HttpServlet.service(HttpServlet.java:687) at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:846) at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:197) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:198) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:496) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:140) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:81) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:342) at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:803) at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66) at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:790) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1459) at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.lang.Thread.run(Thread.java:748)

Using
reactor-core 3.2.10.RELEASE
micrometer-registry-prometheus 1.0.11
micrometer-spring-legacy 1.0.11
micrometer-core 1.0.11

Code snippet for adding histograms:

    @Bean
    public MeterFilter meterFilter() {
        return new MeterFilter() {
            @Override
            public DistributionStatisticConfig configure(Meter.Id id, DistributionStatisticConfig config) {
                if (id.getName().equals("reactor.flow.duration")) {
                    return DistributionStatisticConfig.builder()
                            .percentiles(0.5, 0.75, 0.95, 0.99)
                            .build()
                            .merge(config);
                }
                return config;
            }
        };
    }

(Tried also setting maximumExpectedValue and minimumExpectedValue explicitly, didn't help.)

I see 2 issues here :

1. The DoubleHistogram.autoResize is set to true, while it's integerValuesHistogram is not.
2. We would like to have the option to preconfigure the size of the allocated array (manually setting the highestToLowestValueRatio passed to the DoubleHistogram in it's ctor, according to the DistributionStatisticConfig.maximumExpectedValue)

[shelliglick]

opened a PR here #1808

[jkschneider]

@shelliglick Since the default minimum/maximum expected values yield a highestToLowestValueRatio that HdrHistogram does not accept, I'm not sure what the right thing to do here is.

What values did your histogram see that caused the ArrayIndexOutOfBoundsException?

[marcingrzejszczak]

@shelliglick can you please help us replicate this problem?