Slow(ish) read/deserialization performance

[sheinbergon]

Hello

I've create a JMH benchmark to test Nitrite random read performance.
I insert a varying amount of entities into an in-memory only nitrite ObjectRepository and then query it using an indexed field with a varying degree of dispersion/randomness.

All entities are of a single types that implements Mappable

While searching the database using the index takes a few microseconds, which is great, deserializing the result set (a few dozens of entities) takes a few milliseconds. which is 3 orders of magnitude slower.

Is it possible to improve upon these numbers with nitrite?

Results (numbers are microseconds per operation (us/op) ):

Number of entities	Also read data?	Indexed field disperssion	Iterations	Results
100000	true	5000	12	8839.989
100000	true	10000	12	3845.652
100000	true	20000	12	2009.990
100000	false	5000	12	3.109
100000	false	10000	12	3.686
100000	false	20000	12	3.823
200000	true	5000	12	13159.064
200000	true	10000	12	6740.482
200000	true	20000	12	3755.540
200000	false	5000	12	3.219
200000	false	10000	12	3.181
200000	false	20000	12	3.467

Here's the benchmark code (uses JMH and Lombok annotations):

package org.sheinbergon;


import lombok.Getter;
import lombok.Setter;
import lombok.experimental.Accessors;
import lombok.val;
import org.apache.commons.lang3.RandomStringUtils;
import org.apache.commons.lang3.RandomUtils;
import org.dizitart.no2.Document;
import org.dizitart.no2.IndexOptions;
import org.dizitart.no2.IndexType;
import org.dizitart.no2.Nitrite;
import org.dizitart.no2.mapper.Mappable;
import org.dizitart.no2.mapper.NitriteMapper;
import org.dizitart.no2.objects.Id;
import org.dizitart.no2.objects.ObjectRepository;
import org.dizitart.no2.objects.filters.ObjectFilters;
import org.openjdk.jmh.annotations.*;

import java.math.BigDecimal;
import java.util.List;
import java.util.concurrent.TimeUnit;
import java.util.stream.Collectors;
import java.util.stream.IntStream;

@BenchmarkMode(Mode.AverageTime)
@OutputTimeUnit(TimeUnit.MICROSECONDS)
@State(Scope.Benchmark)
public class NitriteBenchmark {

    private final static int FORKS = 2;
    private final static int WARMUPS = 2;
    private final static int ITERATIONS = 6;
    private final static int MILLISECONDS = 1000;

    private final static int RANDOM_STRING_MIN_LENGTH = 100;
    private final static int RANDOM_STRING_MAX_LENGTH = 200;

    private final static long INDEXED_INTEGER_FIELD_LOWER_BOUND = 0L;

    @Getter
    @Setter
    @Accessors(fluent = true, chain = true)
    private static class NMappable implements Mappable {

        @Getter
        @Setter
        @Accessors(fluent = true, chain = true)
        private static class IMappable implements Mappable {
            private BigDecimal innerNumber1;
            private BigDecimal innerNumber2;
            private BigDecimal innerNumber3;

            @Override
            public Document write(NitriteMapper mapper) {
                val d = new Document();
                d.put("innerNumber1", innerNumber1);
                d.put("innerNumber2", innerNumber2);
                d.put("innerNumber3", innerNumber3);
                return d;
            }

            @Override
            public void read(NitriteMapper mapper, Document document) {
                innerNumber1 = document.get("innerNumber1", BigDecimal.class);
                innerNumber2 = document.get("innerNumber2", BigDecimal.class);
                innerNumber3 = document.get("innerNumber3", BigDecimal.class);
            }
        }

        @Id
        private String text1;
        private String text2;
        private String text3;
        private BigDecimal decimal1;
        private Long integer1;
        private Boolean flag1;
        private IMappable inner;

        @Override
        public Document write(NitriteMapper mapper) {
            val d = new Document();
            d.put("text1", text1);
            d.put("text2", text2);
            d.put("text3", text3);
            d.put("decimal1", decimal1);
            d.put("integer1", integer1);
            d.put("flag1", flag1);
            d.put("inner", mapper.asDocument(inner));
            return d;
        }

        @Override
        public void read(NitriteMapper mapper, Document document) {
            text1 = document.get("text1", String.class);
            text2 = document.get("text2", String.class);
            text3 = document.get("text3", String.class);
            decimal1 = document.get("decimal1", BigDecimal.class);
            integer1 = document.get("integer1", Long.class);
            flag1 = document.get("flag1", Boolean.class);
            inner = mapper.asObject(document.get("inner", Document.class), IMappable.class);
        }
    }

    private Nitrite nitrite;
    private ObjectRepository<NMappable> repository;

    private List<NMappable> entities;

    @Param({"100000", "200000"})
    private int entityCount;

    @Param({"5000", "10000", "20000"})
    private int indexFieldDispersion;

    @Param({"true", "false"})
    private boolean fetchData;

    @Setup
    public void setup() {
        nitrite = Nitrite.builder().openOrCreate();
        repository = nitrite.getRepository(NMappable.class);
        entities = IntStream.range(0, entityCount)
                .mapToObj(index -> randomEntity())
                .collect(Collectors.toList());
        repository.createIndex("integer1", IndexOptions.indexOptions(IndexType.NonUnique));
        repository.insert(entities.toArray(NMappable[]::new));
    }

    @TearDown
    public void teardown() {
        repository.close();
    }

    @Benchmark
    @Fork(value = FORKS, jvmArgsAppend = {
            "--add-exports=java.base/jdk.internal.ref=ALL-UNNAMED",
            "-Xmx4096m",
            "-Xms2048m"})
    @Warmup(iterations = WARMUPS, timeUnit = TimeUnit.MILLISECONDS, time = MILLISECONDS)
    @Measurement(iterations = ITERATIONS, timeUnit = TimeUnit.MILLISECONDS, time = MILLISECONDS)
    public void benchmarkQueries() {
        val entity = entities.get(RandomUtils.nextInt(0, entityCount));
        val result = repository.find(ObjectFilters.eq("integer1", entity.integer1));
        if (fetchData) {
            result.forEach(m -> {
            });
        }
    }

    private NMappable randomEntity() {
        return new NMappable()
                .text1(RandomStringUtils.randomAlphanumeric(RANDOM_STRING_MIN_LENGTH, RANDOM_STRING_MAX_LENGTH))
                .text2(RandomStringUtils.randomAlphanumeric(RANDOM_STRING_MIN_LENGTH, RANDOM_STRING_MAX_LENGTH))
                .text3(RandomStringUtils.randomAlphanumeric(RANDOM_STRING_MIN_LENGTH, RANDOM_STRING_MAX_LENGTH))
                .decimal1(BigDecimal.valueOf(RandomUtils.nextDouble()))
                .integer1(RandomUtils.nextLong(INDEXED_INTEGER_FIELD_LOWER_BOUND, indexFieldDispersion))
                .flag1(RandomUtils.nextBoolean())
                .inner(new NMappable.IMappable()
                        .innerNumber1(BigDecimal.valueOf(RandomUtils.nextLong()))
                        .innerNumber2(BigDecimal.valueOf(RandomUtils.nextDouble()))
                        .innerNumber3(BigDecimal.valueOf(RandomUtils.nextDouble())));
    }

}

[anidotnet]

Excellent work and thank your taking your time to investigate this. Honestly, I have not paid attention to the performance to this detail.

During your testing did you profile it to see what is the hot spot? My initial guess is here -

inner = mapper.asObject(document.get("inner", Document.class), IMappable.class); 

[sheinbergon]

Thank you for your response.

No, I didn't profile the benchmark code.
Even without the inner/nested object de-serailization, this takes milliseconds.
Any improvement suggestions are more then welcomed.

[sheinbergon]

Running a few thread-dumps (without actually profiling the code) - it seems like it's a matter of reflection construction - using fixed/concrete constructors - the benchmarks went from 3000 us to 30!

This means that if you implement a reflective constructor cache in the AbstractMapper you could Drastically improve read performance !!!

However, this seems like a bug related to the Mappable interface itself - getting rid of it altogether also drastically improved performance (though not as good as using the Mappable read/write function concrete types), probably also due to jackson's reflection caches. While the documentation refers to this interface as an OOTB performance improvement, it seems that without upgrading the mapper to support concrete types, it's actually a performance killer.

[anidotnet]

Excellent investigation. I'll come up with a fix soon.

[anidotnet]

Can you run the same benchmark on the latest snapshot 3.2.0-SNAPSHOT now?

[sheinbergon]

Could you publish a snapshot to maven central (or some staging repo)?

BTW - I think you should also update your Mappable section in the documentation to exemplify overriding asObject, as I believe using concrete types in the mapper is still better than cached reflective constructor. Given that section is all about performance, it should be noted as an option.

[anidotnet]

It is already deployed in snapshot repository - https://oss.sonatype.org/content/repositories/snapshots/org/dizitart/nitrite/

But I didn't understand what do you mean by overriding asObject.

[DuskoV]

I'm wondering how i works in comparison with https://www.objectdb.com/

ObjectDB is a killer of H2, and Nitrite is using H2 storage engine, so this might be interesting comparison