You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
PDBStorageTest.testCanAddLargeValues runs the 512 MB test JVM out of heap: four copies of a 63 MB value live at once #1071
PDBStorageTest.testCanAddLargeValues writes values of 4, 32 and 63 MB, and on one leg the 512 MB of the test JVM were not enough for it. build-maven (ubuntu-latest, 17) of #1069 (run 35304975160), a PR which touches nothing near the test:
java.lang.OutOfMemoryError: Java heap space
at com.persistit.Value.ensureFit(Value.java:806)
at com.persistit.Value.putByteArray1(Value.java:3663)
at com.persistit.Value.put(Value.java:3408)
at com.persistit.Value.putByteArray(Value.java:3641)
at org.opends.server.backends.pdb.PDBStorage.bytesToValue(PDBStorage.java:1531)
at org.opends.server.backends.pdb.PDBStorage$WriteableStorageImpl.put(PDBStorage.java:443)
at org.opends.server.backends.pdb.PDBStorageTest$1.run(PDBStorageTest.java:166) <- the 32 MB put
Each value is copied four times on its way in: the source array, ByteString.valueOfBytes() (a copy), toByteArray() in PDBStorage.bytesToValue() (another), and the value buffer of the exchange, which doubles up to 64 MB. With the 63 MB value that is about 250 MB of humongous arrays live at once, on top of the buffer pool of the test storage (76 MB, 20% of the quota) and the test server. The test JVM runs one class per fork (reuseForks=false), so nothing else is in that heap: the margin is the test's own.
Measured under JDK 17.0.20 with -Xmx512m -Xlog:gc*, the settings of the leg:
variant
worst pause (before -> after)
humongous regions (before -> after)
master
436M -> 311M
300 -> 177
the three puts in three writes
452M -> 324M
302 -> 112
ByteString.wrap() for the sources, one write
471M -> 165M
328 -> 31
To Reproduce
Run PDBStorageTest under JDK 17 with -Xmx512m -Xlog:gc* and read the log of the fork: about 310 MB stay live after the collection which the 63 MB put triggers. Whether that leaves room for the next 64 MB allocation is up to the collector's timing.
Expected behavior
The test keeps well inside the heap of the test JVM. Wrapping the sources rather than copying them leaves about 165 MB live; splitting the puts over three writes does not help, since the value buffer of a fresh exchange grows again for each.
Activity
[#1071] Wrap the sources of the large values PDBStorageTest writes, s…