ERS: the sorted hash lists now sort a copy in toList() and answer getFirst() with a scan, so reading one no longer reorders it, relates to github #2456.

Author: dghgit

CONTRIBUTORS.md

@@ -548,3 +548,4 @@ We also wish to acknowledge financial and collaborative support from [CISCO](htt
 - vladhuma \<https://github.com/vladhuma\> - initial implementation of server-side OCSP stapling for the BCJSSE provider, on behalf of Thales Group (PR #1740).
 - Bhargava Shastry \<bshastry&#064;posteo.de\> - reporting, with a self-checking reproducer, that the C509 validity fields were read as unbounded seconds and then given two meanings: displacing a type-3 certificate's validityNotAfter by 2^61 seconds left the reconstructed DER TBSCertificate and the issuer signature untouched, while C509CertificateHolder.isValidOn took the raw value and read an expired certificate as valid.
 - Rob Augustinus \<rob&#064;opensolutions.nl\> - a constant-time analysis of the elliptic-curve and AES code paths, with proof-of-concept code, which independently reached the variable-point scalar multiplication exposure addressed in 1.86.
+- Radim Dejmek \<dejmekr&#064;seznam.cz\> - initial implementation of the rebuild of the ERS hash lists around a single sort, which had been finding the insertion point for each hash by walking a LinkedList (PR #2457).

docs/releasenotes.md

@@ -24,6 +24,7 @@ Date: 2026, TBD
 - The name-constraint host canonicalisation removed a single RFC 1034 root-label dot, the only empty label a name may legally carry, but nothing refused the ones that are not legal: a dNSName, rfc822Name host or uniformResourceIdentifier host such as "example.com.." kept a phantom empty label after the strip and so matched no constraint at all, escaping an excluded subtree naming the host it appears to carry. A tested name whose host carries an empty label - a second trailing dot, a doubled dot or a leading dot - is now refused outright wherever a constraint of that type is in force, rather than canonicalised into a name it is not: removing the extra dots would decide on the caller's behalf that "example.com.." names example.com, which is not how a consumer resolving or comparing the name reads it, and refusing fails closed in both directions where canonicalising would newly admit such a name under a permitted subtree. The single trailing dot is canonicalised as before, a bare "." remains the root label rather than an empty one, and the guard is scoped to the host, so the doubled dot a quoted local part may legally carry is unaffected. Constraints are untouched - one may still begin with a dot, which is how this implementation spells "subdomains only" (github PR #2436).
 - SSLContext.createSSLEngine() from the BCJSSE provider in the 1.86 bctls jar failed with NoSuchMethodError on every JDK from 9 up, leaving engine-based users of the provider (Netty, Vert.x and the like) unable to open a connection. The jdk1.5 and jdk1.9 copies of the package-private SSLEngineUtil had declared create(ContextData) with different return types since 2019 - SSLEngine and ProvSSLEngine - and the root ProvSSLContextSpi, compiled against the first, is paired at runtime with the versions/9 copy on any modern JDK. Until 1.86 the java9 compile had hidden this by implicitly recompiling the whole base tree into META-INF/versions/9 (447 classes, ProvSSLContextSpi among them); the -implicit:none added in 1.86 to stop that duplication exposed the mismatch. The jdk1.9 copy now declares the same return type as the root one, a JDK 25 test creates engines against the built jar, and a new multiReleaseCheck Gradle task on every distributed jar reads the constant pool of each class in the jar and in the sibling BC jars it depends on and fails the build when a member reference does not resolve against the copy of its target that a JDK would pair it with, so the class of defect cannot ship again; the same check runs on arbitrary jars, a published release included, as multiReleaseCheckJar (github #2448).
 - The BCFKS key store derived its scrypt keys with the block size r in place of the parallelization parameter p, while writing the p the caller configured out to the store: BcFKSKeyStoreSpi passed getBlockSize() to SCrypt.generate for both arguments and never read the encoded parallelization parameter at all, so every store whose ScryptConfig gave a p other than its r encoded parameters that do not derive its own keys. The store was self-consistent - BC read back what BC wrote - but a conformant RFC 7914 reader computed a different key and so failed the integrity check and the store decryption, and BC could not open such a store written by anyone else. Derivation now follows RFC 7914. A store written by 1.86 or earlier is still read: the integrity check is retried under the old convention, and where a signature check leaves no MAC to settle it the store decryption is retried instead, in both cases reporting the failure of the encoded parameters rather than of the fallback. The write side is governed by org.bouncycastle.bcfks.scrypt_p_eq_r, default true, which writes p equal to r whatever the ScryptConfig asked for: the two conventions then agree, so a store written here is both RFC 7914 correct and readable by 1.86 and earlier. Clearing the property honours the configured p, which those releases cannot read unless p already equals r; the default is intended to become false in a later release, once enough of the installed base is writing parameters that describe themselves. Loading with a BCFKSLoadStoreParameter carrying a ScryptConfig accepts an encoded p equal to either the configured p or the block size, so a store round trips under the configuration that wrote it whichever way the property was set; every other parameter is compared as before. The parallelization parameter is now bounded alongside the cost parameter before the derivation, as the PKCS#8 and PKCS#12 scrypt paths already bound it.
+- Building an evidence record was cubic in the number of data objects: SortedHashList and SortedIndexedHashList held their hashes in a LinkedList and found each insertion point by walking it with get(index), so a single add() was quadratic in the position it inserted at and building a list of n hashes cubic, and both lists sit on the generation path - the reduced hash tree of ERSArchiveTimeStampGenerator, the Merkle tree of BinaryTreeRootCalculator.computeRootHash(), and the hash list of every ERSDataGroup. Each now collects its hashes and sorts them once, in toList(); the sort is stable and the old insertion placed a hash after the last one comparing equal to it, which is where a stable sort puts it, so the order of the leaves and every root hash are unchanged. getFirst() answers with a scan rather than a sort and toList() sorts a copy, so neither accessor disturbs what has been added. Generating a time-stamp request over 8,000 data objects goes from about 210 seconds to under a tenth of a second, and 100,000 objects, which the old code could not reach in any practical time, takes about 0.2 seconds (github #2456).
 
 ### 2.1.3 Additional Features and Functionality
 

pkix/src/main/java/org/bouncycastle/tsp/ers/SortedHashList.java

@@ -15,8 +15,6 @@ public class SortedHashList
 
     private final List<byte[]> baseList = new ArrayList<byte[]>();
 
-    private boolean isSorted = true;
-
     public SortedHashList()
     {
     }
@@ -28,43 +26,45 @@ public byte[] getFirst()
             throw new NoSuchElementException();
         }
 
-        sort();
+        byte[] first = (byte[])baseList.get(0);
+
+        for (int i = 1; i != baseList.size(); i++)
+        {
+            byte[] next = (byte[])baseList.get(i);
+
+            // strictly less than, so the earliest added of a set of equal hashes is returned
+            if (hashComp.compare(next, first) < 0)
+            {
+                first = next;
+            }
+        }
 
-        return (byte[])baseList.get(0);
+        return first;
     }
 
     public void add(byte[] hash)
     {
         baseList.add(hash);
-        isSorted = false;
     }
 
     public int size()
     {
         return baseList.size();
     }
 
-    public List<byte[]> toList()
-    {
-        sort();
-
-        return new ArrayList<byte[]>(baseList);
-    }
-
     /**
-     * Sorting is deferred to the accessors. Inserting each hash on add() meant searching a
-     * LinkedList for the insertion point with get(index), which is O(index), so a single add()
-     * was O(n^2) and building a list of n hashes was O(n^3).
+     * Return the hashes added so far in ascending order.
      * <p>
-     * Collections.sort() is stable, so hashes comparing equal keep the order they were added
-     * in - which is where inserting after the last equal element used to put them.
+     * The sort is stable, so hashes comparing equal come back in the order they were added in.
+     *
+     * @return a sorted list of the hashes added.
      */
-    private void sort()
+    public List<byte[]> toList()
     {
-        if (!isSorted)
-        {
-            Collections.sort(baseList, hashComp);
-            isSorted = true;
-        }
+        List<byte[]> sorted = new ArrayList<byte[]>(baseList);
+
+        Collections.sort(sorted, hashComp);
+
+        return sorted;
     }
 }

pkix/src/main/java/org/bouncycastle/tsp/ers/SortedIndexedHashList.java

@@ -23,8 +23,6 @@ public int compare(IndexedHash l, IndexedHash r)
 
     private final List<IndexedHash> baseList = new ArrayList<IndexedHash>();
 
-    private boolean isSorted = true;
-
     public SortedIndexedHashList()
     {
     }
@@ -36,40 +34,45 @@ public IndexedHash getFirst()
             throw new NoSuchElementException();
         }
 
-        sort();
+        IndexedHash first = (IndexedHash)baseList.get(0);
+
+        for (int i = 1; i != baseList.size(); i++)
+        {
+            IndexedHash next = (IndexedHash)baseList.get(i);
 
-        return (IndexedHash)baseList.get(0);
+            // strictly less than, so the earliest added of a set of equal hashes is returned
+            if (digestComp.compare(next, first) < 0)
+            {
+                first = next;
+            }
+        }
+
+        return first;
     }
 
     public void add(IndexedHash hash)
     {
         baseList.add(hash);
-        isSorted = false;
     }
 
     public int size()
     {
         return baseList.size();
     }
 
+    /**
+     * Return the hashes added so far in ascending order of digest.
+     * <p>
+     * The sort is stable, so hashes comparing equal come back in the order they were added in.
+     *
+     * @return a sorted list of the hashes added.
+     */
     public List<IndexedHash> toList()
     {
-        sort();
+        List<IndexedHash> sorted = new ArrayList<IndexedHash>(baseList);
 
-        return new ArrayList<IndexedHash>(baseList);
-    }
+        Collections.sort(sorted, digestComp);
 
-    /**
-     * Sorting is deferred to the accessors, for the reason given on SortedHashList.sort():
-     * finding the insertion point with LinkedList.get(index) made building a list of n hashes
-     * O(n^3). Collections.sort() is stable, so hashes comparing equal keep ascending order.
-     */
-    private void sort()
-    {
-        if (!isSorted)
-        {
-            Collections.sort(baseList, digestComp);
-            isSorted = true;
-        }
+        return sorted;
     }
 }

pkix/src/test/java/org/bouncycastle/tsp/test/ERSTest.java

@@ -21,6 +21,7 @@
 import java.util.HashSet;
 import java.util.LinkedList;
 import java.util.List;
+import java.util.NoSuchElementException;
 import java.util.Random;
 
 import junit.framework.TestCase;
@@ -1457,13 +1458,72 @@ private int compareUnsigned(byte[] left, byte[] right)
     }
 
     /**
-     * A reduced hash tree over a large number of data objects. This reaches both sorted lists -
+     * toList() sorts a copy, so the list handed back belongs to the caller and the accessors can
+     * be interleaved with add() in any order. getFirst() answers without sorting at all.
+     */
+    public void testSortedHashListAccessors()
+    {
+        SortedHashList list = new SortedHashList();
+
+        assertEquals(0, list.size());
+        assertTrue(list.toList().isEmpty());
+
+        try
+        {
+            list.getFirst();
+            fail("no exception on empty list");
+        }
+        catch (NoSuchElementException e)
+        {
+            // expected - as LinkedList.getFirst() did
+        }
+
+        byte[] three = Hex.decode("03");
+        byte[] oneA = Hex.decode("01");
+        byte[] two = Hex.decode("02");
+        byte[] oneB = Hex.decode("01");
+
+        byte[][] values = new byte[][]{three, oneA, two, oneB};
+        for (int i = 0; i != values.length; i++)
+        {
+            list.add(values[i]);
+        }
+
+        List<byte[]> first = list.toList();
+
+        assertEquals(4, first.size());
+        // equal hashes come back in the order they were added in
+        assertTrue(oneA == first.get(0));
+        assertTrue(oneB == first.get(1));
+        assertTrue(two == first.get(2));
+        assertTrue(three == first.get(3));
+        assertTrue(oneA == list.getFirst());
+
+        // the caller owns the list returned, and the one before it
+        first.clear();
+
+        List<byte[]> second = list.toList();
+
+        assertEquals(4, list.size());
+        assertEquals(4, second.size());
+        assertTrue(oneA == second.get(0));
+
+        byte[] zero = Hex.decode("00");
+
+        list.add(zero);
+
+        assertEquals(5, list.size());
+        assertTrue(zero == list.getFirst());
+        assertTrue(zero == list.toList().get(0));
+    }
+
+    /**
+     * A reduced hash tree over a large number of data objects, reaching both sorted lists -
      * SortedIndexedHashList from ERSArchiveTimeStampGenerator.getPartialHashtrees(), and
-     * SortedHashList from BinaryTreeRootCalculator.computeRootHash() - and took about 2.5
-     * seconds for these 2,000 objects when the insertion point was found by walking a
-     * LinkedList, rising by roughly a factor of eight per doubling (10,000 objects took 347
-     * seconds). The root is also checked to be independent of the order the objects were added
-     * in, which is what the sorting is there for.
+     * SortedHashList from BinaryTreeRootCalculator.computeRootHash(). Finding each insertion
+     * point by walking a LinkedList made this cubic in the number of data objects; the root is
+     * also checked to be independent of the order the objects were added in, which is what the
+     * sorting is there for.
      */
     public void testLargeDataObjectSet()
         throws Exception