Skip to content

Comments

Server-side batching (SSB)#537

Draft
bevzzz wants to merge 32 commits intomainfrom
feat/ssb
Draft

Server-side batching (SSB)#537
bevzzz wants to merge 32 commits intomainfrom
feat/ssb

Conversation

@bevzzz
Copy link
Collaborator

@bevzzz bevzzz commented Feb 17, 2026

This PR adds BatchContext which implements Server-Side Batching protocol.

List<WeaviateObject<Map<String, Object>>> objects = List.of(...);
List<ObjectReference> references = List.of();
List<CompletableFuture<?>> submitted = new ArrayList<>();

try (BatchContext batch = collection.batch.start()) {
  
  // Optional: Define a retry policy that retries each task up to three times.
  Function<TaskHandle, TaskHandle> retryMax3Times = task -> task
    .result().thenCompose(result -> {
      if (result.error().isEmpty() {
        return result;
      }
      if (task.timesRetried == 3) {
        throw new RuntimeException(result.error().get());
      }
      return retryMax3Times.apply(batch.retry(task));
    });

  // Submit all objects to the batch
  objects.map(batch::add).map(retryMax3Times).forEach(submitted::add);
  
  // Submit all references to the batch
  references.map(batch::add).map(retryMax3Times).forEach(submitted::add;
}

// Await until all tasks complete
CompletableFuture.allOf(submitted).get();

Description

TODO

Extended GrpcTransport interface to implements StreamFactory.
Added documentation to batch's primitives.
Renamed Message -> Batch, StreamMessage -> Message, MessageProducer -> Messeger, and EventHandler -> Eventer.
Extracted message size calculation into MessageSizeUtil.
Added sketch implementation for the Send routine.
Still raw and riddled with comments, but it's a good start.
Copy link

@orca-security-eu orca-security-eu bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Orca Security Scan Summary

Status Check Issues by priority
Passed Passed Infrastructure as Code high 0   medium 0   low 0   info 0 View in Orca
Passed Passed SAST high 0   medium 0   low 0   info 0 View in Orca
Passed Passed Secrets high 0   medium 0   low 0   info 0 View in Orca
Passed Passed Vulnerabilities high 0   medium 0   low 0   info 0 View in Orca

LinkedHashMap::reversed was first introduced in JDK 21.
The ListIterator approach requires allocating a list,
but yields a much simpler code in return.
BatchContext can deal with happy path, i.e. no oom, no shutdowns, etc.
Calculating the size precisely is possible, but requires
writing code that understands the internal structure of
the gRPC messages. A much simpler approach is to estimate
the size approximately and keep a tiny (compared to the
total message size) margin for error.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant