Skip to content

Add support for JSON-RPC batching #67

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Apr 10, 2025
Merged

Add support for JSON-RPC batching #67

merged 5 commits into from
Apr 10, 2025

Conversation

mattt
Copy link
Contributor

@mattt mattt commented Apr 10, 2025

Resolves #62

Because Transport brokers Data objects, it can support sending and receiving JSON-RPC arrays without modification.

Server adds an internal Server.Batch structure and adds logic to decode arrays received from its transport connection. All user code will work without additional changes.

Client adds a public, Client.Batch actor, which is passed as an argument to a new withBatch method. API consumers can now make multiple requests in a single batch and receive their results asynchronously.

Example 1: Batching multiple tool calls and collecting typed tasks:

// Array to hold the task handles for each tool call
var toolTasks: [Task<CallTool.Result, Error>] = []
try await client.withBatch { batch in
    for i in 0..<10 {
        toolTasks.append(
            try await batch.addRequest(
                CallTool.request(.init(name: "square", arguments: ["n": i]))
            )
        )
    }
}

// Process results after the batch is sent
print("Processing \(toolTasks.count) tool results...")
for (index, task) in toolTasks.enumerated() {
    do {
        let result = try await task.value
        print("\(index): \(result.content)")
    } catch {
        print("\(index) failed: \(error)")
    }
}

Example 2: Batching different request types and awaiting individual tasks:

// Declare optional task variables beforehand
var pingTask: Task<Ping.Result, Error>?
var promptTask: Task<GetPrompt.Result, Error>?

try await client.withBatch { batch in
    // Assign the tasks within the batch closure
    pingTask = try await batch.addRequest(Ping.request())
    promptTask = try await batch.addRequest(GetPrompt.request(.init(name: "greeting")))
}

// Await the results after the batch is sent
do {
    if let pingTask = pingTask {
        try await pingTask.value // Await ping result (throws if ping failed)
        print("Ping successful")
    }
    if let promptTask = promptTask {
        let promptResult = try await promptTask.value // Await prompt result
        print("Prompt description: \(promptResult.description ?? "None")")
    }
} catch {
    print("Error processing batch results: \(error)")
}

@mattt mattt merged commit 82f4fd2 into main Apr 10, 2025
4 checks passed
@mattt mattt deleted the mattt/jsonrpc-batch branch April 10, 2025 19:13
@mattt mattt self-assigned this Apr 15, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Development

Successfully merging this pull request may close these issues.

Support JSON-RPC batching
1 participant