Skip to content

docs: Add batch processing example to README (#462) #476

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 4 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
75 changes: 75 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -333,7 +333,81 @@ client.chat()

ChatCompletion chatCompletion = chatCompletionAccumulator.chatCompletion();
```
## Batch Processing

The SDK supports batch processing of multiple requests through the Batch API. This is useful when you need to process a large number of requests efficiently. Here's how to use it:

```java
import com.openai.client.OpenAIClient;
import com.openai.client.okhttp.OpenAIOkHttpClient;
import com.openai.models.batches.Batch;
import com.openai.models.batches.BatchCreateParams;
import com.openai.models.batches.Endpoint;
import com.openai.models.files.FileCreateParams;
import com.openai.models.files.FileObject;
import com.openai.models.files.FilePurpose;
import java.nio.file.Paths;

// First, create a JSONL file containing your requests
// Example content of requests.jsonl:
// {"model": "gpt-4", "messages": [{"role": "user", "content": "Hello"}]}
// {"model": "gpt-4", "messages": [{"role": "user", "content": "World"}]}

// Upload the JSONL file
FileCreateParams fileParams = FileCreateParams.builder()
.purpose(FilePurpose.BATCH)
.file(Paths.get("requests.jsonl"))
.build();
FileObject file = client.files().create(fileParams);

// Create and execute the batch
BatchCreateParams batchParams = BatchCreateParams.builder()
.inputFileId(file.id())
.endpoint(Endpoint.CHAT_COMPLETIONS)
.build();
Batch batch = client.batches().create(batchParams);

// Check batch status
Batch retrievedBatch = client.batches().retrieve(
BatchRetrieveParams.builder()
.batchId(batch.id())
.build()
);

// Get batch results
if (retrievedBatch.status().equals("completed")) {
// Download and process the output file
String outputFileId = retrievedBatch.outputFileId().get();

// Download the output file
try (HttpResponse response = client.files().content(
FileContentParams.builder()
.fileId(outputFileId)
.build()
)) {
// Process the output file line by line
try (BufferedReader reader = new BufferedReader(
new InputStreamReader(response.body())
)) {
String line;
while ((line = reader.readLine()) != null) {
// Each line is a JSON object containing the response
// Process each response as needed
System.out.println("Response: " + line);
}
}
}
}
```

The batch API supports several endpoints:
- `/v1/responses`
- `/v1/chat/completions`
- `/v1/embeddings`
- `/v1/completions`

Each batch can contain up to 50,000 requests, and the input file can be up to 200 MB in size. The batch will be processed within a 24-hour window.
=======
## Structured outputs with JSON schemas

Open AI [Structured Outputs](https://platform.openai.com/docs/guides/structured-outputs?api-mode=chat)
Expand Down Expand Up @@ -533,6 +607,7 @@ If you use `@JsonProperty(required = false)`, the `false` value will be ignored.
must mark all properties as _required_, so the schema generated from your Java classes will respect
that restriction and ignore any annotation that would violate it.


## File uploads

The SDK defines methods that accept files.
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,105 @@
package com.openai.example;

import com.openai.client.OpenAIClientAsync;
import com.openai.client.okhttp.OpenAIOkHttpClientAsync;
import com.openai.core.http.HttpResponse;
import com.openai.models.batches.Batch;
import com.openai.models.batches.BatchCreateParams;
import com.openai.models.batches.BatchRetrieveParams;
import com.openai.models.batches.Endpoint;
import com.openai.models.files.FileContentParams;
import com.openai.models.files.FileCreateParams;
import com.openai.models.files.FileObject;
import com.openai.models.files.FilePurpose;

import java.io.BufferedReader;
import java.io.InputStreamReader;
import java.nio.file.Paths;
import java.util.concurrent.CompletableFuture;

/**
* Example demonstrating how to use the Batch API asynchronously to process multiple requests efficiently.
*
* The Batch API is useful when you need to process a large number of requests (up to 50,000)
* within a 24-hour window. The input file can be up to 200 MB in size.
*
* This example shows:
* 1. Creating a JSONL file with multiple requests
* 2. Uploading the file to OpenAI asynchronously
* 3. Creating and executing a batch asynchronously
* 4. Checking batch status asynchronously
* 5. Processing the results asynchronously
*/
public class BatchProcessingAsyncExample {
public static void main(String[] args) {
// Configures using the `OPENAI_API_KEY`, `OPENAI_ORG_ID`, `OPENAI_PROJECT_ID` and `OPENAI_BASE_URL` environment variables
OpenAIClientAsync client = OpenAIOkHttpClientAsync.fromEnv();

try {
// First, create a JSONL file containing your requests
// Example content of requests.jsonl:
// {"model": "gpt-4", "messages": [{"role": "user", "content": "Hello"}]}
// {"model": "gpt-4", "messages": [{"role": "user", "content": "World"}]}

// Upload the JSONL file asynchronously
FileCreateParams fileParams = FileCreateParams.builder()
.purpose(FilePurpose.BATCH)
.file(Paths.get("requests.jsonl"))
.build();
CompletableFuture<FileObject> fileFuture = client.files().create(fileParams);

// Create and execute the batch asynchronously
fileFuture.thenCompose(file -> {
BatchCreateParams batchParams = BatchCreateParams.builder()
.inputFileId(file.id())
.endpoint(Endpoint.CHAT_COMPLETIONS)
.build();
return client.batches().create(batchParams);
})
// Check batch status asynchronously
.thenCompose(batch -> client.batches().retrieve(
BatchRetrieveParams.builder()
.batchId(batch.id())
.build()
))
// Get batch results asynchronously
.thenCompose(retrievedBatch -> {
if (retrievedBatch.status().equals("completed")) {
String outputFileId = retrievedBatch.outputFileId().get();
return client.files().content(
FileContentParams.builder()
.fileId(outputFileId)
.build()
);
}
return CompletableFuture.completedFuture(null);
})
// Process the results
.thenAccept(response -> {
if (response != null) {
try (BufferedReader reader = new BufferedReader(
new InputStreamReader(response.body())
)) {
String line;
while ((line = reader.readLine()) != null) {
// Each line is a JSON object containing the response
// Process each response as needed
System.out.println("Response: " + line);
}
} catch (Exception e) {
System.out.println("Error processing results: " + e.getMessage());
}
}
})
.exceptionally(e -> {
System.out.println("Something went wrong!");
e.printStackTrace();
return null;
})
.join(); // Wait for all operations to complete
} catch (Exception e) {
System.out.println("Something went wrong!");
e.printStackTrace();
}
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
package com.openai.example;

import com.openai.client.OpenAIClient;
import com.openai.client.okhttp.OpenAIOkHttpClient;
import com.openai.core.http.HttpResponse;
import com.openai.models.batches.Batch;
import com.openai.models.batches.BatchCreateParams;
import com.openai.models.batches.BatchRetrieveParams;
import com.openai.models.batches.Endpoint;
import com.openai.models.files.FileContentParams;
import com.openai.models.files.FileCreateParams;
import com.openai.models.files.FileObject;
import com.openai.models.files.FilePurpose;

import java.io.BufferedReader;
import java.io.InputStreamReader;
import java.nio.file.Paths;

/**
* Example demonstrating how to use the Batch API to process multiple requests efficiently.
*
* The Batch API is useful when you need to process a large number of requests (up to 50,000)
* within a 24-hour window. The input file can be up to 200 MB in size.
*
* This example shows:
* 1. Creating a JSONL file with multiple requests
* 2. Uploading the file to OpenAI
* 3. Creating and executing a batch
* 4. Checking batch status
* 5. Processing the results
*/
public class BatchProcessingExample {
public static void main(String[] args) {
// Configures using the `OPENAI_API_KEY`, `OPENAI_ORG_ID`, `OPENAI_PROJECT_ID` and `OPENAI_BASE_URL` environment variables
OpenAIClient client = OpenAIOkHttpClient.fromEnv();

try {
// First, create a JSONL file containing your requests
// Example content of requests.jsonl:
// {"model": "gpt-4", "messages": [{"role": "user", "content": "Hello"}]}
// {"model": "gpt-4", "messages": [{"role": "user", "content": "World"}]}

// Upload the JSONL file
FileCreateParams fileParams = FileCreateParams.builder()
.purpose(FilePurpose.BATCH)
.file(Paths.get("requests.jsonl"))
.build();
FileObject file = client.files().create(fileParams);

// Create and execute the batch
BatchCreateParams batchParams = BatchCreateParams.builder()
.inputFileId(file.id())
.endpoint(Endpoint.CHAT_COMPLETIONS)
.build();
Batch batch = client.batches().create(batchParams);

// Check batch status
Batch retrievedBatch = client.batches().retrieve(
BatchRetrieveParams.builder()
.batchId(batch.id())
.build()
);

// Get batch results
if (retrievedBatch.status().equals("completed")) {
// Download and process the output file
String outputFileId = retrievedBatch.outputFileId().get();

// Download the output file
try (HttpResponse response = client.files().content(
FileContentParams.builder()
.fileId(outputFileId)
.build()
)) {
// Process the output file line by line
try (BufferedReader reader = new BufferedReader(
new InputStreamReader(response.body())
)) {
String line;
while ((line = reader.readLine()) != null) {
// Each line is a JSON object containing the response
// Process each response as needed
System.out.println("Response: " + line);
}
}
}
}
} catch (Exception e) {
System.out.println("Something went wrong!");
e.printStackTrace();
}
}
}