Skip to content

Commit

Permalink
feat: Add support for Message Batches in anthropic_sdk_dart (#585)
Browse files Browse the repository at this point in the history
  • Loading branch information
davidmigloz authored Oct 22, 2024
1 parent 4f0d9cf commit a41270a
Show file tree
Hide file tree
Showing 13 changed files with 1,680 additions and 5 deletions.
65 changes: 65 additions & 0 deletions packages/anthropic_sdk_dart/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,13 +17,15 @@ Unofficial Dart client for [Anthropic](https://docs.anthropic.com/en/api) API (a
**Supported endpoints:**

- Messages (with tools and streaming support)
- Message Batches

## Table of contents

- [Usage](#usage)
* [Authentication](#authentication)
* [Messages](#messages)
* [Tool use](#tool-use)
* [Message Batches](#message-batches)
- [Advance Usage](#advance-usage)
* [Default HTTP client](#default-http-client)
* [Custom HTTP client](#custom-http-client)
Expand Down Expand Up @@ -242,6 +244,69 @@ await for (final res in stream) {
// {"location": "Boston, MA", "unit": "fahrenheit"}
```

### Message Batches

The Message Batches API is a powerful, cost-effective way to asynchronously process large volumes of Messages requests. This approach is well-suited to tasks that do not require immediate responses, reducing costs by 50% while increasing throughput.

Refer to the [official documentation](https://docs.anthropic.com/en/docs/build-with-claude/message-batches) for more information.

**Prepare and create your batch:**

```dart
const batchRequest = CreateMessageBatchRequest(
requests: [
BatchMessageRequest(
customId: 'request1',
params: CreateMessageRequest(
model: Model.model(Models.claudeInstant12),
temperature: 0,
maxTokens: 1024,
messages: [
Message(
role: MessageRole.user,
content: MessageContent.text(
'List the numbers from 1 to 9 in order.'),
),
],
),
),
BatchMessageRequest(
customId: 'request2',
params: CreateMessageRequest(
model: Model.model(Models.claudeInstant12),
temperature: 0,
maxTokens: 1024,
messages: [
Message(
role: MessageRole.user,
content: MessageContent.text(
'List the numbers from 10 to 19 in order.'),
),
],
),
),
],
);
var batch = await client.createMessageBatch(request: batchRequest);
print(batch.id);
```

**Tracking your batch:**

```dart
do {
await Future<void>.delayed(const Duration(seconds: 5));
batch = await client.retrieveMessageBatch(id: batch.id);
} while (batch.processingStatus == MessageBatchProcessingStatus.inProgress);
```

**Retrieving batch results:**

```dart
batch = await client.retrieveMessageBatch(id: batch.id);
print(batch.resultsUrl);
```

## Advance Usage

### Default HTTP client
Expand Down
1 change: 1 addition & 0 deletions packages/anthropic_sdk_dart/lib/src/client.dart
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,7 @@ class AnthropicClient extends g.AnthropicClient {
baseUrl: baseUrl,
headers: {
'anthropic-version': '2023-06-01',
'anthropic-beta': 'message-batches-2024-09-24',
...?headers,
},
queryParams: queryParams ?? const {},
Expand Down
59 changes: 59 additions & 0 deletions packages/anthropic_sdk_dart/lib/src/generated/client.dart
Original file line number Diff line number Diff line change
Expand Up @@ -392,4 +392,63 @@ class AnthropicClient {
);
return Message.fromJson(_jsonDecode(r));
}

// ------------------------------------------
// METHOD: createMessageBatch
// ------------------------------------------

/// Create a Message Batch
///
/// Send a batch of Message creation requests.
///
/// `request`: The request parameters for creating a message batch.
///
/// `POST` `https://api.anthropic.com/v1/messages/batches`
Future<MessageBatch> createMessageBatch({
required CreateMessageBatchRequest request,
}) async {
final r = await makeRequest(
baseUrl: 'https://api.anthropic.com/v1',
path: '/messages/batches',
method: HttpMethod.post,
isMultipart: false,
requestType: 'application/json',
responseType: 'application/json',
body: request,
headerParams: {
if (apiKey.isNotEmpty) 'x-api-key': apiKey,
},
);
return MessageBatch.fromJson(_jsonDecode(r));
}

// ------------------------------------------
// METHOD: retrieveMessageBatch
// ------------------------------------------

/// Retrieve a Message Batch
///
/// This endpoint is idempotent and can be used to poll for Message Batch
/// completion. To access the results of a Message Batch, make a request to the
/// `results_url` field in the response.
///
/// `id`: The ID of the message batch to retrieve.
///
/// `GET` `https://api.anthropic.com/v1/messages/batches/{id}`
Future<MessageBatch> retrieveMessageBatch({
required String id,
}) async {
final r = await makeRequest(
baseUrl: 'https://api.anthropic.com/v1',
path: '/messages/batches/$id',
method: HttpMethod.get,
isMultipart: false,
requestType: '',
responseType: 'application/json',
headerParams: {
if (apiKey.isNotEmpty) 'x-api-key': apiKey,
},
);
return MessageBatch.fromJson(_jsonDecode(r));
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
// coverage:ignore-file
// GENERATED CODE - DO NOT MODIFY BY HAND
// ignore_for_file: type=lint
// ignore_for_file: invalid_annotation_target
part of anthropic_schema;

// ==========================================
// CLASS: BatchMessageRequest
// ==========================================

/// An individual message request within a batch.
@freezed
class BatchMessageRequest with _$BatchMessageRequest {
const BatchMessageRequest._();

/// Factory constructor for BatchMessageRequest
const factory BatchMessageRequest({
/// Developer-provided ID created for each request in a Message Batch. Useful for
/// matching results to requests, as results may be given out of request order.
///
/// Must be unique for each request within the Message Batch.
@JsonKey(name: 'custom_id') required String customId,

/// The request parameters for creating a message.
required CreateMessageRequest params,
}) = _BatchMessageRequest;

/// Object construction from a JSON representation
factory BatchMessageRequest.fromJson(Map<String, dynamic> json) =>
_$BatchMessageRequestFromJson(json);

/// List of all property names of schema
static const List<String> propertyNames = ['custom_id', 'params'];

/// Perform validations on the schema property values
String? validateSchema() {
return null;
}

/// Map representation of object (not serialized)
Map<String, dynamic> toMap() {
return {
'custom_id': customId,
'params': params,
};
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ sealed class Block with _$Block {
/// The name of the tool being used.
required String name,

/// An object containing the input being passed to the tool, conforming to the tools `input_schema`.
/// An object containing the input being passed to the tool, conforming to the tool's `input_schema`.
required Map<String, dynamic> input,

/// The type of content block.
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
// coverage:ignore-file
// GENERATED CODE - DO NOT MODIFY BY HAND
// ignore_for_file: type=lint
// ignore_for_file: invalid_annotation_target
part of anthropic_schema;

// ==========================================
// CLASS: CreateMessageBatchRequest
// ==========================================

/// The request parameters for creating a message batch.
@freezed
class CreateMessageBatchRequest with _$CreateMessageBatchRequest {
const CreateMessageBatchRequest._();

/// Factory constructor for CreateMessageBatchRequest
const factory CreateMessageBatchRequest({
/// List of requests for prompt completion. Each is an individual request to create a Message.
required List<BatchMessageRequest> requests,
}) = _CreateMessageBatchRequest;

/// Object construction from a JSON representation
factory CreateMessageBatchRequest.fromJson(Map<String, dynamic> json) =>
_$CreateMessageBatchRequestFromJson(json);

/// List of all property names of schema
static const List<String> propertyNames = ['requests'];

/// Perform validations on the schema property values
String? validateSchema() {
return null;
}

/// Map representation of object (not serialized)
Map<String, dynamic> toMap() {
return {
'requests': requests,
};
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,98 @@
// coverage:ignore-file
// GENERATED CODE - DO NOT MODIFY BY HAND
// ignore_for_file: type=lint
// ignore_for_file: invalid_annotation_target
part of anthropic_schema;

// ==========================================
// CLASS: MessageBatch
// ==========================================

/// A batch of message requests.
@freezed
class MessageBatch with _$MessageBatch {
const MessageBatch._();

/// Factory constructor for MessageBatch
const factory MessageBatch({
/// Unique object identifier for the message batch.
required String id,

/// RFC 3339 datetime string representing the time at which the Message Batch was created.
@JsonKey(name: 'created_at') required String createdAt,

/// RFC 3339 datetime string representing the time at which the Message Batch will expire and end processing, which is 24 hours after creation.
@JsonKey(name: 'expires_at') required String expiresAt,

/// Processing status of the Message Batch.
@JsonKey(name: 'processing_status')
required MessageBatchProcessingStatus processingStatus,

/// Tallies requests within the Message Batch, categorized by their status.
@JsonKey(name: 'request_counts')
required MessageBatchRequestCounts requestCounts,

/// URL to a `.jsonl` file containing the results of the Message Batch requests. Specified only once processing ends.
@JsonKey(name: 'results_url', includeIfNull: false) String? resultsUrl,

/// Object type. For Message Batches, this is always `"message_batch"`.
required MessageBatchType type,
}) = _MessageBatch;

/// Object construction from a JSON representation
factory MessageBatch.fromJson(Map<String, dynamic> json) =>
_$MessageBatchFromJson(json);

/// List of all property names of schema
static const List<String> propertyNames = [
'id',
'created_at',
'expires_at',
'processing_status',
'request_counts',
'results_url',
'type'
];

/// Perform validations on the schema property values
String? validateSchema() {
return null;
}

/// Map representation of object (not serialized)
Map<String, dynamic> toMap() {
return {
'id': id,
'created_at': createdAt,
'expires_at': expiresAt,
'processing_status': processingStatus,
'request_counts': requestCounts,
'results_url': resultsUrl,
'type': type,
};
}
}

// ==========================================
// ENUM: MessageBatchProcessingStatus
// ==========================================

/// Processing status of the Message Batch.
enum MessageBatchProcessingStatus {
@JsonValue('in_progress')
inProgress,
@JsonValue('canceling')
canceling,
@JsonValue('ended')
ended,
}

// ==========================================
// ENUM: MessageBatchType
// ==========================================

/// Object type. For Message Batches, this is always `"message_batch"`.
enum MessageBatchType {
@JsonValue('message_batch')
messageBatch,
}
Loading

0 comments on commit a41270a

Please sign in to comment.