supprt openai batch api

[mkXultra]

Improving EPUB Translation Efficiency with ChatGPT Batch API Implementation

This PR implements functionality that significantly improves the efficiency of the EPUB translation process by utilizing ChatGPT's batch API. The main changes are as follows:

Major Feature Additions

1. Implementation of Batch Translation Feature

   • --batch option: Batch process translations using ChatGPT's batch API
   • --batch-use option: Create files using pre-generated batch translation results

2. Batch Processing Workflow

   • Creation and management of batch requests
   • Execution of batch jobs and status checking
   • Retrieval and processing of batch results

Batch Processing Mechanism

Pattern for Creating Batches (--batch option)

1. Initialization

   • Initial setup for batch processing with the batch_init method
   • Sanitization of book names

2. Creation of Translation Queue

   • Add texts to be translated to the queue using the add_to_batch_translate_queue method

3. Generation of Batch Files

   • Creation of batch requests
     • Create translation requests
     • (OPTIONAL) Create context
       • Control update frequency using BATCH_CONTEXT_UPDATE_INTERVAL
       • Efficient context updates with the create_batch_context_messages method
   • Convert batch requests to JSONL files using the create_batch_files method
   • Limit of 40,000 lines per file

4. Batch Execution

   • Control the entire batch process with the batch method
   • Upload files, execute batch jobs, save metadata

5. Saving Batch Information

   • Save batch processing results and metadata as JSON files

Pattern for Using Batches (--batch-use option)

1. Checking Batch Results

   • Confirm completion of batch processing with the is_completed_batch method

2. Retrieval and Processing of Results

   • Retrieve translation results with the batch_translate method
   • Efficiently manage results using cache

Context Generation for Batch Processing

Batch processing implements a different context management method compared to normal sequential processing:

1. Setting Context Update Interval

   • Update context every BATCH_CONTEXT_UPDATE_INTERVAL (default 50)
   • Improve processing efficiency by avoiding frequent updates

2. Specificity of Context Generation

   • Normal processing: Use and cache the immediately preceding translation result as context
   • Batch processing: Generate context using API as translation results are not immediately available

3. Context Generation Logic

   • Collect up to context_paragraph_limit paragraphs of 100+ words, tracing back from the index of the text to be translated
   • Generate context by translating the collected paragraphs using API

4. Efficient Updates and Caching

   • Manage updates with the create_batch_context_messages method
   • Cache generated contexts and reuse until the next update

5. Differences from Existing Behavior

   • Normal processing: Directly use translation results as context
   • Batch processing: Use API-translated original text as context

This method optimizes processing efficiency while maintaining appropriate context information during batch processing. However, due to the nature of batch processing, there may be slight differences in translation quality compared to normal sequential processing as the context generation method differs.

Files and Directories Created by Batch Processing

Batch processing creates the following directory structure and files:

1. Root Directory

   • Location: Current working directory (os.getcwd())
   • Structure:

     current_working_directory/
     └── batch_files/
         ├── {book_name}_info.json
         └── {book_name}/
             ├── 1.jsonl
             ├── 2.jsonl
             ├── 3.jsonl
             └── ...
     

2. batch_files Directory

   • Purpose: Store all files related to batch processing
   • Location: {current_working_directory}/batch_files/

3. {book_name}_info.json

   • Purpose: Save metadata for batch processing
   • Location: {current_working_directory}/batch_files/{book_name}_info.json
   • Contents:
     • Book ID
     • Batch processing date and time
     • Information for each batch file (input file ID, batch ID, start index, end index)

4. {book_name} Directory

   • Purpose: Store batch files related to a specific book
   • Location: {current_working_directory}/batch_files/{book_name}/

5. Batch Request Files ({number}.jsonl)

   • Purpose: Contain batch requests for ChatGPT API
   • Location: {current_working_directory}/batch_files/{book_name}/{number}.jsonl
   • Format: JSONL (JSON Lines)
   • Contents: One request object per line (custom ID, method, URL, body)

Notes:

• {book_name} is a safe directory name generated from the original file name (special characters replaced with '_')
• Each JSONL file contains a maximum of 40,000 lines (requests) and is limited to 100MB or less
• Batch processing results are temporarily stored on OpenAI's servers and retrieved later

Technical Improvements

• Extension of ChatGPTAPI class: Addition of batch processing-related methods
• Enhanced file processing: Generation and management of JSONL files for batches
• Addition of error handling and timeout processing
• Introduction of caching mechanism for improved efficiency

Expected Effects

• Reduced processing time for large-scale translation tasks
• Cost reduction through optimization of API requests
• Improved flexibility through reuse of batch processing results

Important Notes

• When using batch processing, you must first execute batch translation with the --batch option, then use the results with the --batch-use option.
• Batch processing results are temporarily saved to the file system, so adequate disk space is required.
• As the frequency of context updates is adjusted during batch processing, there may be slight differences in translation quality compared to normal sequential processing.

Usage Examples

1.a Executing batch translation (pattern without using context):

python3 make_book.py --book_name test_books/animal_farm.epub \
--model gpt4omini \
--language ja \
--batch 

1.b Executing batch translation (pattern using context):

python3 make_book.py --book_name test_books/animal_farm.epub \
--model gpt4omini \
--language ja \
--use_context \
--batch 

2. Generating bilingual EPUB using batch translation results:

   python3 make_book.py --book_name test_books/animal_farm.epub \
   --model gpt4omini \
   --language ja \
   --batch-use
   

This implementation significantly improves processing efficiency, especially for EPUB files containing large amounts of text. Users can manage the translation process more flexibly, and overall performance is improved. By using batch processing, large-scale translation tasks can be executed efficiently and API requests can be optimized.

[yihong0618]

let's make a config.py file for these?

[mkXultra]

@yihong0618
I created config.py and modified the code to use it. Since this is my first time creating config.py, I'm not sure if I'm using it correctly. Could you please review it?

fd92f7a

[yihong0618]

nice

[yihong0618]

I think this name is not support windows
can we use pathlib or os.path

[mkXultra]

I've fixed it
#423 (comment)

[yihong0618]

ditto seems not support windows file name

[mkXultra]

I fixed it
f1e78eb