openai-ft-validate is a Command-Line Interface (CLI) tool written in Go, designed to validate the structure of JSONL files used in OpenAI fine-tuning processes. This tool ensures that the input files adhere to OpenAI's formatting requirements, helping users avoid errors during fine-tuning.
- Validates JSONL files for structural correctness.
- Checks for required fields and their data types.
- Provides clear error messages for invalid entries.
- Ensures compatibility with OpenAI fine-tuning guidelines.
- Lightweight and easy to use.
To install openai-ft-validate, ensure you have Go installed, then clone the repository and build the binary:
# Clone the repository
git clone https://github.com/Siddhesh-Agarwal/openai-ft-validate
cd openai-ft-validate
# Build the binary
go build -o openai-ft-validateAlternatively, you can download a prebuilt binary (if available) from the releases page.
The tool can be invoked directly from the command line. Below are the usage details:
./openai-ft-validate <path-to-jsonl-file>To validate a JSONL file named training_data.jsonl:
./openai-ft-validate training_data.jsonl-
Success: If the file is valid, you will see a message like:
The JSONL is valid 🎉
-
Failure: If the file contains errors, the tool will output details, such as:
Line 3: Missing required field "prompt". Line 7: Invalid "role" programmer found The JSONL is invalid ❌
- Go 1.16+
For contributors, clone the repository and build the project:
git clone https://github.com/Siddhesh-Agarwal/openai-ft-validate
cd openai-ft-validate
# Build the binary
go buildRun tests to ensure functionality:
This tool is licensed under MIT License.
Contributions are welcome! Please fork the repository, make your changes, and submit a pull request. Ensure your code follows the established coding standards and includes tests for any new features.
For any questions or issues, please open an issue on the GitHub repository or contact Siddhesh Agarwal.