Skip to content

Latest commit

 

History

History
209 lines (162 loc) · 12.5 KB

README.md

File metadata and controls

209 lines (162 loc) · 12.5 KB
Raw

Mistral AI

Version

maintained - yes contributions - welcome

Mistral AI

Overview

The Mistral AI Toolkit makes it easy to use Mistral's open models Mistral-7b and Mixtral-8x7b along with their flagship suite of models Mistral (tiny, small, medium & large) and their latest model in collaboration with NVIDIA Mistral NeMo for creating chatbots and generating contextually relevant text based on prompts. It's designed for everyone, from beginners to experienced developers, allowing quick addition of AI features to projects with simple commands. While it offers simplicity and lightweight integration, it doesn't compromise on power; experienced developers can access the full suite of advanced options available via the API, ensuring robust customization and control. This toolkit is perfect for those looking to efficiently tap into advanced AI without getting bogged down in technical details, yet it still provides the depth needed for complex project requirements.

Key Features

  • Conversational AI: Create interactive, real-time chat experiences (chatbots) or AI assistants.
  • Text Generation: Produce coherent and contextually relevant text and answers from simple prompts.
  • Highly Customizable: Tailor settings like streaming, JSON outputs, system prompts and more to suit your specific requirements.
  • Lightweight Integration: Efficiently designed with minimal dependencies, requiring only the requests package for core functionality.

Prerequisites

  • Python 3.x
  • An API key from Mistral AI

Dependencies

The following Python packages are required:

  • requests: For making HTTP requests to Mistral's API.

The following Python packages are optional:

  • python-dotenv: For managing API keys and other environment variables.

Installation

To use the Mistral AI Toolkit, clone the repository to your local machine and install the required Python packages.

Clone the repository:

git clone https://github.com/RMNCLDYO/mistral-ai-toolkit.git

Navigate to the repositories folder:

cd mistral-ai-toolkit

Install the required dependencies:

pip install -r requirements.txt

Configuration

  1. Obtain an API key from Mistral AI.

  2. You have three options for managing your API key:

    Click here to view the API key configuration options

    • Setting it as an environment variable on your device (recommended for everyday use)

      • Navigate to your terminal.
      • Add your API key like so: 
        export MISTRAL_API_KEY=your_api_key

      This method allows the API key to be loaded automatically when using the wrapper or CLI.

    • Using an .env file (recommended for development):

      • Install python-dotenv if you haven't already: pip install python-dotenv.
      • Create a .env file in the project's root directory.
      • Add your API key to the .env file like so: 
        MISTRAL_API_KEY=your_api_key

      This method allows the API key to be loaded automatically when using the wrapper or CLI, assuming you have python-dotenv installed and set up correctly.

    • Direct Input:

      • If you prefer not to use a .env file, you can directly pass your API key as an argument to the CLI or the wrapper functions.

        CLI

        --api_key "your_api_key"

        Wrapper

        api_key="your_api_key"

      This method requires manually inputting your API key each time you initiate an API call, ensuring flexibility for different deployment environments.

Usage

The Mistral AI Toolkit can be used in two different modes: Chat and Text. Each mode is designed for specific types of interactions with Mistral's open models.

Chat Mode

Chat mode is intended for chatting with an AI model (similar to a chatbot) or building conversational applications.

Example Usage

CLI

python cli.py --chat

Wrapper

from mistral import Chat

Chat().run()

An executable version of this example can be found here. (You must move this file to the root folder before running the program.)

Text Mode

Text mode is suitable for generating text content based on a provided prompt.

Example Usage

CLI

python cli.py --text --prompt "Which one is heavier, a pound of iron or a kilogram of feathers?"

Wrapper

from mistral import Text

Text().run(prompt="Which one is heavier a pound of iron or a kilogram of feathers?")

An executable version of this example can be found here. (You must move this file to the root folder before running the program.)

Advanced Configuration

CLI and Wrapper Options

Description CLI Flags CLI Usage Wrapper Usage
Enable chat mode -c, --chat --chat See mode usage above.
Enable text mode -t, --text --text See mode usage above.
API key for authentication -a, --api_key --api_key "your_api_key" api_key="your_api_key"
Model name -m, --model --model "mistral-large-latest" model="mistral-large-latest"
User prompt -p, --prompt --prompt "Hello, how are you today?" prompt="Hello, how are you today?"
Enable streaming mode -s, --stream --stream stream=True
Enable json mode -js, --json --json json=True
System prompt (instructions) -sp, --system_prompt --system_prompt "You are an advanced AI assistant" system_prompt="You are an advanced AI assistant"
Maximum tokens to generate -mt, --max_tokens --max_tokens 1024 max_tokens=1024
Sampling temperature -tm, --temperature --temperature 0.7 temperature=0.7
Nucleus sampling threshold -tp, --top_p --top_p 0.9 top_p=0.9
Random seed for sampling -rs, --random_seed --random_seed 42 random_seed=42
Enable safe prompt mode -sf, --safe_prompt --safe_prompt safe_prompt=True

To exit the program at any time, you can type exit or quit. This command works similarly whether you're interacting with the program via the CLI or through the Python wrapper ensuring that you can easily and safely conclude your work with the Mistral AI Toolkit without having to resort to interrupt signals or forcibly closing the terminal or command prompt.

Available Models

Model Max Tokens
open-mistral-nemo 128000
open-mistral-7b 32000
open-mixtral-8x7b 32000
open-mixtral-8x22b 64000
mistral-small-latest 32000
mistral-medium-latest (set to be deprecated in the coming months) 32000
mistral-large-latest 32000

Contributing

Contributions are welcome!

Please refer to CONTRIBUTING.md for detailed guidelines on how to contribute to this project.

Reporting Issues

Encountered a bug? We'd love to hear about it. Please follow these steps to report any issues:

  1. Check if the issue has already been reported.
  2. Use the Bug Report template to create a detailed report.
  3. Submit the report here.

Your report will help us make the project better for everyone.

Feature Requests

Got an idea for a new feature? Feel free to suggest it. Here's how:

  1. Check if the feature has already been suggested or implemented.
  2. Use the Feature Request template to create a detailed request.
  3. Submit the request here.

Your suggestions for improvements are always welcome.

Versioning and Changelog

Stay up-to-date with the latest changes and improvements in each version:

  • CHANGELOG.md provides detailed descriptions of each release.

Security

Your security is important to us. If you discover a security vulnerability, please follow our responsible disclosure guidelines found in SECURITY.md. Please refrain from disclosing any vulnerabilities publicly until said vulnerability has been reported and addressed.

License

Licensed under the MIT License. See LICENSE for details.