MISTRAL - TEXT GENERATION
Simon-Pierre Boucher
2024-09-14
Here's a summary of what the Python code is doing:
Environment Setup:
- The code loads environment variables from a
.envfile usingload_dotenv(). It then retrieves the Mistral API key from these environment variables (MISTRAL_API_KEY).
- The code loads environment variables from a
Mistral API Text Generation Function:
- The function
generate_mistral_text()is designed to send a request to the Mistral AI API to generate text. - Parameters such as
model,messages,temperature,top_p,max_tokens, etc., allow customization of the request (e.g., model choice, randomness of output, and token limits). - The API key is passed via a header, and the request body (
data) contains the text generation parameters, such as the input messages and various tuning options. - If optional parameters (
max_tokens,stop, etc.) are provided, they are included in the request. - The function handles any API errors with exception handling and returns the response as a JSON dictionary.
- The function
Response Formatting:
- The
format_mistral_response()function extracts the relevant part of the API response, specifically the assistant's reply, and formats it using Markdown for clearer output (e.g., the assistant's message is bolded).
- The
Example Use:
- The code sets up a specific
model(mistral-small-latest) and a user message asking about quantum entanglement. - It sends the message to the Mistral API using the
generate_mistral_text()function with parameters such astemperatureandmax_tokens. - The response is formatted using the
format_mistral_response()function, and the output is printed.
- The code sets up a specific
This code demonstrates how to interact with the Mistral API for text generation in a conversational context.
In [2]:
import os
import requests
from dotenv import load_dotenv
from IPython.display import display, HTML
import re
# Charger les variables d'environnement depuis le fichier .env
load_dotenv()
# Obtenir la clé API depuis les variables d'environnement
api_key = os.getenv("MISTRAL_API_KEY")
In [3]:
import requests
def generate_mistral_text(api_key, model, messages, temperature=0.7, top_p=1.0, max_tokens=None, min_tokens=None, stream=False, stop=None, random_seed=None, response_format=None, tools=None, tool_choice="auto", safe_prompt=False):
"""
Generate text using Mistral's API.
Parameters:
- api_key (str): The API key for Mistral.
- model (str): The model to use for text generation.
- messages (list): A list of messages to pass to the API in a conversation format.
- temperature (float): Controls randomness in the output (0-1.5).
- top_p (float): Nucleus sampling (0-1).
- max_tokens (int): The maximum number of tokens to generate in the completion.
- min_tokens (int): The minimum number of tokens to generate in the completion.
- stream (bool): Whether to stream back partial progress.
- stop (str or list): Stop generation if this token or one of these tokens is detected.
- random_seed (int): The seed to use for random sampling.
- response_format (dict): The format of the response.
- tools (list): A list of tools to use.
- tool_choice (str): Tool choice for the response ("auto", "none", "any").
- safe_prompt (bool): Whether to inject a safety prompt before all conversations.
Returns:
- response (dict): The API response as a dictionary.
"""
url = "https://api.mistral.ai/v1/chat/completions"
headers = {
"Content-Type": "application/json",
"Accept": "application/json",
"Authorization": f"Bearer {api_key}"
}
data = {
"model": model,
"messages": messages,
"temperature": temperature,
"top_p": top_p,
"stream": stream,
"tool_choice": tool_choice,
"safe_prompt": safe_prompt
}
# Optional parameters
if max_tokens is not None:
data["max_tokens"] = max_tokens
if min_tokens is not None:
data["min_tokens"] = min_tokens
if stop is not None:
data["stop"] = stop
if random_seed is not None:
data["random_seed"] = random_seed
if response_format is not None:
data["response_format"] = response_format
if tools is not None:
data["tools"] = tools
try:
response = requests.post(url, headers=headers, json=data)
response.raise_for_status()
return response.json()
except requests.exceptions.RequestException as e:
print(f"An error occurred: {e}")
return None
In [4]:
def format_mistral_response(response):
"""
Formats the response from Mistral API to display only the assistant's message.
Parameters:
- response (dict): The API response as a dictionary.
Returns:
- formatted_text (str): A formatted string with Markdown for the assistant's message.
"""
if response and "choices" in response:
# Extract the assistant's message from the response
assistant_message = response.get("choices", [])[0].get("message", {}).get("content", "No reply found.")
formatted_text = f"**Assistant:**\n\n{assistant_message}\n"
return formatted_text
else:
return "No valid response received."
In [5]:
model = "mistral-small-latest"
messages = [
{"role": "user", "content": "Explain the concept of quantum entanglement and how it challenges classical notions of locality and realism. What are the implications of entanglement for our understanding of causality and information transfer?"}
]
response = generate_mistral_text(api_key, model, messages, temperature=0.7, max_tokens=2000, top_p=0.9)
formatted_response = format_mistral_response(response)
print(formatted_response)
In [6]:
model = "mistral-medium-latest"
messages = [
{"role": "user", "content": "Explain the concept of quantum entanglement and how it challenges classical notions of locality and realism. What are the implications of entanglement for our understanding of causality and information transfer?"}
]
response = generate_mistral_text(api_key, model, messages, temperature=0.7, max_tokens=2000, top_p=0.9)
formatted_response = format_mistral_response(response)
print(formatted_response)
In [7]:
model = "mistral-large-latest"
messages = [
{"role": "user", "content": "Explain the concept of quantum entanglement and how it challenges classical notions of locality and realism. What are the implications of entanglement for our understanding of causality and information transfer?"}
]
response = generate_mistral_text(api_key, model, messages, temperature=0.7, max_tokens=2000, top_p=0.9)
formatted_response = format_mistral_response(response)
print(formatted_response)
In [8]:
model = "open-mistral-7b"
messages = [
{"role": "user", "content": "Explain the concept of quantum entanglement and how it challenges classical notions of locality and realism. What are the implications of entanglement for our understanding of causality and information transfer?"}
]
response = generate_mistral_text(api_key, model, messages, temperature=0.7, max_tokens=2000, top_p=0.9)
formatted_response = format_mistral_response(response)
print(formatted_response)
In [9]:
model = "open-mixtral-8x7b"
messages = [
{"role": "user", "content": "Explain the concept of quantum entanglement and how it challenges classical notions of locality and realism. What are the implications of entanglement for our understanding of causality and information transfer?"}
]
response = generate_mistral_text(api_key, model, messages, temperature=0.7, max_tokens=2000, top_p=0.9)
formatted_response = format_mistral_response(response)
print(formatted_response)
In [10]:
model = "open-mixtral-8x22b"
messages = [
{"role": "user", "content": "Explain the concept of quantum entanglement and how it challenges classical notions of locality and realism. What are the implications of entanglement for our understanding of causality and information transfer?"}
]
response = generate_mistral_text(api_key, model, messages, temperature=0.7, max_tokens=2000, top_p=0.9)
formatted_response = format_mistral_response(response)
print(formatted_response)