ANTHROPIC - RAG
Simon-Pierre Boucher
2024-09-14
This script is designed to integrate document retrieval into a conversation-based system using Anthropic's API. Here’s a breakdown of the key parts and functionality:
1. Environment Setup¶
- The script loads environment variables, such as the Anthropic API key, from a
.envfile usingdotenv.
2. Document Retrieval: retrieve_relevant_docs()¶
- This function performs a simple keyword-based search through a list of documents and retrieves those that contain keywords from the user’s query.
- It checks for keyword matches in a case-insensitive manner by splitting the user query and comparing it against each document.
3. Main API Call with Retrieval Augmented Generation (RAG): make_anthropic_api_call_with_rag()¶
- This function combines the conversation history, current user message, and relevant documents retrieved from the previous function.
- It augments the user's query by adding relevant document content as a context to improve the quality of the generated response.
- The combined message (user query and context) is sent to the Anthropic API using the model specified (
claude-3-5-sonnet-20240620by default). - The API request includes parameters like
max_tokens, the model version, and the conversation history. - The response is returned as a JSON object.
4. Formatting the Response: format_markdown()¶
- This function takes the API response content in markdown format and converts it into HTML for better display in a Jupyter notebook.
- It processes common markdown elements like bold text, italicized text, and headers, converting them into the corresponding HTML tags.
5. Displaying the API Response: display_api_response()¶
- This function processes the response returned by the API and extracts relevant information like:
- Model information: The model used in the API call.
- Token usage: How many tokens were used for the input and output.
- Response content: The main content generated by the assistant.
- Additional metadata: Information such as the ID of the API call and the stop reason.
- The information is displayed in an HTML format using
IPython.display.
6. Example Usage¶
- The conversation starts with an empty history.
- The user query,
"What is the debt-to-equity ratio of ABC Corp?", is provided. - A list of relevant documents (pertaining to ABC Corp.) is searched, and relevant documents are combined with the query to enhance the response.
- The response from the Anthropic API is retrieved and displayed using
display_api_response().
Example Flow:¶
- Query:
"What is the debt-to-equity ratio of ABC Corp?" - Document Retrieval: The script finds the document that states:
"ABC Corp. has a current debt-to-equity ratio of 0.3...".
- API Augmentation: This document is added to the user’s query as context, making the query more detailed and focused for the API.
- API Call: The combined query is sent to the Anthropic API, and the response is formatted for display.
Summary:¶
- Retrieval-Augmented Generation (RAG): Relevant documents are fetched and included in the conversation context to enhance the quality of the generated responses.
- Markdown-to-HTML Conversion: The assistant's response is converted from markdown to HTML for display purposes.
- Error Handling: The script checks for potential API errors and handles them by printing the error message.
This approach ensures that the assistant's response is contextually aware of relevant information from the provided documents, making the answers more precise.
In [3]:
import os
import requests
from dotenv import load_dotenv
from IPython.display import display, HTML
import re
# Load environment variables from the .env file
load_dotenv()
# Get the API key from environment variables
api_key = os.getenv("ANTHROPIC_API_KEY")
def retrieve_relevant_docs(query, documents):
"""
Simple keyword-based function to retrieve relevant documents.
"""
relevant_docs = []
for doc in documents:
if any(keyword.lower() in doc.lower() for keyword in query.split()):
relevant_docs.append(doc)
return relevant_docs
def make_anthropic_api_call_with_rag(conversation_history, current_message, documents, model="claude-3-5-sonnet-20240620", max_tokens=1024, system_message="You are a helpful assistant."):
"""
Makes a call to the Anthropic API using the provided conversation history and current message
with retrieved relevant documents.
:param conversation_history: List of conversation history messages
:param current_message: Current user message
:param documents: List of available documents for retrieval
:param model: Anthropic model to use
:param max_tokens: Maximum number of tokens in the response
:param system_message: System message defining the assistant's role
:return: JSON response from the Anthropic API
"""
# Retrieve relevant documents based on the current message
relevant_docs = retrieve_relevant_docs(current_message, documents)
# Combine the retrieved documents into a single context
context = "\n\n".join(relevant_docs)
# Add the retrieved context to the current message
augmented_message = f"Context: {context}\n\n{current_message}"
# Check if the last message in the history is already a "user"
if conversation_history and conversation_history[-1]['role'] == "user":
conversation_history.append({"role": "assistant", "content": ""})
# Add the augmented current message as a "user" message
messages = conversation_history + [{"role": "user", "content": augmented_message}]
url = 'https://api.anthropic.com/v1/messages'
headers = {
'x-api-key': api_key,
'anthropic-version': '2023-06-01',
'Content-Type': 'application/json'
}
data = {
"model": model,
"max_tokens": max_tokens,
"system": system_message,
"messages": messages
}
response = requests.post(url, headers=headers, json=data)
return response.json()
def format_markdown(content):
"""
Converts Markdown content to HTML.
:param content: Markdown text
:return: HTML text
"""
# Remove unnecessary line breaks after enumerations
content = re.sub(r'(\d+\..*?)\n\n', r'\1\n', content)
# Convert Markdown to HTML
content = content.replace('\n', '<br>')
content = re.sub(r'\*\*(.*?)\*\*', r'<strong>\1</strong>', content)
content = re.sub(r'\*(.*?)\*', r'<em>\1</em>', content)
content = re.sub(r'### (.*)', r'<h3>\1</h3>', content)
content = re.sub(r'## (.*)', r'<h2>\1</h2>', content)
content = re.sub(r'# (.*)', r'<h1>\1</h1>', content)
return content
def display_api_response(response):
"""
Formats the JSON response from the Anthropic API for HTML display.
:param response: JSON response from the Anthropic API
:return: None
"""
# Check if the response contains errors
if 'error' in response:
print(f"Error: {response['error']['message']}")
return
# Check if the 'content' key exists in the response
if 'content' not in response:
print("Error: The response does not contain 'content'.")
print("Full response:", response)
return
# Extract the role and content from the response
role = response.get('role', 'N/A') # Safely extract the role
content_list = response.get('content', [])
if content_list and isinstance(content_list, list):
text_content = "\n".join([item.get('text', '') for item in content_list])
else:
text_content = "No content available."
usage = response.get('usage', {})
# Format the content with Markdown
formatted_content = format_markdown(text_content)
html = """
<div class="api-response">
"""
# Model Information
html += f"""
<div class="bubble">
<h3>Model Information</h3>
<p><strong>Model:</strong> {response.get('model', 'N/A')}</p>
</div>
"""
# Token Usage
html += f"""
<div class="bubble">
<h3>Token Usage</h3>
<p><strong>Input Tokens:</strong> {usage.get('input_tokens', 'N/A')}</p>
<p><strong>Output Tokens:</strong> {usage.get('output_tokens', 'N/A')}</p>
</div>
"""
# Response Content
html += f"""
<div class="bubble">
<h3>Response Content</h3>
<p><strong>Role:</strong> {role}</p>
<p><strong>Content:</strong></p>
<div>{formatted_content}</div>
</div>
"""
# Additional Metadata
html += f"""
<div class="bubble">
<h3>Additional Metadata</h3>
<p><strong>ID:</strong> {response.get('id', 'N/A')}</p>
<p><strong>Type:</strong> {response.get('type', 'N/A')}</p>
<p><strong>Stop Reason:</strong> {response.get('stop_reason', 'N/A')}</p>
</div>
"""
html += "</div>"
display(HTML(html))
In [2]:
conversation_history = []
current_message = "What is the debt-to-equity ratio of ABC Corp?"
documents = [
"ABC Corp. reported a revenue of 50 million for Q2 2024, a 10 percent increase from Q1 2024. The company's net income for the quarter was 5 million, reflecting a 5 percent profit margin.",
"ABC Corp. has a current debt-to-equity ratio of 0.3, indicating that the company has a low level of debt compared to its equity.",
"The market capitalization of ABC Corp. is currently 300 million, based on a share price of 30 and 10 million shares outstanding.",
"In Q2 2024, ABC Corp. announced a dividend of 0.50 per share, which will be distributed to shareholders on October 1, 2024.",
"ABC Corp.'s gross profit margin for Q2 2024 was 40 percent, reflecting strong control over cost of goods sold and efficient operations."
]
In [3]:
response = make_anthropic_api_call_with_rag(conversation_history, current_message, documents,model="claude-3-5-sonnet-20240620")
display_api_response(response)
In [4]:
response = make_anthropic_api_call_with_rag(conversation_history, current_message, documents,model="claude-3-sonnet-20240229")
display_api_response(response)
In [5]:
response = make_anthropic_api_call_with_rag(conversation_history, current_message, documents,model="claude-3-opus-20240229")
display_api_response(response)
In [6]:
response = make_anthropic_api_call_with_rag(conversation_history, current_message, documents,model="claude-3-haiku-20240307")
display_api_response(response)