@@ -29,7 +29,6 @@ If you have ideas for new examples or guides, share them on the [issues page](ht
## Table of recipes
### Skills
- [Citations](https://github.com/anthropics/anthropic-cookbook/tree/main/skills/citations): Learn how to prompt Claude to cite sources in its responses.
- [Classification](https://github.com/anthropics/anthropic-cookbook/tree/main/skills/classification): Explore techniques for text and data classification using Claude.
- [Retrieval Augmented Generation](https://github.com/anthropics/anthropic-cookbook/tree/main/skills/retrieval_augmented_generation): Learn how to enhance Claude's responses with external knowledge.
- [Summarization](https://github.com/anthropics/anthropic-cookbook/tree/main/skills/summarization): Discover techniques for effective text summarization with Claude.
@@ -10,8 +10,6 @@ Welcome to the Skills section of the Anthropic Cookbook! This directory contains
- **[Retrieval Augmented Generation with Contextual Embeddings](./contextual-embeddings/guide.ipynb)**: Learn how to use a new technique to improve the performance of your RAG system. In traditional RAG, documents are typically split into smaller chunks for efficient retrieval. While this approach works well for many applications, it can lead to problems when individual chunks lack sufficient context. Contextual Embeddings solve this problem by adding relevant context to each chunk before embedding. You'll learn how to use contextual embeddings with semantic search, BM25 search, and reranking to improve performance.
- **[Citations with Claude](./citations/guide.ipynb)**: Learn how to leverage Claude for accurate and verifiable information retrieval. This guide demonstrates how to implement citations in two key scenarios: Q&A over a help center and extracting quotes from large documents. You'll explore prompt engineering techniques, post-processing methods, and strategies for creating user-friendly, interactive experiences with cited information.
- **[Summarization with Claude](./summarization/guide.ipynb)**: Explore Claude's ability to summarize and synthesize information from multiple sources. This guide covers a variety of summarization techniques, including multi-shot, domain-based, and chunking methods, as well as strategies for handling long-form content and multiple documents. We also explore evaluating summaries, which can be a balance of art, subjectivity, and the right approach!
- **[Text-to-SQL with Claude](./text_to_sql/guide.ipynb)**: This guide covers how to generate complex SQL queries from natural language using prompting techniques, self-improvement, and RAG. We'll also explore how to evaluate and improve the accuracy of generated SQL queries, with evals that test for syntax, data correctness, row count, and more.
To use Promptfoo you will need to have node.js & npm installed on your system. For more information follow [this guide](https://docs.npmjs.com/downloading-and-installing-node-js-and-npm)
You can install promptfoo using npm or run it directly using npx. In this guide we will use npx.
*Note: For this example you will not need to run `npx promptfoo@latest init` there is already an initialized `promptfooconfig.yaml` file in this directory*
See the official docs [here](https://www.promptfoo.dev/docs/getting-started)
### Getting Started
The evaluation is orchestrated by the `promptfooconfig.yaml` file. In this file we define the following sections:
- Prompts
- We are testing a single prompt which is contructed in `prompt.py`
- Providers
- In this example we will evaluate our prompt against three different Claude models
- Tests
- Our tests are defined in `dataset.csv`. We have over a dozen (question, expected citation) pairs. Questions that should not result in a citation are expected to have the value -1
- Transform
- In the `defaultTest` section we define a transform function. This is a python function which extracts the citation from the output string. If no citation is found we return -1 instead.
### Run the eval
To get started with Promptfoo open your terminal and navigate to this directory (`./evaluation`).
Before running your evaluation you must define the following enviroment variables:
`export ANTHROPIC_API_KEY=YOUR_API_KEY`
From the `evaluation` directory, run the following command.
`npx promptfoo@latest eval`
If you would like to increase the concurrency of the requests (default = 4), run the following command.
`npx promptfoo@latest eval -j 25`
When the evaluation is complete the terminal will print the results for each row in the dataset.
textwrap.dedent("""You will be acting as a conversational AI customer support assistant for the ecommerce website PetWorld. Your goal is to help answer customer questions in a friendly and helpful manner, using PetWorld's help center articles as your knowledge base.
Here are the help center articles you have available, provided in <article> tags with unique IDs:
<help_center_articles>
{HELP_CENTER_ARTICLES}
</help_center_articles>
And here is the user's question, provided in a <user_question> tag:
<user_question>
{USER_QUESTION}
</user_question>
To formulate your response, follow these steps:
1. Carefully read the user's question to understand what they are asking about.
2. Search through the provided help center articles to find the most relevant information to answer the question. Focus on finding an article that directly addresses the user's specific question.
3. If you find a relevant article, use the information in it to write a friendly response that fully answers the user's question. Aim to provide a complete answer using only information from the help center articles.
4. At the end of your response, include a citation like this - [Article ID] - where "Article ID" is replaced by the ID number of the help center article you used to answer the question.
5. If after searching the help center articles you determine that none of them contain the information needed to answer the user's question, simply respond with "I'm afraid I don't know the answer to that question. Let me know if there is anything else I can assist with!"
Here is an example of what a good response looks like:
<user_question>What is your return policy on dog food?</user_question>
<answer>At PetWorld, we offer a 30 day return window on all dog food purchases. You can return the unused portion for a full refund within 30 days of purchase. We also offer a 100% satisfaction guarantee - if your dog doesn't love their food, we'll give you your money back! Let me know if you have any other questions. [1]</answer>
Now it's your turn! Please provide your response to the user's question inside <answer> tags. Remember - only use information from the provided help center articles, and if you can't find the answer there, let the user know you don't have that information. Always aim to be friendly and helpful in your tone.""")
"To complete the following guide you will need to install the `anthropic` package and obtain and Anthropic API key. "
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"!pip install anthropic pypdf"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [],
"source": [
"import anthropic\n",
"import os\n",
"\n",
"client = anthropic.Anthropic(\n",
" api_key='' # YOUR API KEY HERE,\n",
")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Overview\n",
"\n",
"Large Language Models (LLMs) like Claude have demonstrated remarkable capabilities in understanding and generating human-like text across a wide range of applications. However, one of the challenges with LLMs is ensuring the accuracy and explainability of the information they provide. This is where citations become invaluable.\n",
"\n",
"Citations are references to specific sources of information that support or substantiate the claims or statements made by the LLM. By incorporating citations into Claude's responses, we can significantly enhance the transparency and usefulness of the Claude-generated content.\n",
"\n",
"### The Value of Citations\n",
"\n",
"**Verifiability and User Affordance**: Citations provide clickable links or clear references to source material, allowing end users to verify information independently and explore topics further. This enhances credibility and empowers users to access additional context easily.\n",
"\n",
"**Explainability**:\n",
"By linking responses to specific sources, citations make Claude's reasoning process more transparent. This aids in building trust and provides valuable metadata for analysis and quality control. For instance, in a voice-based AI assistant, it might not make sense to verbalize the citations. Instead, they could be logged separately for analysis or quality assurance purposes.\n",
"\n",
"### Practical Applications\n",
"\n",
"In this guide, we will explore how to effectively use citations with Claude in two specific Q&A scenarios:\n",
"\n",
"1. Answering queries over a help center knowledge base\n",
"2. Extracting information from call transcripts\n",
"\n",
"Through these examples, you'll learn how to prompt Claude to provide citations, how to process and present these citations, and how to evaluate them to enhance the overall quality and trustworthiness of your AI-powered applications.\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Use case: Q&A against customer help center\n",
"\n",
"Question-and-Answer (Q&A) systems built on top of customer help centers are a popular application of Claude. These systems allow customers to quickly find answers to their questions without having to manually search through multiple help articles. Importantly, these systems often require citations, as they provide direct links to the relevant help articles, allowing users to verify the information and dive deeper into topics if needed.\n",
"\n",
"\n",
"For our example, let's introduce \"PetWorld\", a fictional e-commerce company that specializes in pet supplies and accessories. PetWorld has a wide range of products for various pets, from dogs and cats to reptiles and small mammals. They've been growing rapidly and want to improve their customer support by implementing an AI-powered chatbot that can answer customer queries by referencing their extensive online help center.\n",
"\n",
"\n",
"PetWorld's help center covers a variety of topics, including:\n",
"\n",
"- Product information and care instructions\n",
"- Shipping and delivery policies\n",
"- Returns and exchanges\n",
"- Account management\n",
"- Pet care tips and advice\n",
"\n",
"By implementing a citation-based Q&A system, PetWorld aims to:\n",
"\n",
"- Reduce the workload on their customer service team\n",
"- Provide 24/7 support to their customers\n",
"- Ensure customers can easily find and verify information in their help center\n",
"- Improve customer satisfaction by providing quick, accurate answers with links to further reading\n",
"\n",
"In the following sections, we'll explore how to build such a system using Claude, enabling PetWorld to create a more efficient and user-friendly customer support experience."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Help Center\n",
"\n",
"In this example we provide a ficitonal help center comprising of 10 articles. Each article includes a title and the body of text. We will use these articles to build and test a prompt which is able to answer user questions based on these articles and cite the response.\n",
"\n",
"Note: In this example we will not explore retrieving specific articles from the help center. However, in many partical applications a help center may consist of dozens or hundreds of articles. Often times this will require implementing more complex retrieval (RAG) which will be covered in a separate skills cookbook.\n",
"\n",
"All articles can be within in this cookbook under `data/help_center_articles`"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"(0.txt) title: How to Change Your Password\n",
"(1.txt) title: 30-Day Return Policy\n",
"(2.txt) title: How to Change Delivery Address\n",
"In this example we will create a prompt which takes a **user question** and the **help center articles** as inputs and return a helpful response. In this prompt template we apply several best practices including assigning a role and providing an example. Review the Anthropic [prompt engineering documentation](https://docs.anthropic.com/en/docs/prompt-engineering) to learn more.\n"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [],
"source": [
"sysyem_prompt = \"You will be acting as a conversational AI customer support assistant for the ecommerce website PetWorld. Your goal is to help answer customer questions in a friendly and helpful manner, using PetWorld's help center articles as your knowledge base.\"\n",
"\n",
"prompt_template = '''\n",
"Here are the help center articles you have available, provided in <article> tags with unique IDs:\n",
"\n",
"<help_center_articles>\n",
"{HELP_CENTER_ARTICLES}\n",
"</help_center_articles>\n",
"\n",
"And here is the user's question, provided in a <user_question> tag:\n",
"\n",
"<user_question>\n",
"{USER_QUESTION}\n",
"</user_question>\n",
"\n",
"To formulate your response, follow these steps:\n",
"\n",
"1. Carefully read the user's question to understand what they are asking about. \n",
"2. Search through the provided help center articles to find the most relevant information to answer the question. Focus on finding an article that directly addresses the user's specific question.\n",
"3. If you find a relevant article, use the information in it to write a friendly response that fully answers the user's question. Aim to provide a complete answer using only information from the help center articles.\n",
"4. At the end of your response, include a citation like this - [Article ID] - where \"Article ID\" is replaced by the ID number of the help center article you used to answer the question.\n",
"5. If after searching the help center articles you determine that none of them contain the information needed to answer the user's question, simply respond with \"I'm afraid I don't know the answer to that question. Let me know if there is anything else I can assist with!\"\n",
"\n",
"Here is an example of what a good response looks like:\n",
"\n",
"<user_question>What is your return policy on dog food?</user_question>\n",
"\n",
"<answer>At PetWorld, we offer a 30 day return window on all dog food purchases. You can return the unused portion for a full refund within 30 days of purchase. We also offer a 100% satisfaction guarantee - if your dog doesn't love their food, we'll give you your money back! Let me know if you have any other questions. [1]</answer>\n",
"\n",
"Now it's your turn! Please provide your response to the user's question inside <answer> tags. Remember - only use information from the provided help center articles, and if you can't find the answer there, let the user know you don't have that information. Always aim to be friendly and helpful in your tone.'''"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Next we will apply preprocessing to the help center articles"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"<article id=\"0\">\n",
"<title>How to Change Your Password</title>\n",
"<content>To change your password, log in to your PetWorld account and navigate to the \"Account Settings\" page. Click on the \"Security\" tab, then select \"Change Password.\" Enter your current password, then type your new password twice to confirm. Make sure your new password is at least 8 characters long and includes a mix of uppercase and lowercase letters, numbers, and symbols. Click \"Save Changes\" to update your password. For security reasons, you'll be logged out and need to sign in again with your new password.</content>\n",
"</article>\n",
"<article id=\"1\">\n",
"<title>30-Day Return Policy</title>\n",
"<content>PetWorld offers a 30-day return policy on most items. If you're not satisfied with your purchase, you can return it within 30 days of the delivery date for a full refund or exchange. The item must be unused, in its original packaging, and in resalable condition. To initiate a return, log in to your account, go to \"Order History,\" selec\n",
"...\n",
"kages, and access exclusive deals. To update your account information, log in and go to \"Account Settings.\" Here, you can change your contact details, manage saved addresses and payment methods, and set your communication preferences. If you need to close your account, please contact our customer support team.</content>\n",
"</article>\n",
"<article id=\"9\">\n",
"<title>Pet Insurance Overview</title>\n",
"<content>While PetWorld doesn't directly offer pet insurance, we've partnered with leading providers to offer our customers exclusive discounts. Pet insurance can help cover unexpected veterinary costs for illnesses, injuries, and in some cases, routine care. Coverage and costs vary by provider and plan. To explore pet insurance options, log in to your PetWorld account and visit the \"Pet Insurance\" section. There, you'll find information about our partner providers and can request quotes. Remember to carefully review policy details and exclusions before purchasing any insurance plan.</content>\n",
"</article>\n",
"\n"
]
}
],
"source": [
"def get_articles_as_string():\n",
" # Get all .txt files in the directory and sort them\n",
" filenames = sorted([f for f in os.listdir(articles_dir) if f.endswith('.txt')])\n",
"Finally, we will call Claude 3.5 Sonnet with the prompt template, our formatted help center articles, and the user question."
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"I'll be happy to help you change your delivery address! To update an address for an order, here's what you need to do:\n",
"\n",
"1. Log in to your PetWorld account\n",
"2. Go to \"Order History\"\n",
"3. Find the order you want to modify\n",
"4. Click \"Change Shipping Address\"\n",
"5. Enter your new address and save the changes\n",
"\n",
"Please note that you can only change the address if your order hasn't shipped yet. If your order is already in transit, you'll need to contact our customer support team for help.\n",
"\n",
"For future orders, you can also add or edit addresses in your account settings under \"Saved Addresses\" - this way you'll have them ready to use for your next purchase!\n",
"\n",
"Is there anything else I can help you with? [2]\n"
]
}
],
"source": [
"def answer_question(user_question):\n",
" # Generate the prompt with the user question and help center articles\n",
"raw_output = answer_question('Confused how to change my address')\n",
"print(raw_output)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Post processing and running\n",
"\n",
"When working with citations, it's often beneficial to apply post-processing techniques to the output. One useful approach is to add URLs and other metadata to the citations after Claude has generated the response. This method offers several advantages:\n",
"\n",
"- It saves on output tokens, as Claude doesn't need to generate full URLs.\n",
"- It limits the possibility of errors in URL generation.\n",
"- It provides flexibility in how citations are presented to the end-user.\n",
"\n",
"In our PetWorld example, let's say each article ID maps to a specific URL on their help center website. For instance:\n",
"\n",
"- `0.txt` maps to `https://help.petworld.com/article/0`\n",
"- `1.txt` maps to `https://help.petworld.com/article/1`\n",
"- And so on...\n",
"\n",
"We can apply a post-processing step to Claude's output to turn these citations into clickable hyperlinks. Here's a Python function that could accomplish this:"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<pre>\n",
"I'll be happy to help you change your delivery address! To update an address for an order, here's what you need to do:\n",
"\n",
"1. Log in to your PetWorld account\n",
"2. Go to \"Order History\"\n",
"3. Find the order you want to modify\n",
"4. Click \"Change Shipping Address\"\n",
"5. Enter your new address and save the changes\n",
"\n",
"Please note that you can only change the address if your order hasn't shipped yet. If your order is already in transit, you'll need to contact our customer support team for help.\n",
"\n",
"For future orders, you can also add or edit addresses in your account settings under \"Saved Addresses\" - this way you'll have them ready to use for your next purchase!\n",
"\n",
"Is there anything else I can help you with? <a href=\"https://help.petworld.com/article/2\">[2]</a></pre>"
"Now that we have a working prompt end-to-end we can measure and evaluate Claude's ability to correct cite sources in this task. This is separate from evaluating the answer quality itself which will be covered in a separate cookbook.\n",
"\n",
"To evaluate citations you will need to create a golden set of (question, article ID) pairs. You will be evaluating that given a user question Claude is citing the expected article ID(s). In the evaluation we also test questions that are not answered by the help center and confirm the response does not include a citation.\n",
"\n",
"We will use `promptfoo` for this portion. Head over to `evaluation/README.md` to get started.\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Use case: Q&A over a Large Document\n",
"\n",
"In our previous example, we explored how to prompt Claude for Q&A over a help center, providing citations at the article level. This approach is ideal when dealing with numerous short articles, as it allows end users to easily read entire referenced articles.\n",
"\n",
"However, when working with larger documents, a different citation strategy becomes necessary. For instance, if you're dealing with a 40-page document, a citation referencing the entire document provides little value to the user. In such cases, it's more beneficial to cite specific quotes or passages that directly inform the answer.\n",
"\n",
"In this section, we'll develop a prompt that enables Claude to perform Q&A over a large document while extracting and citing relevant quotes. This approach offers several advantages:\n",
"\n",
"- Precision: It pinpoints the exact information used to formulate the answer.\n",
"- Verifiability: Users can quickly check the source of specific claims.\n",
"- Context: It provides users with the surrounding context of the information.\n",
"- Efficiency: It saves users time by highlighting the most relevant parts of a large document."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Preprocessing\n",
"\n",
"In this example we will work with a large PDF, \"Constitutional AI: Harmlessness from AI Feedback\" published by Anthropic. This document is 34 pages. Since the API does not natively ingest files in PDF format we will apply some preprocessing to extract the text."
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Constitutional AI: Harmlessness from AI Feedback\n",
"Yuntao Bai∗, Saurav Kadavath, Sandipan Kundu, Amanda Askell, Jackson Kernion,\n",
"Andy Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini, Cameron McKinnon,\n",
"Carol Chen, Catherine Olsson, Christopher Olah, Danny Hernandez, Dawn Drain,\n",
"Deep Ganguli, Dustin Li, Eli Tran-Johnson, Ethan Perez, Jamie Kerr, Jared Mueller,\n",
"Jeffrey Ladish, Joshua Landau, Kamal Ndousse, Kamile Lukosuite, Liane Lovitt,\n",
"Michael Sellitto, Nelson Elhage, Nicholas Schiefer, Noemi Mercado, Nova DasSarma,\n",
"Robert Lasenby, Robin Larson, Sam Ringer, Scott Johnston, Shauna Kravec,\n",
"Sheer El Showk, Stanislav Fort, Tamera Lanham, Timothy Telleen-Lawton, Tom Conerly,\n",
"Tom Henighan, Tristan Hume, Samuel R. Bowman, Zac Hatfield-Dodds, Ben Mann,\n",
"Dario Amodei, Nicholas Joseph, Sam McCandlish, Tom Brown, Jared Kaplan∗\n",
"Anthropic\n",
"Abstract\n",
"As AI systems become more capable, we would like to enlist their help to supervise\n",
"other AIs. We experiment with methods for training a harmless AI assistant\n"
"paper_text = ''.join(page.extract_text() for page in reader.pages)\n",
"print(paper_text[:1000])"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Prompt\n",
"\n",
"Our prompt template will be similar to the previous example. We apply the same prompting best practices including beginning with a role, following with document contents wrapped in XML, and finishing with the full instructions. We include additional instructions to describe the quote extraction process and describe the desired output format.\n"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {},
"outputs": [],
"source": [
"system_prompt = \"You are an AI research assistant. Your task is to provide detailed answers to questions related to the content of the provided paper.\"\n",
"prompt_template = '''\n",
"Here is the paper you will be working with:\n",
"<paper>\n",
"{PAPER_CONTENT}\n",
"</paper>\n",
"\n",
"And here is the user's question, provided in a <user_question> tag:\n",
"<user_question>\n",
"{USER_QUESTION}\n",
"</user_question>\n",
"\n",
"To formulate your response, follow these steps:\n",
"1. Find the quotes from the paper that are the most relevant to answering the question. These quotes can be quite long if necessary (even multiple paragraphs). You may need to use many quotes to answer a single question, including code snippits and other examples.\n",
"2. Assign numbers to these quotes in the order they were found.\n",
"3. Based on the document and quotes, answer the question. Directly quote the documentation when possible, including examples.\n",
"4. When answering the question provide citations references in square brackets containing the number generated in step 2 (the number the citation was found)\n",
"5. Structure the output in the following format. Provide no preable or postamble:\n",
"<citations>\n",
"{{\n",
" \"citations\": [\n",
" {{\n",
" \n",
" \"number\": \"integer\",\n",
" \"passage\": \"string\"\n",
" }},\n",
" ...\n",
" ]\n",
"}}\n",
"</citations>\n",
"\n",
"<answer>A plain text answer, formatted as Markdown[1]</answer>\"\"\"\n",
"\n",
"Now it's your turn. First find and output the relevant quotes in the format described. Then provide your response to the user's question inside <answer> tags. Remember - only use information from the provided paper, and if you can't find the answer there, let the user know you don't have that information.'''"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Now we call 3.5 Sonnet with the populated prompt template and output the response "
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"{\n",
" \"citations\": [\n",
" {\n",
" \"number\": 1,\n",
" \"passage\": \"As AI systems become more capable, we would like to enlist their help to supervise other AIs. We experiment with methods for training a harmless AI assistant through self-improvement, without any human labels identifying harmful outputs. The only human oversight is provided through a list of rules or principles, and so we refer to the method as 'Constitutional AI'.\"\n",
" },\n",
" {\n",
" \"number\": 2,\n",
" \"passage\": \"The process involves both a supervised learning and a reinforcement learning phase. In the supervised phase we sample from an initial model, then generate self-critiques and revisions, and then finetune the original model on revised responses. In the RL phase, we sample from the finetuned model, use a model to evaluate which of the two samples is better, and then train a preference model from this dataset of AI preferences.\"\n",
" },\n",
" {\n",
" \"number\": 3,\n",
" \"passage\": \"As a result we are able to train a harmless but non-evasive AI assistant that engages with harmful queries by explaining its objections to them. Both the SL and RL methods can leverage chain-of-thought style reasoning to improve the human-judged performance and transparency of AI decision making. These methods make it possible to control AI behavior more precisely and with far fewer human labels.\"\n",
" }\n",
" ]\n",
"}\n",
"</citations>\n",
"\n",
"<answer>The main idea of this paper is the development of \"Constitutional AI\" - a method for training AI systems to be harmless through self-supervision, rather than relying on human feedback labels[1]. The process works in two phases: 1) a supervised learning phase where the AI generates self-critiques and revisions of its responses, and 2) a reinforcement learning phase where the AI evaluates and improves its own outputs[2]. The key innovation is that this approach can create AI assistants that are both harmless and non-evasive (meaning they explain their objections to harmful requests rather than simply refusing to engage), while requiring far fewer human labels than traditional methods[3].\n"
]
}
],
"source": [
"def answer_question(user_question):\n",
" # Generate the prompt with the user question and help center articles\n",
"raw_output = answer_question('What is the main idea of the paper?')\n",
"print(raw_output)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Post processing\n",
"\n",
"The structured output format we've developed, combining specific citations with the AI's answer, offers significant advantages in working with large documents. By providing precise quotes and their locations, we enable users to quickly verify information without reading the entire source material. It is possible to build a UX on top of this approach where readers can easily access and validate relevant portions of the document. Ultimately, this method enhances the transparency and utility of AI-assisted Q&A systems, making large documents more accessible and their content more verifiable."
]
}
],
"metadata": {
"kernelspec": {
"display_name": "py311",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.6"
}
},
"nbformat": 4,
"nbformat_minor": 2
}
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
cal-anthropic