mirror of
https://github.com/humanlayer/12-factor-agents.git
synced 2025-08-20 18:59:53 +03:00
1504 lines
48 KiB
Plaintext
1504 lines
48 KiB
Plaintext
{
|
|
"cells": [
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "a55820ee",
|
|
"metadata": {},
|
|
"source": [
|
|
"# Building the 12-factor agent template from scratch in Python"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "ba52e30a",
|
|
"metadata": {},
|
|
"source": [
|
|
"Steps to start from a bare Python repo and build up a 12-factor agent. This walkthrough will guide you through creating a Python agent that follows the 12-factor methodology with BAML."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "75b26c9b",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Chapter 0 - Hello World"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "fa4b9e07",
|
|
"metadata": {},
|
|
"source": [
|
|
"Let's start with a basic Python setup and a hello world program."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "4e464227",
|
|
"metadata": {},
|
|
"source": [
|
|
"This guide will walk you through building agents in Python with BAML.\n",
|
|
"\n",
|
|
"We'll start simple with a hello world program and gradually build up to a full agent.\n",
|
|
"\n",
|
|
"For this notebook, you'll need to have your OpenAI API key saved in Google Colab secrets.\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "99dac1bb",
|
|
"metadata": {},
|
|
"source": [
|
|
"Here's our simple hello world program:"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "9c6946fd",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"# ./walkthrough/00-main.py\n",
|
|
"def hello():\n",
|
|
" print('hello, world!')\n",
|
|
"\n",
|
|
"def main():\n",
|
|
" hello()"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "5523efac",
|
|
"metadata": {},
|
|
"source": [
|
|
"Let's run it to verify it works:"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "6a437eb2",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"main()"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "d9aa0df6",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Chapter 1 - CLI and Agent Loop"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "970c65da",
|
|
"metadata": {},
|
|
"source": [
|
|
"Now let's add BAML and create our first agent with a CLI interface."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "976a0fca",
|
|
"metadata": {},
|
|
"source": [
|
|
"In this chapter, we'll integrate BAML to create an AI agent that can respond to user input.\n",
|
|
"\n",
|
|
"## What is BAML?\n",
|
|
"\n",
|
|
"BAML (Boundary Markup Language) is a domain-specific language designed to help developers build reliable AI workflows and agents. Created by [BoundaryML](https://www.boundaryml.com/) (a Y Combinator W23 company), BAML adds the engineering to prompt engineering.\n",
|
|
"\n",
|
|
"### Why BAML?\n",
|
|
"\n",
|
|
"- **Type-safe outputs**: Get fully type-safe outputs from LLMs, even when streaming\n",
|
|
"- **Language agnostic**: Works with Python, TypeScript, Ruby, Go, and more\n",
|
|
"- **LLM agnostic**: Works with any LLM provider (OpenAI, Anthropic, etc.)\n",
|
|
"- **Better performance**: State-of-the-art structured outputs that outperform even OpenAI's native function calling\n",
|
|
"- **Developer-friendly**: Native VSCode extension with syntax highlighting, autocomplete, and interactive playground\n",
|
|
"\n",
|
|
"### Learn More\n",
|
|
"\n",
|
|
"- 📚 [Official Documentation](https://docs.boundaryml.com/home)\n",
|
|
"- 💻 [GitHub Repository](https://github.com/BoundaryML/baml)\n",
|
|
"- 🎯 [What is BAML?](https://docs.boundaryml.com/guide/introduction/what-is-baml)\n",
|
|
"- 📖 [BAML Examples](https://github.com/BoundaryML/baml-examples)\n",
|
|
"- 🏢 [Company Website](https://www.boundaryml.com/)\n",
|
|
"- 📰 [Blog: AI Agents Need a New Syntax](https://www.boundaryml.com/blog/ai-agents-need-new-syntax)\n",
|
|
"\n",
|
|
"BAML turns prompt engineering into schema engineering, where you focus on defining the structure of your data rather than wrestling with prompts. This approach leads to more reliable and maintainable AI applications.\n",
|
|
"\n",
|
|
"### Note on Developer Experience\n",
|
|
"\n",
|
|
"BAML works much better in VS Code with their official extension, which provides syntax highlighting, autocomplete, inline testing, and an interactive playground. However, for this notebook tutorial, we'll work with BAML files directly without the enhanced IDE features.\n",
|
|
"\n",
|
|
"First, let's set up BAML support in our notebook.\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "ba1f7191",
|
|
"metadata": {},
|
|
"source": [
|
|
"### BAML Setup\n",
|
|
"\n",
|
|
"Don't worry too much about this setup code - it will make sense later! For now, just know that:\n",
|
|
"- BAML is a tool for working with language models\n",
|
|
"- We need some special setup code to make it work nicely in Google Colab\n",
|
|
"- The `get_baml_client()` function will be used to interact with AI models"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "9910f8a3",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"!pip install baml-py==0.202.0 pydantic"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "a4ad6e77",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"import subprocess\n",
|
|
"import os\n",
|
|
"\n",
|
|
"# Try to import Google Colab userdata, but don't fail if not in Colab\n",
|
|
"try:\n",
|
|
" from google.colab import userdata\n",
|
|
" IN_COLAB = True\n",
|
|
"except ImportError:\n",
|
|
" IN_COLAB = False\n",
|
|
"\n",
|
|
"def baml_generate():\n",
|
|
" try:\n",
|
|
" result = subprocess.run(\n",
|
|
" [\"baml-cli\", \"generate\"],\n",
|
|
" check=True,\n",
|
|
" capture_output=True,\n",
|
|
" text=True\n",
|
|
" )\n",
|
|
" if result.stdout:\n",
|
|
" print(\"[baml-cli generate]\\n\", result.stdout)\n",
|
|
" if result.stderr:\n",
|
|
" print(\"[baml-cli generate]\\n\", result.stderr)\n",
|
|
" except subprocess.CalledProcessError as e:\n",
|
|
" msg = (\n",
|
|
" f\"`baml-cli generate` failed with exit code {e.returncode}\\n\"\n",
|
|
" f\"--- STDOUT ---\\n{e.stdout}\\n\"\n",
|
|
" f\"--- STDERR ---\\n{e.stderr}\"\n",
|
|
" )\n",
|
|
" raise RuntimeError(msg) from None\n",
|
|
"\n",
|
|
"def get_baml_client():\n",
|
|
" \"\"\"\n",
|
|
" a bunch of fun jank to work around the google colab import cache\n",
|
|
" \"\"\"\n",
|
|
" # Set API key from Colab secrets or environment\n",
|
|
" if IN_COLAB:\n",
|
|
" os.environ['OPENAI_API_KEY'] = userdata.get('OPENAI_API_KEY')\n",
|
|
" elif 'OPENAI_API_KEY' not in os.environ:\n",
|
|
" print(\"Warning: OPENAI_API_KEY not set. Please set it in your environment.\")\n",
|
|
" \n",
|
|
" baml_generate()\n",
|
|
" \n",
|
|
" # Force delete all baml_client modules from sys.modules\n",
|
|
" import sys\n",
|
|
" modules_to_delete = [key for key in sys.modules.keys() if key.startswith('baml_client')]\n",
|
|
" for module in modules_to_delete:\n",
|
|
" del sys.modules[module]\n",
|
|
" \n",
|
|
" # Now import fresh\n",
|
|
" import baml_client\n",
|
|
" return baml_client.sync_client.b\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "b99ba982",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"!baml-cli init"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "ee716f3a",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"!ls baml_src"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "894474da",
|
|
"metadata": {},
|
|
"source": [
|
|
"Now let's create our agent that will use BAML to process user input.\n",
|
|
"\n",
|
|
"First, we'll define the core agent logic:\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "dbf9d929",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"# ./walkthrough/01-agent.py\n",
|
|
"import json\n",
|
|
"from typing import Dict, Any, List\n",
|
|
"\n",
|
|
"# tool call or a respond to human tool\n",
|
|
"AgentResponse = Any # This will be the return type from b.DetermineNextStep\n",
|
|
"\n",
|
|
"class Event:\n",
|
|
" def __init__(self, type: str, data: Any):\n",
|
|
" self.type = type\n",
|
|
" self.data = data\n",
|
|
"\n",
|
|
"class Thread:\n",
|
|
" def __init__(self, events: List[Dict[str, Any]]):\n",
|
|
" self.events = events\n",
|
|
" \n",
|
|
" def serialize_for_llm(self):\n",
|
|
" # can change this to whatever custom serialization you want to do, XML, etc\n",
|
|
" # e.g. https://github.com/got-agents/agents/blob/59ebbfa236fc376618f16ee08eb0f3bf7b698892/linear-assistant-ts/src/agent.ts#L66-L105\n",
|
|
" return json.dumps(self.events)\n",
|
|
"\n",
|
|
"# right now this just runs one turn with the LLM, but\n",
|
|
"# we'll update this function to handle all the agent logic\n",
|
|
"def agent_loop(thread: Thread) -> AgentResponse:\n",
|
|
" b = get_baml_client() # This will be defined by the BAML setup\n",
|
|
" next_step = b.DetermineNextStep(thread.serialize_for_llm())\n",
|
|
" return next_step"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "b9421cd4",
|
|
"metadata": {},
|
|
"source": [
|
|
"Next, we need to define the BAML function that our agent will use.\n",
|
|
"\n",
|
|
"### Understanding BAML Syntax\n",
|
|
"\n",
|
|
"BAML files define:\n",
|
|
"- **Classes**: Structured output schemas (like `DoneForNow` below)\n",
|
|
"- **Functions**: AI-powered functions that take inputs and return structured outputs\n",
|
|
"- **Tests**: Example inputs/outputs to validate your prompts\n",
|
|
"\n",
|
|
"This BAML file defines what our agent can do:\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "58d8bda5",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"!curl -fsSL -o baml_src/agent.baml https://raw.githubusercontent.com/humanlayer/12-factor-agents/refs/heads/main/workshops/2025-07-16/./walkthrough/01-agent.baml && cat baml_src/agent.baml"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "1edc5279",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"!ls baml_src"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "ee489cc1",
|
|
"metadata": {},
|
|
"source": [
|
|
"Now let's create our main function that accepts a message parameter:\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "f4fea69e",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"# ./walkthrough/01-main.py\n",
|
|
"def main(message=\"hello from the notebook!\"):\n",
|
|
" # Create a new thread with the user's message as the initial event\n",
|
|
" thread = Thread([{\"type\": \"user_input\", \"data\": message}])\n",
|
|
" \n",
|
|
" # Run the agent loop with the thread\n",
|
|
" result = agent_loop(thread)\n",
|
|
" print(result)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "fe3fd9c7",
|
|
"metadata": {},
|
|
"source": [
|
|
"Let's test our agent! Try calling main() with different messages:\n",
|
|
"- `main(\"What's the weather like?\")`\n",
|
|
"- `main(\"Tell me a joke\")`\n",
|
|
"- `main(\"How are you doing today?\")`\n",
|
|
"\n",
|
|
"in this case, we'll use the baml_generate function to\n",
|
|
"generate the pydantic and python bindings from our\n",
|
|
"baml source, but in the future we'll skip this step as it\n",
|
|
"is done automatically by the get_baml_client() function\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "7fc1ee38",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"baml_generate()"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "8756df71",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"main(\"Hello from the Python notebook!\")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "9b5ca88c",
|
|
"metadata": {},
|
|
"source": []
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "e79f4d84",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Chapter 2 - Add Calculator Tools"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "4659d5ef",
|
|
"metadata": {},
|
|
"source": [
|
|
"Let's add some calculator tools to our agent."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "73df701a",
|
|
"metadata": {},
|
|
"source": [
|
|
"Let's start by adding a tool definition for the calculator.\n",
|
|
"\n",
|
|
"These are simple structured outputs that we'll ask the model to\n",
|
|
"return as a \"next step\" in the agentic loop.\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "c538cd53",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"!curl -fsSL -o baml_src/tool_calculator.baml https://raw.githubusercontent.com/humanlayer/12-factor-agents/refs/heads/main/workshops/2025-07-16/./walkthrough/02-tool_calculator.baml && cat baml_src/tool_calculator.baml"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "1df07ff3",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"!ls baml_src"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "1ffe3854",
|
|
"metadata": {},
|
|
"source": [
|
|
"Now, let's update the agent's DetermineNextStep method to\n",
|
|
"expose the calculator tools as potential next steps.\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "d6f9ee99",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"!curl -fsSL -o baml_src/agent.baml https://raw.githubusercontent.com/humanlayer/12-factor-agents/refs/heads/main/workshops/2025-07-16/./walkthrough/02-agent.baml && cat baml_src/agent.baml"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "147bd22c",
|
|
"metadata": {},
|
|
"source": [
|
|
"Now let's update our main function to show the tool call:\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "f8f99089",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"# ./walkthrough/02-main.py\n",
|
|
"def main(message=\"hello from the notebook!\"):\n",
|
|
" # Create a new thread with the user's message\n",
|
|
" thread = Thread([{\"type\": \"user_input\", \"data\": message}])\n",
|
|
" \n",
|
|
" # Get BAML client\n",
|
|
" b = get_baml_client()\n",
|
|
" \n",
|
|
" # Get the next step from the agent - just show the tool call\n",
|
|
" next_step = b.DetermineNextStep(thread.serialize_for_llm())\n",
|
|
" \n",
|
|
" # Print the raw response to show the tool call\n",
|
|
" print(next_step)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "ffb6c213",
|
|
"metadata": {},
|
|
"source": [
|
|
"Let's try out the calculator! The agent should recognize that you want to perform a calculation\n",
|
|
"and return the appropriate tool call instead of just a message.\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "7afaa326",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"main(\"can you add 3 and 4\")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "599d21dd",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Chapter 3 - Process Tool Calls in a Loop"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "d80e3f9f",
|
|
"metadata": {},
|
|
"source": [
|
|
"Now let's add a real agentic loop that can run the tools and get a final answer from the LLM."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "427fbc77",
|
|
"metadata": {},
|
|
"source": [
|
|
"In this chapter, we'll enhance our agent to process tool calls in a loop. This means:\n",
|
|
"- The agent can call multiple tools in sequence\n",
|
|
"- Each tool result is fed back to the agent\n",
|
|
"- The agent continues until it has a final answer\n",
|
|
"\n",
|
|
"Let's update our agent to handle tool calls properly:\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "ac8ae567",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"# ./walkthrough/03-agent.py\n",
|
|
"import json\n",
|
|
"from typing import Dict, Any, List\n",
|
|
"\n",
|
|
"class Thread:\n",
|
|
" def __init__(self, events: List[Dict[str, Any]]):\n",
|
|
" self.events = events\n",
|
|
" \n",
|
|
" def serialize_for_llm(self):\n",
|
|
" # can change this to whatever custom serialization you want to do, XML, etc\n",
|
|
" # e.g. https://github.com/got-agents/agents/blob/59ebbfa236fc376618f16ee08eb0f3bf7b698892/linear-assistant-ts/src/agent.ts#L66-L105\n",
|
|
" return json.dumps(self.events)\n",
|
|
"\n",
|
|
"\n",
|
|
"def agent_loop(thread: Thread) -> str:\n",
|
|
" b = get_baml_client()\n",
|
|
" \n",
|
|
" while True:\n",
|
|
" next_step = b.DetermineNextStep(thread.serialize_for_llm())\n",
|
|
" print(\"nextStep\", next_step)\n",
|
|
" \n",
|
|
" if next_step.intent == \"done_for_now\":\n",
|
|
" # response to human, return the next step object\n",
|
|
" return next_step.message\n",
|
|
" elif next_step.intent == \"add\":\n",
|
|
" thread.events.append({\n",
|
|
" \"type\": \"tool_call\",\n",
|
|
" \"data\": next_step.__dict__\n",
|
|
" })\n",
|
|
" result = next_step.a + next_step.b\n",
|
|
" print(\"tool_response\", result)\n",
|
|
" thread.events.append({\n",
|
|
" \"type\": \"tool_response\",\n",
|
|
" \"data\": result\n",
|
|
" })\n",
|
|
" continue\n",
|
|
" else:\n",
|
|
" raise ValueError(f\"Unknown intent: {next_step.intent}\")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "e875f4c2",
|
|
"metadata": {},
|
|
"source": [
|
|
"Now let's update our main function to use the new agent loop:\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "2aead128",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"# ./walkthrough/03-main.py\n",
|
|
"def main(message=\"hello from the notebook!\"):\n",
|
|
" # Create a new thread with the user's message\n",
|
|
" thread = Thread([{\"type\": \"user_input\", \"data\": message}])\n",
|
|
" \n",
|
|
" # Run the agent loop with full tool handling\n",
|
|
" result = agent_loop(thread)\n",
|
|
" \n",
|
|
" # Print the final response\n",
|
|
" print(f\"\\nFinal response: {result}\")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "a29bf07d",
|
|
"metadata": {},
|
|
"source": [
|
|
"Let's try it out! The agent should now call the tool and return the calculated result:\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "c6c6a0ca",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"main(\"can you add 3 and 4\")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "4c20a7d5",
|
|
"metadata": {},
|
|
"source": [
|
|
"You should see the agent:\n",
|
|
"1. Recognize it needs to use the add tool\n",
|
|
"2. Call the tool with the correct parameters\n",
|
|
"3. Get the result (7)\n",
|
|
"4. Generate a final response incorporating the result\n",
|
|
"\n",
|
|
"For more complex calculations, we need to handle all calculator operations. Let's add support for subtract, multiply, and divide:\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "561c0b54",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"# ./walkthrough/03b-agent.py\n",
|
|
"import json\n",
|
|
"from typing import Dict, Any, List, Union\n",
|
|
"\n",
|
|
"class Thread:\n",
|
|
" def __init__(self, events: List[Dict[str, Any]]):\n",
|
|
" self.events = events\n",
|
|
" \n",
|
|
" def serialize_for_llm(self):\n",
|
|
" # can change this to whatever custom serialization you want to do, XML, etc\n",
|
|
" # e.g. https://github.com/got-agents/agents/blob/59ebbfa236fc376618f16ee08eb0f3bf7b698892/linear-assistant-ts/src/agent.ts#L66-L105\n",
|
|
" return json.dumps(self.events)\n",
|
|
"\n",
|
|
"def handle_next_step(next_step, thread: Thread) -> Thread:\n",
|
|
" result: float\n",
|
|
" \n",
|
|
" if next_step.intent == \"add\":\n",
|
|
" result = next_step.a + next_step.b\n",
|
|
" print(\"tool_response\", result)\n",
|
|
" thread.events.append({\n",
|
|
" \"type\": \"tool_response\",\n",
|
|
" \"data\": result\n",
|
|
" })\n",
|
|
" return thread\n",
|
|
" elif next_step.intent == \"subtract\":\n",
|
|
" result = next_step.a - next_step.b\n",
|
|
" print(\"tool_response\", result)\n",
|
|
" thread.events.append({\n",
|
|
" \"type\": \"tool_response\",\n",
|
|
" \"data\": result\n",
|
|
" })\n",
|
|
" return thread\n",
|
|
" elif next_step.intent == \"multiply\":\n",
|
|
" result = next_step.a * next_step.b\n",
|
|
" print(\"tool_response\", result)\n",
|
|
" thread.events.append({\n",
|
|
" \"type\": \"tool_response\",\n",
|
|
" \"data\": result\n",
|
|
" })\n",
|
|
" return thread\n",
|
|
" elif next_step.intent == \"divide\":\n",
|
|
" result = next_step.a / next_step.b\n",
|
|
" print(\"tool_response\", result)\n",
|
|
" thread.events.append({\n",
|
|
" \"type\": \"tool_response\",\n",
|
|
" \"data\": result\n",
|
|
" })\n",
|
|
" return thread\n",
|
|
"\n",
|
|
"def agent_loop(thread: Thread) -> str:\n",
|
|
" b = get_baml_client()\n",
|
|
" \n",
|
|
" while True:\n",
|
|
" next_step = b.DetermineNextStep(thread.serialize_for_llm())\n",
|
|
" print(\"nextStep\", next_step)\n",
|
|
" \n",
|
|
" thread.events.append({\n",
|
|
" \"type\": \"tool_call\",\n",
|
|
" \"data\": next_step.__dict__\n",
|
|
" })\n",
|
|
" \n",
|
|
" if next_step.intent == \"done_for_now\":\n",
|
|
" # response to human, return the next step object\n",
|
|
" return next_step.message\n",
|
|
" elif next_step.intent in [\"add\", \"subtract\", \"multiply\", \"divide\"]:\n",
|
|
" thread = handle_next_step(next_step, thread)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "7c612b06",
|
|
"metadata": {},
|
|
"source": [
|
|
"Now let's test subtraction:\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "4be4af22",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"main(\"can you subtract 3 from 4\")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "1da0ad58",
|
|
"metadata": {},
|
|
"source": [
|
|
"Test multiplication:\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "49d5e040",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"main(\"can you multiply 3 and 4\")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "d5a27929",
|
|
"metadata": {},
|
|
"source": [
|
|
"Finally, let's test a complex multi-step calculation:\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "431414aa",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"main(\"can you multiply 3 and 4, then divide the result by 2 and then add 12 to that result\")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "99ab35d5",
|
|
"metadata": {},
|
|
"source": [
|
|
"Congratulations! You've taken your first step into hand-rolling an agent loop.\n",
|
|
"\n",
|
|
"Key concepts you've learned:\n",
|
|
"- **Thread Management**: Tracking conversation history and tool calls\n",
|
|
"- **Tool Execution**: Processing different tool types and returning results\n",
|
|
"- **Agent Loop**: Continuing until the agent has a final answer\n",
|
|
"\n",
|
|
"From here, we'll start incorporating more intermediate and advanced concepts for 12-factor agents.\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "9ba4e319",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Chapter 4 - Add Tests to agent.baml"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "6bf77db0",
|
|
"metadata": {},
|
|
"source": [
|
|
"Let's add some tests to our BAML agent."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "c6f0d38a",
|
|
"metadata": {},
|
|
"source": [
|
|
"In this chapter, we'll learn about BAML testing - a powerful feature that helps ensure your agents behave correctly.\n",
|
|
"\n",
|
|
"## Why Test BAML Functions?\n",
|
|
"\n",
|
|
"- **Catch regressions**: Ensure changes don't break existing behavior\n",
|
|
"- **Document behavior**: Tests serve as living documentation\n",
|
|
"- **Validate edge cases**: Test complex scenarios and conversation flows\n",
|
|
"- **CI/CD integration**: Run tests automatically in your pipeline\n",
|
|
"\n",
|
|
"Let's start with a simple test that checks the agent's ability to handle basic interactions:\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "cd0ae03f",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"!curl -fsSL -o baml_src/agent.baml https://raw.githubusercontent.com/humanlayer/12-factor-agents/refs/heads/main/workshops/2025-07-16/./walkthrough/04-agent.baml && cat baml_src/agent.baml"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "5bf05182",
|
|
"metadata": {},
|
|
"source": [
|
|
"Run the tests to see them in action:\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "30bbcac5",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"!baml-cli test"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "2cbbf5db",
|
|
"metadata": {},
|
|
"source": [
|
|
"Now let's improve the tests with assertions! Assertions let you verify specific properties of the agent's output.\n",
|
|
"\n",
|
|
"## BAML Assertion Syntax\n",
|
|
"\n",
|
|
"Assertions use the `@@assert` directive:\n",
|
|
"```\n",
|
|
"@@assert(name, {{condition}})\n",
|
|
"```\n",
|
|
"\n",
|
|
"- `name`: A descriptive name for the assertion\n",
|
|
"- `condition`: A boolean expression using `this` to access the output\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "dbbc5283",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"!curl -fsSL -o baml_src/agent.baml https://raw.githubusercontent.com/humanlayer/12-factor-agents/refs/heads/main/workshops/2025-07-16/./walkthrough/04b-agent.baml && cat baml_src/agent.baml"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "ecf9cb68",
|
|
"metadata": {},
|
|
"source": [
|
|
"Run the tests again to see assertions in action:\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "8d0611f3",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"!baml-cli test"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "8789e20e",
|
|
"metadata": {},
|
|
"source": [
|
|
"Finally, let's add more complex test cases that test multi-step conversations.\n",
|
|
"\n",
|
|
"These tests simulate an entire conversation flow, including:\n",
|
|
"- User input\n",
|
|
"- Tool calls made by the agent\n",
|
|
"- Tool responses\n",
|
|
"- Final agent response\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "abf5be5b",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"!curl -fsSL -o baml_src/agent.baml https://raw.githubusercontent.com/humanlayer/12-factor-agents/refs/heads/main/workshops/2025-07-16/./walkthrough/04c-agent.baml && cat baml_src/agent.baml"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "8ce0f9de",
|
|
"metadata": {},
|
|
"source": [
|
|
"Run the comprehensive test suite:\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "4afe82b8",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"!baml-cli test"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "5d0ba42b",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Key Testing Concepts\n",
|
|
"\n",
|
|
"1. **Test Structure**: Each test specifies functions, arguments, and assertions\n",
|
|
"2. **Progressive Testing**: Start simple, then test complex scenarios\n",
|
|
"3. **Conversation History**: Test how the agent handles multi-turn conversations\n",
|
|
"4. **Tool Integration**: Verify the agent correctly uses tools in sequence\n",
|
|
"\n",
|
|
"With these tests in place, you can confidently modify your agent knowing that core functionality is protected by automated tests!\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "bf15b77e",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Chapter 5 - Multiple Human Tools"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "e69dbeca",
|
|
"metadata": {},
|
|
"source": [
|
|
"In this section, we'll add support for multiple tools that serve to contact humans.\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "f3e29142",
|
|
"metadata": {},
|
|
"source": [
|
|
"So far, our agent only returns a final answer with \"done_for_now\". But what if the agent needs clarification?\n",
|
|
"\n",
|
|
"Let's add a new tool that allows the agent to request more information from the user.\n",
|
|
"\n",
|
|
"## Why Human-in-the-Loop?\n",
|
|
"\n",
|
|
"- **Handle ambiguous inputs**: When user input is unclear or contains typos\n",
|
|
"- **Request missing information**: When the agent needs more context\n",
|
|
"- **Confirm sensitive operations**: Before performing important actions\n",
|
|
"- **Interactive workflows**: Build conversational agents that engage users\n",
|
|
"\n",
|
|
"First, let's update our BAML file to include a ClarificationRequest tool:\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "9b42b75e",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"!curl -fsSL -o baml_src/agent.baml https://raw.githubusercontent.com/humanlayer/12-factor-agents/refs/heads/main/workshops/2025-07-16/./walkthrough/05-agent.baml && cat baml_src/agent.baml"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "7be2af7d",
|
|
"metadata": {},
|
|
"source": [
|
|
"Now let's update our agent to handle clarification requests:\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "21a3f526",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"# ./walkthrough/05-agent.py\n",
|
|
"# Agent implementation with clarification support\n",
|
|
"import json\n",
|
|
"\n",
|
|
"def agent_loop(thread, clarification_handler, max_iterations=3):\n",
|
|
" \"\"\"Run the agent loop until we get a final answer (max 3 iterations).\"\"\"\n",
|
|
" iteration_count = 0\n",
|
|
" while iteration_count < max_iterations:\n",
|
|
" iteration_count += 1\n",
|
|
" print(f\"🔄 Agent loop iteration {iteration_count}/{max_iterations}\")\n",
|
|
" \n",
|
|
" # Get the client\n",
|
|
" baml_client = get_baml_client()\n",
|
|
" \n",
|
|
" # Serialize the thread\n",
|
|
" thread_json = json.dumps(thread.events, indent=2)\n",
|
|
" \n",
|
|
" # Call the agent\n",
|
|
" result = baml_client.DetermineNextStep(thread_json)\n",
|
|
" \n",
|
|
" # Check what type of result we got based on intent\n",
|
|
" if hasattr(result, 'intent'):\n",
|
|
" if result.intent == 'done_for_now':\n",
|
|
" return result.message\n",
|
|
" elif result.intent == 'request_more_information':\n",
|
|
" # Get clarification from the human\n",
|
|
" clarification = clarification_handler(result.message)\n",
|
|
" \n",
|
|
" # Add the clarification to the thread\n",
|
|
" thread.events.append({\n",
|
|
" \"type\": \"clarification_request\",\n",
|
|
" \"data\": result.message\n",
|
|
" })\n",
|
|
" thread.events.append({\n",
|
|
" \"type\": \"clarification_response\",\n",
|
|
" \"data\": clarification\n",
|
|
" })\n",
|
|
" \n",
|
|
" # Continue the loop with the clarification\n",
|
|
" elif result.intent in ['add', 'subtract', 'multiply', 'divide']:\n",
|
|
" # Execute the appropriate tool based on intent\n",
|
|
" if result.intent == 'add':\n",
|
|
" result_value = result.a + result.b\n",
|
|
" operation = f\"add({result.a}, {result.b})\"\n",
|
|
" elif result.intent == 'subtract':\n",
|
|
" result_value = result.a - result.b\n",
|
|
" operation = f\"subtract({result.a}, {result.b})\"\n",
|
|
" elif result.intent == 'multiply':\n",
|
|
" result_value = result.a * result.b\n",
|
|
" operation = f\"multiply({result.a}, {result.b})\"\n",
|
|
" elif result.intent == 'divide':\n",
|
|
" if result.b == 0:\n",
|
|
" result_value = \"Error: Division by zero\"\n",
|
|
" else:\n",
|
|
" result_value = result.a / result.b\n",
|
|
" operation = f\"divide({result.a}, {result.b})\"\n",
|
|
" \n",
|
|
" print(f\"🔧 Calling tool: {operation} = {result_value}\")\n",
|
|
" \n",
|
|
" # Add the tool call and result to the thread\n",
|
|
" thread.events.append({\n",
|
|
" \"type\": \"tool_call\",\n",
|
|
" \"data\": {\n",
|
|
" \"tool\": \"calculator\",\n",
|
|
" \"operation\": operation,\n",
|
|
" \"result\": result_value\n",
|
|
" }\n",
|
|
" })\n",
|
|
" else:\n",
|
|
" return \"Error: Unexpected result type\"\n",
|
|
" \n",
|
|
" # If we've reached max iterations without a final answer\n",
|
|
" return f\"Agent reached maximum iterations ({max_iterations}) without completing the task.\"\n",
|
|
"\n",
|
|
"class Thread:\n",
|
|
" \"\"\"Simple thread to track conversation history.\"\"\"\n",
|
|
" def __init__(self, events):\n",
|
|
" self.events = events"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "5f017c77",
|
|
"metadata": {},
|
|
"source": [
|
|
"Finally, let's create a main function that handles human interaction:\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "e648be92",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"# ./walkthrough/05-main.py\n",
|
|
"def get_human_input(prompt):\n",
|
|
" \"\"\"Get input from human, handling both Colab and local environments.\"\"\"\n",
|
|
" print(f\"\\n🤔 {prompt}\")\n",
|
|
" \n",
|
|
" if IN_COLAB:\n",
|
|
" # In Colab, use actual input\n",
|
|
" response = input(\"Your response: \")\n",
|
|
" else:\n",
|
|
" # In local testing, return a fixed response\n",
|
|
" response = \"I meant to multiply 3 and 4\"\n",
|
|
" print(f\"📝 [Auto-response for testing]: {response}\")\n",
|
|
" \n",
|
|
" return response\n",
|
|
"\n",
|
|
"def main(message=\"hello from the notebook!\"):\n",
|
|
" # Function to handle clarification requests\n",
|
|
" def handle_clarification(question):\n",
|
|
" return get_human_input(f\"The agent needs clarification: {question}\")\n",
|
|
" \n",
|
|
" # Create a new thread with the user's message\n",
|
|
" thread = Thread([{\"type\": \"user_input\", \"data\": message}])\n",
|
|
" \n",
|
|
" print(f\"🚀 Starting agent with message: '{message}'\")\n",
|
|
" \n",
|
|
" # Run the agent loop\n",
|
|
" result = agent_loop(thread, handle_clarification)\n",
|
|
" \n",
|
|
" # Print the final response\n",
|
|
" print(f\"\\n✅ Final response: {result}\")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "2f4b962e",
|
|
"metadata": {},
|
|
"source": [
|
|
"Let's test with an ambiguous input that should trigger a clarification request:\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "948684f2",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"main(\"can you multiply 3 and FD*(#F&&\")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "54b7d0d4",
|
|
"metadata": {},
|
|
"source": [
|
|
"You should see:\n",
|
|
"1. The agent recognizes the input is unclear\n",
|
|
"2. It asks for clarification\n",
|
|
"3. In Colab, you'll be prompted to type a response\n",
|
|
"4. In local testing, an auto-response is provided\n",
|
|
"5. The agent continues with the clarified input\n",
|
|
"\n",
|
|
"## Interactive Testing in Colab\n",
|
|
"\n",
|
|
"When running in Google Colab, the `input()` function will create an interactive text box where you can type your response. Try different clarifications to see how the agent adapts!\n",
|
|
"\n",
|
|
"## Key Concepts\n",
|
|
"\n",
|
|
"- **Human Tools**: Special tool types that return control to the human\n",
|
|
"- **Conversation Flow**: The agent can pause execution to get human input\n",
|
|
"- **Context Preservation**: The full conversation history is maintained\n",
|
|
"- **Flexible Handling**: Different behaviors for different environments\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "253d3f6f",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Chapter 6 - Customize Your Prompt with Reasoning"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "87dc996a",
|
|
"metadata": {},
|
|
"source": [
|
|
"In this section, we'll explore how to customize the prompt of the agent with reasoning steps.\n",
|
|
"\n",
|
|
"This is core to [factor 2 - own your prompts](https://github.com/humanlayer/12-factor-agents/blob/main/content/factor-2-own-your-prompts.md)\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "7694a842",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Why Add Reasoning to Prompts?\n",
|
|
"\n",
|
|
"Adding explicit reasoning steps to your prompts can significantly improve agent performance:\n",
|
|
"\n",
|
|
"- **Better decisions**: The model thinks through problems step-by-step\n",
|
|
"- **Transparency**: You can see the model's thought process\n",
|
|
"- **Fewer errors**: Structured thinking reduces mistakes\n",
|
|
"- **Debugging**: Easier to identify where reasoning went wrong\n",
|
|
"\n",
|
|
"Let's update our agent prompt to include a reasoning step:\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "2b38033a",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"!curl -fsSL -o baml_src/agent.baml https://raw.githubusercontent.com/humanlayer/12-factor-agents/refs/heads/main/workshops/2025-07-16/./walkthrough/06-agent.baml && cat baml_src/agent.baml"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "30aff7de",
|
|
"metadata": {},
|
|
"source": [
|
|
"Now let's test it with a simple calculation to see the reasoning in action:\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "515f9755",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"main(\"can you multiply 3 and 4\")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "2f69536c",
|
|
"metadata": {},
|
|
"source": [
|
|
"The model uses explicit reasoning steps to think through the problem before making a decision.\n",
|
|
"\n",
|
|
"## Advanced Prompt Engineering\n",
|
|
"\n",
|
|
"You can enhance your prompts further by:\n",
|
|
"- Adding specific reasoning templates for different tasks\n",
|
|
"- Including examples of good reasoning\n",
|
|
"- Structuring the reasoning with numbered steps\n",
|
|
"- Adding checks for common mistakes\n",
|
|
"\n",
|
|
"The key is to guide the model's thinking process while still allowing flexibility.\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "8274aff0",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Chapter 7 - Customize Your Context Window"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "f930c899",
|
|
"metadata": {},
|
|
"source": [
|
|
"In this section, we'll explore how to customize the context window of the agent.\n",
|
|
"\n",
|
|
"This is core to [factor 3 - own your context window](https://github.com/humanlayer/12-factor-agents/blob/main/content/factor-3-own-your-context-window.md)\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "1d4235ed",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Context Window Serialization\n",
|
|
"\n",
|
|
"How you format your conversation history can significantly impact:\n",
|
|
"- **Token usage**: Some formats are more efficient\n",
|
|
"- **Model understanding**: Clear structure helps the model\n",
|
|
"- **Debugging**: Readable formats help development\n",
|
|
"\n",
|
|
"Let's implement two serialization formats: pretty-printed JSON and XML.\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "dccf9a9f",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"# ./walkthrough/07-agent.py\n",
|
|
"# Agent with configurable serialization formats\n",
|
|
"import json\n",
|
|
"\n",
|
|
"class Thread:\n",
|
|
" \"\"\"Thread that can serialize to different formats.\"\"\"\n",
|
|
" def __init__(self, events):\n",
|
|
" self.events = events\n",
|
|
" \n",
|
|
" def serialize_as_json(self):\n",
|
|
" \"\"\"Serialize thread events to pretty-printed JSON.\"\"\"\n",
|
|
" return json.dumps(self.events, indent=2)\n",
|
|
" \n",
|
|
" def serialize_as_xml(self):\n",
|
|
" \"\"\"Serialize thread events to XML format for better token efficiency.\"\"\"\n",
|
|
" import yaml\n",
|
|
" xml_parts = [\"<thread>\"]\n",
|
|
" \n",
|
|
" for event in self.events:\n",
|
|
" event_type = event['type']\n",
|
|
" event_data = event['data']\n",
|
|
" \n",
|
|
" if event_type == 'user_input':\n",
|
|
" xml_parts.append(f' <user_input>{event_data}</user_input>')\n",
|
|
" elif event_type == 'tool_call':\n",
|
|
" # Use YAML for tool call args - more compact than nested XML\n",
|
|
" yaml_content = yaml.dump(event_data, default_flow_style=False).strip()\n",
|
|
" xml_parts.append(f' <{event_data[\"tool\"]}>')\n",
|
|
" xml_parts.append(' ' + '\\n '.join(yaml_content.split('\\n')))\n",
|
|
" xml_parts.append(f' </{event_data[\"tool\"]}>')\n",
|
|
" elif event_type == 'clarification_request':\n",
|
|
" xml_parts.append(f' <clarification_request>{event_data}</clarification_request>')\n",
|
|
" elif event_type == 'clarification_response':\n",
|
|
" xml_parts.append(f' <clarification_response>{event_data}</clarification_response>')\n",
|
|
" \n",
|
|
" xml_parts.append(\"</thread>\")\n",
|
|
" return \"\\n\".join(xml_parts)\n",
|
|
"\n",
|
|
"def agent_loop(thread, clarification_handler, use_xml=True):\n",
|
|
" \"\"\"Run the agent loop with configurable serialization.\"\"\"\n",
|
|
" while True:\n",
|
|
" # Get the client\n",
|
|
" baml_client = get_baml_client()\n",
|
|
" \n",
|
|
" # Serialize the thread based on format preference\n",
|
|
" if use_xml:\n",
|
|
" thread_str = thread.serialize_as_xml()\n",
|
|
" print(f\"📄 Using XML serialization ({len(thread_str)} chars)\")\n",
|
|
" else:\n",
|
|
" thread_str = thread.serialize_as_json()\n",
|
|
" print(f\"📄 Using JSON serialization ({len(thread_str)} chars)\")\n",
|
|
" \n",
|
|
" # Call the agent\n",
|
|
" result = baml_client.DetermineNextStep(thread_str)\n",
|
|
" \n",
|
|
" # Check what type of result we got based on intent\n",
|
|
" if hasattr(result, 'intent'):\n",
|
|
" if result.intent == 'done_for_now':\n",
|
|
" return result.message\n",
|
|
" elif result.intent == 'request_more_information':\n",
|
|
" # Get clarification from the human\n",
|
|
" clarification = clarification_handler(result.message)\n",
|
|
" \n",
|
|
" # Add the clarification to the thread\n",
|
|
" thread.events.append({\n",
|
|
" \"type\": \"clarification_request\",\n",
|
|
" \"data\": result.message\n",
|
|
" })\n",
|
|
" thread.events.append({\n",
|
|
" \"type\": \"clarification_response\",\n",
|
|
" \"data\": clarification\n",
|
|
" })\n",
|
|
" \n",
|
|
" # Continue the loop with the clarification\n",
|
|
" elif result.intent in ['add', 'subtract', 'multiply', 'divide']:\n",
|
|
" # Execute the appropriate tool based on intent\n",
|
|
" if result.intent == 'add':\n",
|
|
" result_value = result.a + result.b\n",
|
|
" operation = f\"add({result.a}, {result.b})\"\n",
|
|
" elif result.intent == 'subtract':\n",
|
|
" result_value = result.a - result.b\n",
|
|
" operation = f\"subtract({result.a}, {result.b})\"\n",
|
|
" elif result.intent == 'multiply':\n",
|
|
" result_value = result.a * result.b\n",
|
|
" operation = f\"multiply({result.a}, {result.b})\"\n",
|
|
" elif result.intent == 'divide':\n",
|
|
" if result.b == 0:\n",
|
|
" result_value = \"Error: Division by zero\"\n",
|
|
" else:\n",
|
|
" result_value = result.a / result.b\n",
|
|
" operation = f\"divide({result.a}, {result.b})\"\n",
|
|
" \n",
|
|
" print(f\"🔧 Calling tool: {operation} = {result_value}\")\n",
|
|
" \n",
|
|
" # Add the tool call and result to the thread\n",
|
|
" thread.events.append({\n",
|
|
" \"type\": \"tool_call\",\n",
|
|
" \"data\": {\n",
|
|
" \"tool\": \"calculator\",\n",
|
|
" \"operation\": operation,\n",
|
|
" \"result\": result_value\n",
|
|
" }\n",
|
|
" })\n",
|
|
" else:\n",
|
|
" return \"Error: Unexpected result type\""
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "e02d1361",
|
|
"metadata": {},
|
|
"source": [
|
|
"Now let's create a main function that can switch between formats:\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "03c71da7",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"# ./walkthrough/07-main.py\n",
|
|
"def main(message=\"hello from the notebook!\", use_xml=True):\n",
|
|
" # Function to handle clarification requests\n",
|
|
" def handle_clarification(question):\n",
|
|
" return get_human_input(f\"The agent needs clarification: {question}\")\n",
|
|
" \n",
|
|
" # Create a new thread with the user's message\n",
|
|
" thread = Thread([{\"type\": \"user_input\", \"data\": message}])\n",
|
|
" \n",
|
|
" print(f\"🚀 Starting agent with message: '{message}'\")\n",
|
|
" print(f\"📋 Using {'XML' if use_xml else 'JSON'} format for thread serialization\")\n",
|
|
" \n",
|
|
" # Run the agent loop with XML serialization\n",
|
|
" result = agent_loop(thread, handle_clarification, use_xml=use_xml)\n",
|
|
" \n",
|
|
" # Print the final response\n",
|
|
" print(f\"\\n✅ Final response: {result}\")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "1d1718ab",
|
|
"metadata": {},
|
|
"source": [
|
|
"Let's test with JSON format first:\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "41b41a22",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"main(\"can you multiply 3 and 4, then divide the result by 2\", use_xml=False)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "d1bb4844",
|
|
"metadata": {},
|
|
"source": [
|
|
"Now let's try the same with XML format:\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "2ab2a144",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"main(\"can you multiply 3 and 4, then divide the result by 2\", use_xml=True)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "8883acac",
|
|
"metadata": {},
|
|
"source": [
|
|
"## XML vs JSON Trade-offs\n",
|
|
"\n",
|
|
"**XML Benefits**:\n",
|
|
"- More token-efficient for nested data\n",
|
|
"- Clear hierarchy with opening/closing tags\n",
|
|
"- Better for long conversations\n",
|
|
"\n",
|
|
"**JSON Benefits**:\n",
|
|
"- Familiar to most developers\n",
|
|
"- Easy to parse and debug\n",
|
|
"- Native to JavaScript/Python\n",
|
|
"\n",
|
|
"Choose based on your specific needs and token constraints!\n"
|
|
]
|
|
}
|
|
],
|
|
"metadata": {},
|
|
"nbformat": 4,
|
|
"nbformat_minor": 5
|
|
}
|