alihan/12-factor-agents
1
0
Fork 0
mirror of https://github.com/humanlayer/12-factor-agents.git synced 2025-08-20 18:59:53 +03:00
Code Issues Packages Projects Releases Wiki Activity

wip

Browse Source
This commit is contained in:
dexhorthy
2025-05-17 12:33:09 -07:00
parent 2b486695ae
commit f9637d7a50
29 changed files with 3042 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

462
workshops/2025-05-17/walkthrough.yaml
View File

@@ -220,3 +220,465 @@ sections:
from here, we're going to start incorporating some more intermediate and advanced
concepts for 12-factor agents.
- name: baml-tests
title: "Chapter 4 - Add Tests to agent.baml"
text: "Let's add some tests to our BAML agent."
steps:
- text: to start, leave the baml logs enabled
command: |
export BAML_LOG=debug
- text: |
next, let's add some tests to the agent
We'll start with a simple test that checks the agent's ability to handle
a basic calculation.
file: {src: ./walkthrough/04-agent.baml, dest: baml_src/agent.baml}
- text: "Run the tests"
command: |
npx baml-cli test
- text: |
now, let's improve the test with assertions!
Assertions are a great way to make sure the agent is working as expected,
and can easily be extended to check for more complex behavior.
file: {src: ./walkthrough/04b-agent.baml, dest: baml_src/agent.baml}
- text: "Run the tests"
command: |
npx baml-cli test
- text: |
as you add more tests, you can disable the logs to keep the output clean.
You may want to turn them on as you iterate on specific tests.
command: |
export BAML_LOG=off
- text: |
now, let's add some more complex test cases,
where we resume from in the middle of an in-progress
agentic context window
file: {src: ./walkthrough/04c-agent.baml, dest: baml_src/agent.baml}
- text: |
let's try to run it
command: |
npx baml-cli test
- name: human-tools
title: "Chapter 5 - Multiple Human Tools"
text: |
In this section, we'll add support for multiple tools that serve to
contact humans.
steps:
- text: "for this section, we'll disable the baml logs. You can optionally enable them if you want to see more details."
command: |
export BAML_LOG=off
- text: |
first, let's add a tool that can request clarification from a human
this will be different from the "done_for_now" tool,
and can be used to more flexibly handle different types of human interactions
in your agent.
file: {src: ./walkthrough/05-agent.baml, dest: baml_src/agent.baml}
- text: |
next, let's re-generate the client code
NOTE - if you're using the VSCode extension for BAML,
the client will be regenerated automatically when you save the file
in your editor.
command: |
npx baml-cli generate
incremental: true
- text: |
now, let's update the agent to use the new tool
file: {src: ./walkthrough/05-agent.ts, dest: src/agent.ts}
- text: |
next, let's update the CLI to handle clarification requests
by requesting input from the user on the CLI
file: {src: ./walkthrough/05-cli.ts, dest: src/cli.ts}
- text: |
let's try it out
command: |
npx tsx src/index.ts 'can you multiply 3 and FD*(#F&& '
- text: |
next, let's add a test that checks the agent's ability to handle
a clarification request
file: {src: ./walkthrough/05b-agent.baml, dest: baml_src/agent.baml}
- text: |
and now we can run the tests again
command: |
npx baml-cli test
- text: |
you'll notice the new test passes, but the hello world test fails
This is because the agent's default behavior is to return "done_for_now"
file: {src: ./walkthrough/05c-agent.baml, dest: baml_src/agent.baml}
- text: "Verify tests pass"
command: |
npx baml-cli test
- name: customize-prompt
title: "Chapter 6 - Customize Your Prompt with Reasoning"
text: |
In this section, we'll explore how to customize the prompt of the agent
with reasoning steps.
this is core to [factor 2 - own your prompts](https://github.com/humanlayer/12-factor-agents/blob/main/content/factor-2-own-your-prompts.md)
there's a deep dive on reasoning on AI That Works [reasoning models versus reasoning steps](https://github.com/hellovai/ai-that-works/tree/main/2025-04-07-reasoning-models-vs-prompts)
steps:
- text: "for this section, it will be helpful to leave the baml logs enabled"
command: |
export BAML_LOG=debug
- text: |
update the agent prompt to include a reasoning step
file: {src: ./walkthrough/06-agent.baml, dest: baml_src/agent.baml}
- text: generate the updated client
command: |
npx baml-cli generate
incremental: true
- text: |
now, you can try it out with a simple prompt
command: |
npx tsx src/index.ts 'can you multiply 3 and 4'
results:
- text: you should see output from the baml logs showing the reasoning steps
- text: |
#### optional challenge
add a field to your tool output format that includes the reasoning steps in the output!
- name: context-window
title: "Chapter 7 - Customize Your Context Window"
text: |
In this section, we'll explore how to customize the context window
of the agent.
this is core to [factor 3 - own your context window](https://github.com/humanlayer/12-factor-agents/blob/main/content/factor-3-own-your-context-window.md)
steps:
- text: |
update the agent to pretty-print the Context window for the model
file: {src: ./walkthrough/07-agent.ts, dest: src/agent.ts}
- text: "Test the formatting"
command: |
BAML_LOG=info npx tsx src/index.ts 'can you multiply 3 and 4, then divide the result by 2 and then add 12 to that result'
- text: |
next, let's update the agent to use XML formatting instead
this is a very popular format for passing data to a model,
among other things, because of the token efficiency of XML.
file: {src: ./walkthrough/07b-agent.ts, dest: src/agent.ts}
- text: |
let's try it out
command: |
BAML_LOG=info npx tsx src/index.ts 'can you multiply 3 and 4, then divide the result by 2 and then add 12 to that result'
- text: |
lets update our tests to match the new output format
file: {src: ./walkthrough/07c-agent.baml, dest: baml_src/agent.baml}
- text: |
check out the updated tests
command: |
npx baml-cli test
- name: api-endpoints
title: "Chapter 8 - Adding API Endpoints"
text: "Add an Express server to expose the agent via HTTP."
steps:
- text: "for this section, we'll disable the baml logs. You can optionally enable them if you want to see more details."
command: |
export BAML_LOG=off
- text: "Install Express and types"
command: |
npm install express && npm install --save-dev @types/express supertest
incremental: true
- text: "Add the server implementation"
file: {src: ./walkthrough/08-server.ts, dest: src/server.ts}
- text: "Start the server"
command: |
npx tsx src/server.ts
- text: "Test with curl (in another terminal)"
command: |
curl -X POST http://localhost:3000/thread \
-H "Content-Type: application/json" \
-d '{"message":"can you add 3 and 4"}'
results:
- text: |
You should get an answer from the agent which includes the
agentic trace, ending in a message like:
code: |
{"intent":"done_for_now","message":"The sum of 3 and 4 is 7."}
- name: state-management
title: "Chapter 9 - In-Memory State and Async Clarification"
text: "Add state management and async clarification support."
steps:
- text: "for this section, we'll disable the baml logs. You can optionally enable them if you want to see more details."
command: |
export BAML_LOG=off
- text: "Add some simple in-memory state management for threads"
file: {src: ./walkthrough/09-state.ts, dest: src/state.ts}
- text: |
update the server to use the state management
* Add thread state management using `ThreadStore`
* return thread IDs and response URLs from the /thread endpoint
* implement GET /thread/:id
* implement POST /thread/:id/response
file: {src: ./walkthrough/09-server.ts, dest: src/server.ts}
- text: "Start the server"
command: |
npx tsx src/server.ts
- text: "Test clarification flow"
command: |
curl -X POST http://localhost:3000/thread \
-H "Content-Type: application/json" \
-d '{"message":"can you multiply 3 and xyz"}'
- name: human-approval
title: "Chapter 10 - Adding Human Approval"
text: "Add support for human approval of operations."
steps:
- text: "for this section, we'll disable the baml logs. You can optionally enable them if you want to see more details."
command: |
export BAML_LOG=off
- text: |
update the server to handle human approvals
* Import `handleNextStep` to execute approved actions
* Add two payload types to distinguish approvals from responses
* Handle responses and approvals differently in the endpoint
* Show better error messages when things go wrongs
file: {src: ./walkthrough/10-server.ts, dest: src/server.ts}
- text: "Add a few methods to the agent to handle approvals and responses"
file: {src: ./walkthrough/10-agent.ts, dest: src/agent.ts}
- text: "Start the server"
command: |
npx tsx src/server.ts
- text: "Test division with approval"
command: |
curl -X POST http://localhost:3000/thread \
-H "Content-Type: application/json" \
-d '{"message":"can you divide 3 by 4"}'
results:
- text: "You should see:"
code: |
{
"thread_id": "2b243b66-215a-4f37-8bc6-9ace3849043b",
"events": [
{
"type": "user_input",
"data": "can you divide 3 by 4"
},
{
"type": "tool_call",
"data": {
"intent": "divide",
"a": 3,
"b": 4,
"response_url": "/thread/2b243b66-215a-4f37-8bc6-9ace3849043b/response"
}
}
]
}
- text: "reject the request with another curl call, changing the thread ID"
command: |
curl -X POST 'http://localhost:3000/thread/{thread_id}/response' \
-H "Content-Type: application/json" \
-d '{"type": "approval", "approved": false, "comment": "I dont think thats right, use 5 instead of 4"}'
results:
- text: 'You should see: the last tool call is now `"intent":"divide","a":3,"b":5`'
code: |
{
"events": [
{
"type": "user_input",
"data": "can you divide 3 by 4"
},
{
"type": "tool_call",
"data": {
"intent": "divide",
"a": 3,
"b": 4,
"response_url": "/thread/2b243b66-215a-4f37-8bc6-9ace3849043b/response"
}
},
{
"type": "tool_response",
"data": "user denied the operation with feedback: \"I dont think thats right, use 5 instead of 4\""
},
{
"type": "tool_call",
"data": {
"intent": "divide",
"a": 3,
"b": 5,
"response_url": "/thread/1f1f5ff5-20d7-4114-97b4-3fc52d5e0816/response"
}
}
]
}
- text: "now you can approve the operation"
command: |
curl -X POST 'http://localhost:3000/thread/{thread_id}/response' \
-H "Content-Type: application/json" \
-d '{"type": "approval", "approved": true}'
results:
- text: "you should see the final message includes the tool response and final result!"
code: |
...
{
"type": "tool_response",
"data": 0.5
},
{
"type": "done_for_now",
"message": "I divided 3 by 6 and the result is 0.5. If you have any more operations or queries, feel free to ask!",
"response_url": "/thread/2b469403-c497-4797-b253-043aae830209/response"
}
- name: humanlayer-approval
title: "Chapter 11 - Human Approvals over email"
text: |
in this section, we'll add support for human approvals over email.
This will start a little bit contrived, just to get the concepts down -
We'll start by invoking the workflow from the CLI but approvals for `divide`
and `request_more_information` will be handled over email,
then the final `done_for_now` answer will be printed back to the CLI
While contrived, this is a great example of the flexibility you get from
[factor 7 - contact humans with tools](https://github.com/humanlayer/12-factor-agents/blob/main/content/factor-7-contact-humans-with-tools.md)
steps:
- text: "for this section, we'll disable the baml logs. You can optionally enable them if you want to see more details."
command: |
export BAML_LOG=off
- text: "Install HumanLayer"
command: |
npm install humanlayer
incremental: true
- text: "Update CLI to send `divide` and `request_more_information` to a human via email"
file: {src: ./walkthrough/11-cli.ts, dest: src/cli.ts}
- text: "Run the CLI"
command: |
npx tsx src/index.ts 'can you divide 4 by 5'
results:
- text: "The last line of your program should mention human review step"
code: |
nextStep { intent: 'divide', a: 4, b: 5 }
HumanLayer: Requested human approval from HumanLayer cloud
- text: |
go ahead and respond to the email with some feedback:
![reject-email](https://github.com/humanlayer/12-factor-agents/blob/main/workshops/2025-05/walkthrough/11-email-reject.png?raw=true)
- text: |
you should get another email with an updated attempt based on your feedback!
You can go ahead and approve this one:
![appove-email](https://github.com/humanlayer/12-factor-agents/blob/main/workshops/2025-05/walkthrough/11-email-approve.png?raw=true)
results:
- text: and your final output will look like
code: |
nextStep {
intent: 'done_for_now',
message: 'The division of 4 by 5 is 0.8. If you have any other calculations or questions, feel free to ask!'
}
The division of 4 by 5 is 0.8. If you have any other calculations or questions, feel free to ask!
- text: |
lets implement the `request_more_information` flow as well
file: {src: ./walkthrough/11b-cli.ts, dest: src/cli.ts}
- text: |
lets test the require_approval flow as by asking for a calculation
with garbled input:
command: |
npx tsx src/index.ts 'can you multiply 4 and xyz'
- text: "You should get an email with a request for clarification"
command: |
Can you clarify what 'xyz' represents in this context? Is it a specific number, variable, or something else?
- text: you can response with something like
command: |
use 8 instead of xyz
results:
- text: you should see a final result on the CLI like
code: |
I have multiplied 4 and xyz, using the value 8 for xyz, resulting in 32.
- text: |
as a final step, lets explore using a custom html template for the email
file: {src: ./walkthrough/11c-cli.ts, dest: src/cli.ts}
- text: |
first try with divide:
command: |
npx tsx src/index.ts 'can you divide 4 by 5'
results:
- text: |
you should see a slightly different email with the custom template
![custom-template-email](https://github.com/humanlayer/12-factor-agents/blob/main/workshops/2025-05/walkthrough/11-email-custom.png?raw=true)
feel free to run with the flow and then you can try updating the template to your liking
(if you're using cursor, something as simple as highlighting the template and asking to "make it better"
should do the trick)
try triggering "request_more_information" as well!
- text: |
thats it - in the next chapter, we'll build a fully email-driven
workflow agent that uses webhooks for human approval
- name: humanlayer-webhook
title: "Chapter XX - HumanLayer Webhook Integration"
text: |
the previous sections used the humanlayer SDK in "synchronous mode" - that
means every time we wait for human approval, we sit in a loop
polling until the human response if received.
That's obviously not ideal, especially for production workloads,
so in this section we'll implement [factor 6 - launch / pause / resume with simple APIs](https://github.com/humanlayer/12-factor-agents/blob/main/content/factor-6-launch-pause-resume.md)
by updating the server to end processing after contacting a human, and use webhooks to receive the results.
steps:
- text: |
add code to initialize humanlayer in the server
file: {src: ./walkthrough/12-1-server-init.ts, dest: src/server.ts}
- text: |
next, lets update the /thread endpoint to
1. handle requests asynchronously, returning immediately
2. create a human contact on request_more_information and done_for_now calls
# file: {src: }
- text: |
Update the server to be able to handle request_clarification responses
- remove the old /response endpoint and types
- update the /thread endpoint to run processing asynchronously, return immediately
- send a state.threadId when requesting human responses
- add a handleHumanResponse function to process the human response
- add a /webhook endpoint to handle the webhook response
file: {src: ./walkthrough/12a-server.ts, dest: src/server.ts}
- text: "Start the server in another terminal"
command: |
npx tsx src/server.ts
- text: |
now that the server is running, send a payload to the '/thread' endpoint
- text: __ do the response step
- text: __ now handle approvals for divide
- text: __ now also handle done_for_now

61
workshops/2025-05-17/walkthrough/04-agent.baml Normal file
View File

@@ -0,0 +1,61 @@
class DoneForNow {
intent "done_for_now"
message string
}
client<llm> Qwen3 {
provider "openai-generic"
options {
base_url env.BASETEN_BASE_URL
api_key env.BASETEN_API_KEY
}
}
function DetermineNextStep(
thread: string
) -> CalculatorTools | DoneForNow {
client Qwen3
prompt #"
{{ _.role("system") }}
/nothink
You are a helpful assistant that can help with tasks.
{{ _.role("user") }}
You are working on the following thread:
{{ thread }}
What should the next step be?
{{ ctx.output_format }}
"#
}
test HelloWorld {
functions [DetermineNextStep]
args {
thread #"
{
"type": "user_input",
"data": "hello!"
}
"#
}
}
test MathOperation {
functions [DetermineNextStep]
args {
thread #"
{
"type": "user_input",
"data": "can you multiply 3 and 4?"
}
"#
}
}

63
workshops/2025-05-17/walkthrough/04b-agent.baml Normal file
View File

@@ -0,0 +1,63 @@
class DoneForNow {
intent "done_for_now"
message string
}
client<llm> Qwen3 {
provider "openai-generic"
options {
base_url env.BASETEN_BASE_URL
api_key env.BASETEN_API_KEY
}
}
function DetermineNextStep(
thread: string
) -> CalculatorTools | DoneForNow {
client Qwen3
prompt #"
{{ _.role("system") }}
/nothink
You are a helpful assistant that can help with tasks.
{{ _.role("user") }}
You are working on the following thread:
{{ thread }}
What should the next step be?
{{ ctx.output_format }}
"#
}
test HelloWorld {
functions [DetermineNextStep]
args {
thread #"
{
"type": "user_input",
"data": "hello!"
}
"#
}
@@assert(hello, {{this.intent == "done_for_now"}})
}
test MathOperation {
functions [DetermineNextStep]
args {
thread #"
{
"type": "user_input",
"data": "can you multiply 3 and 4?"
}
"#
}
@@assert(math_operation, {{this.intent == "multiply"}})
}

114
workshops/2025-05-17/walkthrough/04c-agent.baml Normal file
View File

@@ -0,0 +1,114 @@
class DoneForNow {
intent "done_for_now"
message string
}
client<llm> Qwen3 {
provider "openai-generic"
options {
base_url env.BASETEN_BASE_URL
api_key env.BASETEN_API_KEY
}
}
function DetermineNextStep(
thread: string
) -> CalculatorTools | DoneForNow {
client Qwen3
prompt #"
{{ _.role("system") }}
/nothink
You are a helpful assistant that can help with tasks.
{{ _.role("user") }}
You are working on the following thread:
{{ thread }}
What should the next step be?
{{ ctx.output_format }}
"#
}
test HelloWorld {
functions [DetermineNextStep]
args {
thread #"
{
"type": "user_input",
"data": "hello!"
}
"#
}
@@assert(intent, {{this.intent == "done_for_now"}})
}
test MathOperation {
functions [DetermineNextStep]
args {
thread #"
{
"type": "user_input",
"data": "can you multiply 3 and 4?"
}
"#
}
@@assert(intent, {{this.intent == "multiply"}})
}
test LongMath {
functions [DetermineNextStep]
args {
thread #"
[
{
"type": "user_input",
"data": "can you multiply 3 and 4, then divide the result by 2 and then add 12 to that result?"
},
{
"type": "tool_call",
"data": {
"intent": "multiply",
"a": 3,
"b": 4
}
},
{
"type": "tool_response",
"data": 12
},
{
"type": "tool_call",
"data": {
"intent": "divide",
"a": 12,
"b": 2
}
},
{
"type": "tool_response",
"data": 6
},
{
"type": "tool_call",
"data": {
"intent": "add",
"a": 6,
"b": 12
}
},
{
"type": "tool_response",
"data": 18
}
]
"#
}
@@assert(intent, {{this.intent == "done_for_now"}})
@@assert(answer, {{"18" in this.message}})
}

127
workshops/2025-05-17/walkthrough/05-agent.baml Normal file
View File

@@ -0,0 +1,127 @@
// human tools are async requests to a human
type HumanTools = ClarificationRequest | DoneForNow
class ClarificationRequest {
intent "request_more_information" @description("you can request more information from me")
message string
}
class DoneForNow {
intent "done_for_now"
message string @description(#"
message to send to the user about the work that was done.
"#)
}
client<llm> Qwen3 {
provider "openai-generic"
options {
base_url env.BASETEN_BASE_URL
api_key env.BASETEN_API_KEY
}
}
function DetermineNextStep(
thread: string
) -> HumanTools | CalculatorTools {
client Qwen3
prompt #"
{{ _.role("system") }}
/nothink
You are a helpful assistant that can help with tasks.
{{ _.role("user") }}
You are working on the following thread:
{{ thread }}
What should the next step be?
{{ ctx.output_format }}
"#
}
test HelloWorld {
functions [DetermineNextStep]
args {
thread #"
{
"type": "user_input",
"data": "hello!"
}
"#
}
@@assert(intent, {{this.intent == "done_for_now"}})
}
test MathOperation {
functions [DetermineNextStep]
args {
thread #"
{
"type": "user_input",
"data": "can you multiply 3 and 4?"
}
"#
}
@@assert(intent, {{this.intent == "multiply"}})
}
test LongMath {
functions [DetermineNextStep]
args {
thread #"
[
{
"type": "user_input",
"data": "can you multiply 3 and 4, then divide the result by 2 and then add 12 to that result?"
},
{
"type": "tool_call",
"data": {
"intent": "multiply",
"a": 3,
"b": 4
}
},
{
"type": "tool_response",
"data": 12
},
{
"type": "tool_call",
"data": {
"intent": "divide",
"a": 12,
"b": 2
}
},
{
"type": "tool_response",
"data": 6
},
{
"type": "tool_call",
"data": {
"intent": "add",
"a": 6,
"b": 12
}
},
{
"type": "tool_response",
"data": 18
}
]
"#
}
@@assert(intent, {{this.intent == "done_for_now"}})
@@assert(answer, {{"18" in this.message}})
}

87
workshops/2025-05-17/walkthrough/05-agent.ts Normal file
View File

@@ -0,0 +1,87 @@
import { AddTool, SubtractTool, DivideTool, MultiplyTool, b } from "../baml_client";
export interface Event {
type: string
data: any;
}
export class Thread {
events: Event[] = [];
constructor(events: Event[]) {
this.events = events;
}
serializeForLLM() {
// can change this to whatever custom serialization you want to do, XML, etc
// e.g. https://github.com/got-agents/agents/blob/59ebbfa236fc376618f16ee08eb0f3bf7b698892/linear-assistant-ts/src/agent.ts#L66-L105
return JSON.stringify(this.events);
}
}
export type CalculatorTool = AddTool | SubtractTool | MultiplyTool | DivideTool;
export async function handleNextStep(nextStep: CalculatorTool, thread: Thread): Promise<Thread> {
let result: number;
switch (nextStep.intent) {
case "add":
result = nextStep.a + nextStep.b;
console.log("tool_response", result);
thread.events.push({
"type": "tool_response",
"data": result
});
return thread;
case "subtract":
result = nextStep.a - nextStep.b;
console.log("tool_response", result);
thread.events.push({
"type": "tool_response",
"data": result
});
return thread;
case "multiply":
result = nextStep.a * nextStep.b;
console.log("tool_response", result);
thread.events.push({
"type": "tool_response",
"data": result
});
return thread;
case "divide":
result = nextStep.a / nextStep.b;
console.log("tool_response", result);
thread.events.push({
"type": "tool_response",
"data": result
});
return thread;
}
}
export async function agentLoop(thread: Thread): Promise<Thread> {
while (true) {
const nextStep = await b.DetermineNextStep(thread.serializeForLLM());
console.log("nextStep", nextStep);
thread.events.push({
"type": "tool_call",
"data": nextStep
});
switch (nextStep.intent) {
case "done_for_now":
case "request_more_information":
// response to human, return the thread
return thread;
case "add":
case "subtract":
case "multiply":
case "divide":
thread = await handleNextStep(nextStep, thread);
}
}
}

50
workshops/2025-05-17/walkthrough/05-cli.ts Normal file
View File

@@ -0,0 +1,50 @@
// cli.ts lets you invoke the agent loop from the command line
import { agentLoop, Thread, Event } from "../src/agent";
export async function cli() {
// Get command line arguments, skipping the first two (node and script name)
const args = process.argv.slice(2);
if (args.length === 0) {
console.error("Error: Please provide a message as a command line argument");
process.exit(1);
}
// Join all arguments into a single message
const message = args.join(" ");
// Create a new thread with the user's message as the initial event
const thread = new Thread([{ type: "user_input", data: message }]);
// Run the agent loop with the thread
const result = await agentLoop(thread);
let lastEvent = result.events.slice(-1)[0];
while (lastEvent.data.intent === "request_more_information") {
const message = await askHuman(lastEvent.data.message);
thread.events.push({ type: "human_response", data: message });
const result = await agentLoop(thread);
lastEvent = result.events.slice(-1)[0];
}
// print the final result
// optional - you could loop here too
console.log(lastEvent.data.message);
process.exit(0);
}
async function askHuman(message: string) {
const readline = require('readline').createInterface({
input: process.stdin,
output: process.stdout
});
return new Promise((resolve) => {
readline.question(`${message}\n> `, (answer: string) => {
resolve(answer);
});
});
}

156
workshops/2025-05-17/walkthrough/05b-agent.baml Normal file
View File

@@ -0,0 +1,156 @@
// human tools are async requests to a human
type HumanTools = ClarificationRequest | DoneForNow
class ClarificationRequest {
intent "request_more_information" @description("you can request more information from me")
message string
}
class DoneForNow {
intent "done_for_now"
message string @description(#"
message to send to the user about the work that was done.
"#)
}
client<llm> Qwen3 {
provider "openai-generic"
options {
base_url env.BASETEN_BASE_URL
api_key env.BASETEN_API_KEY
}
}
function DetermineNextStep(
thread: string
) -> HumanTools | CalculatorTools {
client Qwen3
prompt #"
{{ _.role("system") }}
/nothink
You are a helpful assistant that can help with tasks.
{{ _.role("user") }}
You are working on the following thread:
{{ thread }}
What should the next step be?
{{ ctx.output_format }}
"#
}
test HelloWorld {
functions [DetermineNextStep]
args {
thread #"
{
"type": "user_input",
"data": "hello!"
}
"#
}
@@assert(intent, {{this.intent == "done_for_now"}})
}
test MathOperation {
functions [DetermineNextStep]
args {
thread #"
{
"type": "user_input",
"data": "can you multiply 3 and 4?"
}
"#
}
@@assert(intent, {{this.intent == "multiply"}})
}
test LongMath {
functions [DetermineNextStep]
args {
thread #"
[
{
"type": "user_input",
"data": "can you multiply 3 and 4, then divide the result by 2 and then add 12 to that result?"
},
{
"type": "tool_call",
"data": {
"intent": "multiply",
"a": 3,
"b": 4
}
},
{
"type": "tool_response",
"data": 12
},
{
"type": "tool_call",
"data": {
"intent": "divide",
"a": 12,
"b": 2
}
},
{
"type": "tool_response",
"data": 6
},
{
"type": "tool_call",
"data": {
"intent": "add",
"a": 6,
"b": 12
}
},
{
"type": "tool_response",
"data": 18
}
]
"#
}
@@assert(intent, {{this.intent == "done_for_now"}})
@@assert(answer, {{"18" in this.message}})
}
test MathOperationWithClarification {
functions [DetermineNextStep]
args {
thread #"
[{"type":"user_input","data":"can you multiply 3 and feee9ff10"}]
"#
}
@@assert(intent, {{this.intent == "request_more_information"}})
}
test MathOperationPostClarification {
functions [DetermineNextStep]
args {
thread #"
[
{"type":"user_input","data":"can you multiply 3 and FD*(#F&& ?"},
{"type":"tool_call","data":{"intent":"request_more_information","message":"It seems like there was a typo or mistake in your request. Could you please clarify or provide the correct numbers you would like to multiply?"}},
{"type":"human_response","data":"lets try 12 instead"},
]
"#
}
@@assert(intent, {{this.intent == "multiply"}})
@@assert(a, {{this.b == 12}})
@@assert(b, {{this.a == 3}})
}

146
workshops/2025-05-17/walkthrough/05c-agent.baml Normal file
View File

@@ -0,0 +1,146 @@
// human tools are async requests to a human
type HumanTools = ClarificationRequest | DoneForNow
class ClarificationRequest {
intent "request_more_information" @description("you can request more information from me")
message string
}
class DoneForNow {
intent "done_for_now"
message string @description(#"
message to send to the user about the work that was done.
"#)
}
function DetermineNextStep(
thread: string
) -> HumanTools | CalculatorTools {
client "openai/gpt-4o"
prompt #"
{{ _.role("system") }}
You are a helpful assistant that can help with tasks.
{{ _.role("user") }}
You are working on the following thread:
{{ thread }}
What should the next step be?
{{ ctx.output_format }}
"#
}
test HelloWorld {
functions [DetermineNextStep]
args {
thread #"
{
"type": "user_input",
"data": "hello!"
}
"#
}
@@assert(intent, {{this.intent == "request_more_information"}})
}
test MathOperation {
functions [DetermineNextStep]
args {
thread #"
{
"type": "user_input",
"data": "can you multiply 3 and 4?"
}
"#
}
@@assert(intent, {{this.intent == "multiply"}})
}
test LongMath {
functions [DetermineNextStep]
args {
thread #"
[
{
"type": "user_input",
"data": "can you multiply 3 and 4, then divide the result by 2 and then add 12 to that result?"
},
{
"type": "tool_call",
"data": {
"intent": "multiply",
"a": 3,
"b": 4
}
},
{
"type": "tool_response",
"data": 12
},
{
"type": "tool_call",
"data": {
"intent": "divide",
"a": 12,
"b": 2
}
},
{
"type": "tool_response",
"data": 6
},
{
"type": "tool_call",
"data": {
"intent": "add",
"a": 6,
"b": 12
}
},
{
"type": "tool_response",
"data": 18
}
]
"#
}
@@assert(intent, {{this.intent == "done_for_now"}})
@@assert(answer, {{"18" in this.message}})
}
test MathOperationWithClarification {
functions [DetermineNextStep]
args {
thread #"
[{"type":"user_input","data":"can you multiply 3 and feee9ff10"}]
"#
}
@@assert(intent, {{this.intent == "request_more_information"}})
}
test MathOperationPostClarification {
functions [DetermineNextStep]
args {
thread #"
[
{"type":"user_input","data":"can you multiply 3 and FD*(#F&& ?"},
{"type":"tool_call","data":{"intent":"request_more_information","message":"It seems like there was a typo or mistake in your request. Could you please clarify or provide the correct numbers you would like to multiply?"}},
{"type":"human_response","data":"lets try 12 instead"},
]
"#
}
@@assert(intent, {{this.intent == "multiply"}})
@@assert(a, {{this.b == 12}})
@@assert(b, {{this.a == 3}})
}

152
workshops/2025-05-17/walkthrough/06-agent.baml Normal file
View File

@@ -0,0 +1,152 @@
// human tools are async requests to a human
type HumanTools = ClarificationRequest | DoneForNow
class ClarificationRequest {
intent "request_more_information" @description("you can request more information from me")
message string
}
class DoneForNow {
intent "done_for_now"
message string @description(#"
message to send to the user about the work that was done.
"#)
}
function DetermineNextStep(
thread: string
) -> HumanTools | CalculatorTools {
client "openai/gpt-4o"
prompt #"
{{ _.role("system") }}
You are a helpful assistant that can help with tasks.
{{ _.role("user") }}
You are working on the following thread:
{{ thread }}
What should the next step be?
{{ ctx.output_format }}
First, always plan out what to do next, for example:
- ...
- ...
- ...
{...} // schema
"#
}
test HelloWorld {
functions [DetermineNextStep]
args {
thread #"
{
"type": "user_input",
"data": "hello!"
}
"#
}
@@assert(intent, {{this.intent == "request_more_information"}})
}
test MathOperation {
functions [DetermineNextStep]
args {
thread #"
{
"type": "user_input",
"data": "can you multiply 3 and 4?"
}
"#
}
@@assert(intent, {{this.intent == "multiply"}})
}
test LongMath {
functions [DetermineNextStep]
args {
thread #"
[
{
"type": "user_input",
"data": "can you multiply 3 and 4, then divide the result by 2 and then add 12 to that result?"
},
{
"type": "tool_call",
"data": {
"intent": "multiply",
"a": 3,
"b": 4
}
},
{
"type": "tool_response",
"data": 12
},
{
"type": "tool_call",
"data": {
"intent": "divide",
"a": 12,
"b": 2
}
},
{
"type": "tool_response",
"data": 6
},
{
"type": "tool_call",
"data": {
"intent": "add",
"a": 6,
"b": 12
}
},
{
"type": "tool_response",
"data": 18
}
]
"#
}
@@assert(intent, {{this.intent == "done_for_now"}})
@@assert(answer, {{"18" in this.message}})
}
test MathOperationWithClarification {
functions [DetermineNextStep]
args {
thread #"
[{"type":"user_input","data":"can you multiply 3 and feee9ff10"}]
"#
}
@@assert(intent, {{this.intent == "request_more_information"}})
}
test MathOperationPostClarification {
functions [DetermineNextStep]
args {
thread #"
[
{"type":"user_input","data":"can you multiply 3 and FD*(#F&& ?"},
{"type":"tool_call","data":{"intent":"request_more_information","message":"It seems like there was a typo or mistake in your request. Could you please clarify or provide the correct numbers you would like to multiply?"}},
{"type":"human_response","data":"lets try 12 instead"},
]
"#
}
@@assert(intent, {{this.intent == "multiply"}})
@@assert(a, {{this.b == 12}})
@@assert(b, {{this.a == 3}})
}

87
workshops/2025-05-17/walkthrough/07-agent.ts Normal file
View File

@@ -0,0 +1,87 @@
import { AddTool, SubtractTool, DivideTool, MultiplyTool, b } from "../baml_client";
export interface Event {
type: string
data: any;
}
export class Thread {
events: Event[] = [];
constructor(events: Event[]) {
this.events = events;
}
serializeForLLM() {
// can change this to whatever custom serialization you want to do, XML, etc
// e.g. https://github.com/got-agents/agents/blob/59ebbfa236fc376618f16ee08eb0f3bf7b698892/linear-assistant-ts/src/agent.ts#L66-L105
return JSON.stringify(this.events, null, 2);
}
}
export type CalculatorTool = AddTool | SubtractTool | MultiplyTool | DivideTool;
export async function handleNextStep(nextStep: CalculatorTool, thread: Thread): Promise<Thread> {
let result: number;
switch (nextStep.intent) {
case "add":
result = nextStep.a + nextStep.b;
console.log("tool_response", result);
thread.events.push({
"type": "tool_response",
"data": result
});
return thread;
case "subtract":
result = nextStep.a - nextStep.b;
console.log("tool_response", result);
thread.events.push({
"type": "tool_response",
"data": result
});
return thread;
case "multiply":
result = nextStep.a * nextStep.b;
console.log("tool_response", result);
thread.events.push({
"type": "tool_response",
"data": result
});
return thread;
case "divide":
result = nextStep.a / nextStep.b;
console.log("tool_response", result);
thread.events.push({
"type": "tool_response",
"data": result
});
return thread;
}
}
export async function agentLoop(thread: Thread): Promise<Thread> {
while (true) {
const nextStep = await b.DetermineNextStep(thread.serializeForLLM());
console.log("nextStep", nextStep);
thread.events.push({
"type": "tool_call",
"data": nextStep
});
switch (nextStep.intent) {
case "done_for_now":
case "request_more_information":
// response to human, return the thread
return thread;
case "add":
case "subtract":
case "multiply":
case "divide":
thread = await handleNextStep(nextStep, thread);
}
}
}

99
workshops/2025-05-17/walkthrough/07b-agent.ts Normal file
View File

@@ -0,0 +1,99 @@
import { AddTool, SubtractTool, DivideTool, MultiplyTool, b } from "../baml_client";
export interface Event {
type: string
data: any;
}
export class Thread {
events: Event[] = [];
constructor(events: Event[]) {
this.events = events;
}
serializeForLLM() {
return this.events.map(e => this.serializeOneEvent(e)).join("\n");
}
trimLeadingWhitespace(s: string) {
return s.replace(/^[ \t]+/gm, '');
}
serializeOneEvent(e: Event) {
return this.trimLeadingWhitespace(`
<${e.data?.intent || e.type}>
${
typeof e.data !== 'object' ? e.data :
Object.keys(e.data).filter(k => k !== 'intent').map(k => `${k}: ${e.data[k]}`).join("\n")}
</${e.data?.intent || e.type}>
`)
}
}
export type CalculatorTool = AddTool | SubtractTool | MultiplyTool | DivideTool;
export async function handleNextStep(nextStep: CalculatorTool, thread: Thread): Promise<Thread> {
let result: number;
switch (nextStep.intent) {
case "add":
result = nextStep.a + nextStep.b;
console.log("tool_response", result);
thread.events.push({
"type": "tool_response",
"data": result
});
return thread;
case "subtract":
result = nextStep.a - nextStep.b;
console.log("tool_response", result);
thread.events.push({
"type": "tool_response",
"data": result
});
return thread;
case "multiply":
result = nextStep.a * nextStep.b;
console.log("tool_response", result);
thread.events.push({
"type": "tool_response",
"data": result
});
return thread;
case "divide":
result = nextStep.a / nextStep.b;
console.log("tool_response", result);
thread.events.push({
"type": "tool_response",
"data": result
});
return thread;
}
}
export async function agentLoop(thread: Thread): Promise<Thread> {
while (true) {
const nextStep = await b.DetermineNextStep(thread.serializeForLLM());
console.log("nextStep", nextStep);
thread.events.push({
"type": "tool_call",
"data": nextStep
});
switch (nextStep.intent) {
case "done_for_now":
case "request_more_information":
// response to human, return the thread
return thread;
case "add":
case "subtract":
case "multiply":
case "divide":
thread = await handleNextStep(nextStep, thread);
}
}
}

153
workshops/2025-05-17/walkthrough/07c-agent.baml Normal file
View File

@@ -0,0 +1,153 @@
// human tools are async requests to a human
type HumanTools = ClarificationRequest | DoneForNow
class ClarificationRequest {
intent "request_more_information" @description("you can request more information from me")
message string
}
class DoneForNow {
intent "done_for_now"
message string @description(#"
message to send to the user about the work that was done.
"#)
}
function DetermineNextStep(
thread: string
) -> HumanTools | CalculatorTools {
client "openai/gpt-4o"
prompt #"
{{ _.role("system") }}
You are a helpful assistant that can help with tasks.
{{ _.role("user") }}
You are working on the following thread:
{{ thread }}
What should the next step be?
{{ ctx.output_format }}
Always think about what to do next first, like:
- ...
- ...
- ...
{...} // schema
"#
}
test HelloWorld {
functions [DetermineNextStep]
args {
thread #"
<user_input>
hello!
</user_input>
"#
}
@@assert(intent, {{this.intent == "request_more_information"}})
}
test MathOperation {
functions [DetermineNextStep]
args {
thread #"
<user_input>
can you multiply 3 and 4?
</user_input>
"#
}
@@assert(intent, {{this.intent == "multiply"}})
}
test LongMath {
functions [DetermineNextStep]
args {
thread #"
<user_input>
can you multiply 3 and 4, then divide the result by 2 and then add 12 to that result?
</user_input>
<multiply>
a: 3
b: 4
</multiply>
<tool_response>
12
</tool_response>
<divide>
a: 12
b: 2
</divide>
<tool_response>
6
</tool_response>
<add>
a: 6
b: 12
</add>
<tool_response>
18
</tool_response>
"#
}
@@assert(intent, {{this.intent == "done_for_now"}})
@@assert(answer, {{"18" in this.message}})
}
test MathOperationWithClarification {
functions [DetermineNextStep]
args {
thread #"
<user_input>
can you multiply 3 and fe1iiaff10
</user_input>
"#
}
@@assert(intent, {{this.intent == "request_more_information"}})
}
test MathOperationPostClarification {
functions [DetermineNextStep]
args {
thread #"
<user_input>
can you multiply 3 and FD*(#F&& ?
</user_input>
<request_more_information>
message: It seems like there was a typo or mistake in your request. Could you please clarify or provide the correct numbers you would like to multiply?
</request_more_information>
<human_response>
lets try 12 instead
</human_response>
"#
}
@@assert(intent, {{this.intent == "multiply"}})
@@assert(b, {{this.a == 3}})
@@assert(a, {{this.b == 12}})
}

29
workshops/2025-05-17/walkthrough/08-server.ts Normal file
View File

@@ -0,0 +1,29 @@
import express from 'express';
import { Thread, agentLoop } from '../src/agent';
const app = express();
app.use(express.json());
app.set('json spaces', 2);
// POST /thread - Start new thread
app.post('/thread', async (req, res) => {
const thread = new Thread([{
type: "user_input",
data: req.body.message
}]);
const result = await agentLoop(thread);
res.json(result);
});
// GET /thread/:id - Get thread status
app.get('/thread/:id', (req, res) => {
// optional - add state
res.status(404).json({ error: "Not implemented yet" });
});
const port = process.env.PORT || 3000;
app.listen(port, () => {
console.log(`Server running on port ${port}`);
});
export { app };

75
workshops/2025-05-17/walkthrough/09-server.ts Normal file
View File

@@ -0,0 +1,75 @@
import express from 'express';
import { Thread, agentLoop } from '../src/agent';
import { ThreadStore } from '../src/state';
const app = express();
app.use(express.json());
app.set('json spaces', 2);
const store = new ThreadStore();
// POST /thread - Start new thread
app.post('/thread', async (req, res) => {
const thread = new Thread([{
type: "user_input",
data: req.body.message
}]);
const threadId = store.create(thread);
const newThread = await agentLoop(thread);
store.update(threadId, newThread);
const lastEvent = newThread.events[newThread.events.length - 1];
// If we exited the loop, include the response URL so the client can
// push a new message onto the thread
lastEvent.data.response_url = `/thread/${threadId}/response`;
console.log("returning last event from endpoint", lastEvent);
res.json({
thread_id: threadId,
...newThread
});
});
// GET /thread/:id - Get thread status
app.get('/thread/:id', (req, res) => {
const thread = store.get(req.params.id);
if (!thread) {
return res.status(404).json({ error: "Thread not found" });
}
res.json(thread);
});
// POST /thread/:id/response - Handle clarification response
app.post('/thread/:id/response', async (req, res) => {
let thread = store.get(req.params.id);
if (!thread) {
return res.status(404).json({ error: "Thread not found" });
}
thread.events.push({
type: "human_response",
data: req.body.message
});
// loop until stop event
const newThread = await agentLoop(thread);
store.update(req.params.id, newThread);
const lastEvent = newThread.events[newThread.events.length - 1];
lastEvent.data.response_url = `/thread/${req.params.id}/response`;
console.log("returning last event from endpoint", lastEvent);
res.json(newThread);
});
const port = process.env.PORT || 3000;
app.listen(port, () => {
console.log(`Server running on port ${port}`);
});
export { app };

23
workshops/2025-05-17/walkthrough/09-state.ts Normal file
View File

@@ -0,0 +1,23 @@
import crypto from 'crypto';
import { Thread } from '../src/agent';
// you can replace this with any simple state management,
// e.g. redis, sqlite, postgres, etc
export class ThreadStore {
private threads: Map<string, Thread> = new Map();
create(thread: Thread): string {
const id = crypto.randomUUID();
this.threads.set(id, thread);
return id;
}
get(id: string): Thread | undefined {
return this.threads.get(id);
}
update(id: string, thread: Thread): void {
this.threads.set(id, thread);
}
}

111
workshops/2025-05-17/walkthrough/10-agent.ts Normal file
View File

@@ -0,0 +1,111 @@
import { AddTool, SubtractTool, DivideTool, MultiplyTool, b } from "../baml_client";
export interface Event {
type: string
data: any;
}
export class Thread {
events: Event[] = [];
constructor(events: Event[]) {
this.events = events;
}
serializeForLLM() {
return this.events.map(e => this.serializeOneEvent(e)).join("\n");
}
trimLeadingWhitespace(s: string) {
return s.replace(/^[ \t]+/gm, '');
}
serializeOneEvent(e: Event) {
return this.trimLeadingWhitespace(`
<${e.data?.intent || e.type}>
${
typeof e.data !== 'object' ? e.data :
Object.keys(e.data).filter(k => k !== 'intent').map(k => `${k}: ${e.data[k]}`).join("\n")}
</${e.data?.intent || e.type}>
`)
}
awaitingHumanResponse(): boolean {
const lastEvent = this.events[this.events.length - 1];
return ['request_more_information', 'done_for_now'].includes(lastEvent.data.intent);
}
awaitingHumanApproval(): boolean {
const lastEvent = this.events[this.events.length - 1];
return lastEvent.data.intent === 'divide';
}
}
export type CalculatorTool = AddTool | SubtractTool | MultiplyTool | DivideTool;
export async function handleNextStep(nextStep: CalculatorTool, thread: Thread): Promise<Thread> {
let result: number;
switch (nextStep.intent) {
case "add":
result = nextStep.a + nextStep.b;
console.log("tool_response", result);
thread.events.push({
"type": "tool_response",
"data": result
});
return thread;
case "subtract":
result = nextStep.a - nextStep.b;
console.log("tool_response", result);
thread.events.push({
"type": "tool_response",
"data": result
});
return thread;
case "multiply":
result = nextStep.a * nextStep.b;
console.log("tool_response", result);
thread.events.push({
"type": "tool_response",
"data": result
});
return thread;
case "divide":
result = nextStep.a / nextStep.b;
console.log("tool_response", result);
thread.events.push({
"type": "tool_response",
"data": result
});
return thread;
}
}
export async function agentLoop(thread: Thread): Promise<Thread> {
while (true) {
const nextStep = await b.DetermineNextStep(thread.serializeForLLM());
console.log("nextStep", nextStep);
thread.events.push({
"type": "tool_call",
"data": nextStep
});
switch (nextStep.intent) {
case "done_for_now":
case "request_more_information":
// response to human, return the thread
return thread;
case "divide":
// divide is scary, return it for human approval
return thread;
case "add":
case "subtract":
case "multiply":
thread = await handleNextStep(nextStep, thread);
}
}
}

112
workshops/2025-05-17/walkthrough/10-server.ts Normal file
View File

@@ -0,0 +1,112 @@
import express from 'express';
import { Thread, agentLoop, handleNextStep } from '../src/agent';
import { ThreadStore } from '../src/state';
const app = express();
app.use(express.json());
app.set('json spaces', 2);
const store = new ThreadStore();
// POST /thread - Start new thread
app.post('/thread', async (req, res) => {
const thread = new Thread([{
type: "user_input",
data: req.body.message
}]);
const threadId = store.create(thread);
const newThread = await agentLoop(thread);
store.update(threadId, newThread);
const lastEvent = newThread.events[newThread.events.length - 1];
// If we exited the loop, include the response URL so the client can
// push a new message onto the thread
lastEvent.data.response_url = `/thread/${threadId}/response`;
console.log("returning last event from endpoint", lastEvent);
res.json({
thread_id: threadId,
...newThread
});
});
// GET /thread/:id - Get thread status
app.get('/thread/:id', (req, res) => {
const thread = store.get(req.params.id);
if (!thread) {
return res.status(404).json({ error: "Thread not found" });
}
res.json(thread);
});
type ApprovalPayload = {
type: "approval";
approved: boolean;
comment?: string;
}
type ResponsePayload = {
type: "response";
response: string;
}
type Payload = ApprovalPayload | ResponsePayload;
// POST /thread/:id/response - Handle clarification response
app.post('/thread/:id/response', async (req, res) => {
let thread = store.get(req.params.id);
if (!thread) {
return res.status(404).json({ error: "Thread not found" });
}
const body: Payload = req.body;
let lastEvent = thread.events[thread.events.length - 1];
if (thread.awaitingHumanResponse() && body.type === 'response') {
thread.events.push({
type: "human_response",
data: body.response
});
} else if (thread.awaitingHumanApproval() && body.type === 'approval' && !body.approved) {
// push feedback onto the thread
thread.events.push({
type: "tool_response",
data: `user denied the operation with feedback: "${body.comment}"`
});
} else if (thread.awaitingHumanApproval() && body.type === 'approval' && body.approved) {
// approved, run the tool, pushing results onto the thread
await handleNextStep(lastEvent.data, thread);
} else {
res.status(400).json({
error: "Invalid request: " + body.type,
awaitingHumanResponse: thread.awaitingHumanResponse(),
awaitingHumanApproval: thread.awaitingHumanApproval()
});
return;
}
// loop until stop event
const newThread = await agentLoop(thread);
store.update(req.params.id, newThread);
lastEvent = newThread.events[newThread.events.length - 1];
lastEvent.data.response_url = `/thread/${req.params.id}/response`;
console.log("returning last event from endpoint", lastEvent);
res.json(newThread);
});
const port = process.env.PORT || 3000;
app.listen(port, () => {
console.log(`Server running on port ${port}`);
});
export { app };

103
workshops/2025-05-17/walkthrough/11-cli.ts Normal file
View File

@@ -0,0 +1,103 @@
// cli.ts lets you invoke the agent loop from the command line
import { humanlayer } from "humanlayer";
import { agentLoop, Thread, Event } from "../src/agent";
export async function cli() {
// Get command line arguments, skipping the first two (node and script name)
const args = process.argv.slice(2);
if (args.length === 0) {
console.error("Error: Please provide a message as a command line argument");
process.exit(1);
}
// Join all arguments into a single message
const message = args.join(" ");
// Create a new thread with the user's message as the initial event
const thread = new Thread([{ type: "user_input", data: message }]);
// Run the agent loop with the thread
let newThread = await agentLoop(thread);
let lastEvent = newThread.events.slice(-1)[0];
while (lastEvent.data.intent !== "done_for_now") {
const responseEvent = await askHuman(lastEvent);
thread.events.push(responseEvent);
newThread = await agentLoop(thread);
lastEvent = newThread.events.slice(-1)[0];
}
// print the final result
// optional - you could loop here too
console.log(lastEvent.data.message);
process.exit(0);
}
async function askHuman(lastEvent: Event): Promise<Event> {
if (process.env.HUMANLAYER_API_KEY) {
return await askHumanEmail(lastEvent);
} else {
return await askHumanCLI(lastEvent.data.message);
}
}
async function askHumanCLI(message: string): Promise<Event> {
const readline = require('readline').createInterface({
input: process.stdin,
output: process.stdout
});
return new Promise((resolve) => {
readline.question(`${message}\n> `, (answer: string) => {
resolve({ type: "human_response", data: answer });
});
});
}
export async function askHumanEmail(lastEvent: Event): Promise<Event> {
if (!process.env.HUMANLAYER_EMAIL) {
throw new Error("missing or invalid parameters: HUMANLAYER_EMAIL");
}
const hl = humanlayer({ //reads apiKey from env
// name of this agent
runId: "12fa-cli-agent",
verbose: true,
contactChannel: {
// agent should request permission via email
email: {
address: process.env.HUMANLAYER_EMAIL,
}
}
})
if (lastEvent.data.intent === "divide") {
// fetch approval synchronously - this will block until reply
const response = await hl.fetchHumanApproval({
spec: {
fn: "divide",
kwargs: {
a: lastEvent.data.a,
b: lastEvent.data.b
}
}
})
if (response.approved) {
const result = lastEvent.data.a / lastEvent.data.b;
console.log("tool_response", result);
return {
"type": "tool_response",
"data": result
};
} else {
return {
"type": "tool_response",
"data": `user denied operation ${lastEvent.data.intent}
with feedback: ${response.comment}`
};
}
}
throw new Error(`unknown tool: ${lastEvent.data.intent}`)
}

BIN
workshops/2025-05-17/walkthrough/11-email-approve.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 178 KiB

BIN
workshops/2025-05-17/walkthrough/11-email-custom.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 66 KiB

BIN
workshops/2025-05-17/walkthrough/11-email-reject.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 182 KiB

116
workshops/2025-05-17/walkthrough/11b-cli.ts Normal file
View File

@@ -0,0 +1,116 @@
// cli.ts lets you invoke the agent loop from the command line
import { humanlayer } from "humanlayer";
import { agentLoop, Thread, Event } from "../src/agent";
export async function cli() {
// Get command line arguments, skipping the first two (node and script name)
const args = process.argv.slice(2);
if (args.length === 0) {
console.error("Error: Please provide a message as a command line argument");
process.exit(1);
}
// Join all arguments into a single message
const message = args.join(" ");
// Create a new thread with the user's message as the initial event
const thread = new Thread([{ type: "user_input", data: message }]);
// Run the agent loop with the thread
let newThread = await agentLoop(thread);
let lastEvent = newThread.events.slice(-1)[0];
while (lastEvent.data.intent !== "done_for_now") {
const responseEvent = await askHuman(lastEvent);
thread.events.push(responseEvent);
newThread = await agentLoop(thread);
lastEvent = newThread.events.slice(-1)[0];
}
// print the final result
// optional - you could loop here too
console.log(lastEvent.data.message);
process.exit(0);
}
async function askHuman(lastEvent: Event): Promise<Event> {
if (process.env.HUMANLAYER_API_KEY) {
return await askHumanEmail(lastEvent);
} else {
return await askHumanCLI(lastEvent.data.message);
}
}
async function askHumanCLI(message: string): Promise<Event> {
const readline = require('readline').createInterface({
input: process.stdin,
output: process.stdout
});
return new Promise((resolve) => {
readline.question(`${message}\n> `, (answer: string) => {
resolve({ type: "human_response", data: answer });
});
});
}
export async function askHumanEmail(lastEvent: Event): Promise<Event> {
if (!process.env.HUMANLAYER_EMAIL) {
throw new Error("missing or invalid parameters: HUMANLAYER_EMAIL");
}
const hl = humanlayer({ //reads apiKey from env
// name of this agent
runId: "12fa-cli-agent",
verbose: true,
contactChannel: {
// agent should request permission via email
email: {
address: process.env.HUMANLAYER_EMAIL,
}
}
})
if (lastEvent.data.intent === "request_more_information") {
// fetch response synchronously - this will block until reply
const response = await hl.fetchHumanResponse({
spec: {
msg: lastEvent.data.message
}
})
return {
"type": "tool_response",
"data": response
}
}
if (lastEvent.data.intent === "divide") {
// fetch approval synchronously - this will block until reply
const response = await hl.fetchHumanApproval({
spec: {
fn: "divide",
kwargs: {
a: lastEvent.data.a,
b: lastEvent.data.b
}
}
})
if (response.approved) {
const result = lastEvent.data.a / lastEvent.data.b;
console.log("tool_response", result);
return {
"type": "tool_response",
"data": result
};
} else {
return {
"type": "tool_response",
"data": `user denied operation ${lastEvent.data.intent}
with feedback: ${response.comment}`
};
}
}
throw new Error(`unknown tool: ${lastEvent.data.intent}`)
}

125
workshops/2025-05-17/walkthrough/11c-cli.ts Normal file
View File

@@ -0,0 +1,125 @@
// cli.ts lets you invoke the agent loop from the command line
import { humanlayer } from "humanlayer";
import { agentLoop, Thread, Event } from "../src/agent";
export async function cli() {
// Get command line arguments, skipping the first two (node and script name)
const args = process.argv.slice(2);
if (args.length === 0) {
console.error("Error: Please provide a message as a command line argument");
process.exit(1);
}
// Join all arguments into a single message
const message = args.join(" ");
// Create a new thread with the user's message as the initial event
const thread = new Thread([{ type: "user_input", data: message }]);
// Run the agent loop with the thread
let newThread = await agentLoop(thread);
let lastEvent = newThread.events.slice(-1)[0];
while (lastEvent.data.intent !== "done_for_now") {
const responseEvent = await askHuman(lastEvent);
thread.events.push(responseEvent);
newThread = await agentLoop(thread);
lastEvent = newThread.events.slice(-1)[0];
}
// print the final result
// optional - you could loop here too
console.log(lastEvent.data.message);
process.exit(0);
}
async function askHuman(lastEvent: Event): Promise<Event> {
if (process.env.HUMANLAYER_API_KEY) {
return await askHumanEmail(lastEvent);
} else {
return await askHumanCLI(lastEvent.data.message);
}
}
async function askHumanCLI(message: string): Promise<Event> {
const readline = require('readline').createInterface({
input: process.stdin,
output: process.stdout
});
return new Promise((resolve) => {
readline.question(`${message}\n> `, (answer: string) => {
resolve({ type: "human_response", data: answer });
});
});
}
export async function askHumanEmail(lastEvent: Event): Promise<Event> {
if (!process.env.HUMANLAYER_EMAIL) {
throw new Error("missing or invalid parameters: HUMANLAYER_EMAIL");
}
const hl = humanlayer({ //reads apiKey from env
// name of this agent
runId: "12fa-cli-agent",
verbose: true,
contactChannel: {
// agent should request permission via email
email: {
address: process.env.HUMANLAYER_EMAIL,
// custom email body - jinja
template: `{% if type == 'request_more_information' %}
{{ event.spec.msg }}
{% else %}
agent {{ event.run_id }} is requesting approval for {{event.spec.fn}}
with args: {{event.spec.kwargs}}
<br><br>
reply to this email to approve
{% endif %}`
}
}
})
if (lastEvent.data.intent === "request_more_information") {
// fetch response synchronously - this will block until reply
const response = await hl.fetchHumanResponse({
spec: {
msg: lastEvent.data.message
}
})
return {
"type": "tool_response",
"data": response
}
}
if (lastEvent.data.intent === "divide") {
// fetch approval synchronously - this will block until reply
const response = await hl.fetchHumanApproval({
spec: {
fn: "divide",
kwargs: {
a: lastEvent.data.a,
b: lastEvent.data.b
}
}
})
if (response.approved) {
const result = lastEvent.data.a / lastEvent.data.b;
console.log("tool_response", result);
return {
"type": "tool_response",
"data": result
};
} else {
return {
"type": "tool_response",
"data": `user denied operation ${lastEvent.data.intent}
with feedback: ${response.comment}`
};
}
}
throw new Error(`unknown tool: ${lastEvent.data.intent}`)
}

131
workshops/2025-05-17/walkthrough/12-1-server-init.ts Normal file
View File

@@ -0,0 +1,131 @@
import express from 'express';
import { Thread, agentLoop, handleNextStep } from '../src/agent';
import { ThreadStore } from '../src/state';
import { humanlayer } from 'humanlayer';
const app = express();
app.use(express.json());
app.set('json spaces', 2);
const store = new ThreadStore();
const getHumanlayer = () => {
const HUMANLAYER_EMAIL = process.env.HUMANLAYER_EMAIL;
if (!HUMANLAYER_EMAIL) {
throw new Error("missing or invalid parameters: HUMANLAYER_EMAIL");
}
const HUMANLAYER_API_KEY = process.env.HUMANLAYER_API_KEY;
if (!HUMANLAYER_API_KEY) {
throw new Error("missing or invalid parameters: HUMANLAYER_API_KEY");
}
return humanlayer({
runId: `12fa-agent`,
contactChannel: {
email: { address: HUMANLAYER_EMAIL }
}
});
}
// POST /thread - Start new thread
app.post('/thread', async (req, res) => {
const thread = new Thread([{
type: "user_input",
data: req.body.message
}]);
const threadId = store.create(thread);
const newThread = await agentLoop(thread);
store.update(threadId, newThread);
const lastEvent = newThread.events[newThread.events.length - 1];
// If we exited the loop, include the response URL so the client can
// push a new message onto the thread
lastEvent.data.response_url = `/thread/${threadId}/response`;
console.log("returning last event from endpoint", lastEvent);
res.json({
thread_id: threadId,
...newThread
});
});
// GET /thread/:id - Get thread status
app.get('/thread/:id', (req, res) => {
const thread = store.get(req.params.id);
if (!thread) {
return res.status(404).json({ error: "Thread not found" });
}
res.json(thread);
});
type ApprovalPayload = {
type: "approval";
approved: boolean;
comment?: string;
}
type ResponsePayload = {
type: "response";
response: string;
}
type Payload = ApprovalPayload | ResponsePayload;
// POST /thread/:id/response - Handle clarification response
app.post('/thread/:id/response', async (req, res) => {
let thread = store.get(req.params.id);
if (!thread) {
return res.status(404).json({ error: "Thread not found" });
}
const body: Payload = req.body;
let lastEvent = thread.events[thread.events.length - 1];
if (thread.awaitingHumanResponse() && body.type === 'response') {
thread.events.push({
type: "human_response",
data: body.response
});
} else if (thread.awaitingHumanApproval() && body.type === 'approval' && !body.approved) {
// push feedback onto the thread
thread.events.push({
type: "tool_response",
data: `user denied the operation with feedback: "${body.comment}"`
});
} else if (thread.awaitingHumanApproval() && body.type === 'approval' && body.approved) {
// approved, run the tool, pushing results onto the thread
await handleNextStep(lastEvent.data, thread);
} else {
res.status(400).json({
error: "Invalid request: " + body.type,
awaitingHumanResponse: thread.awaitingHumanResponse(),
awaitingHumanApproval: thread.awaitingHumanApproval()
});
return;
}
// loop until stop event
const result = await agentLoop(thread);
store.update(req.params.id, result);
lastEvent = result.events[result.events.length - 1];
lastEvent.data.response_url = `/thread/${req.params.id}/response`;
console.log("returning last event from endpoint", lastEvent);
res.json(result);
});
const port = process.env.PORT || 3000;
app.listen(port, () => {
console.log(`Server running on port ${port}`);
});
export { app };

106
workshops/2025-05-17/walkthrough/12-server.ts Normal file
View File

@@ -0,0 +1,106 @@
import express, { Request, Response } from 'express';
import { Thread, agentLoop, handleNextStep } from '../src/agent';
import { ThreadStore } from '../src/state';
import { humanlayer, V1Beta2HumanContactCompleted } from 'humanlayer';
const app = express();
app.use(express.json());
app.set('json spaces', 2);
const store = new ThreadStore();
const getHumanlayer = () => {
const HUMANLAYER_EMAIL = process.env.HUMANLAYER_EMAIL;
if (!HUMANLAYER_EMAIL) {
throw new Error("missing or invalid parameters: HUMANLAYER_EMAIL");
}
const HUMANLAYER_API_KEY = process.env.HUMANLAYER_API_KEY;
if (!HUMANLAYER_API_KEY) {
throw new Error("missing or invalid parameters: HUMANLAYER_API_KEY");
}
return humanlayer({
runId: `12fa-agent`,
contactChannel: {
email: { address: HUMANLAYER_EMAIL }
}
});
}
// POST /thread - Start new thread
app.post('/thread', async (req: Request, res: Response) => {
const thread = new Thread([{
type: "user_input",
data: req.body.message
}]);
// run agent loop asynchronously, return immediately
Promise.resolve().then(async () => {
const threadId = store.create(thread);
const newThread = await agentLoop(thread);
store.update(threadId, newThread);
const lastEvent = newThread.events[newThread.events.length - 1];
if (thread.awaitingHumanResponse()) {
const hl = getHumanlayer();
// create a human contact - returns immediately
hl.createHumanContact({
spec: {
msg: lastEvent.data.message,
state: {
thread_id: threadId,
}
}
});
}
});
res.json({ status: "processing" });
});
// GET /thread/:id - Get thread status
app.get('/thread/:id', (req: Request, res: Response) => {
const thread = store.get(req.params.id);
if (!thread) {
return res.status(404).json({ error: "Thread not found" });
}
res.json(thread);
});
type WebhookResponse = V1Beta2HumanContactCompleted;
const handleHumanResponse = async (req: Request, res: Response) => {
}
app.post('/webhook', async (req: Request, res: Response) => {
console.log("webhook response", req.body);
const response = req.body as WebhookResponse;
// response is guaranteed to be set on a webhook
const humanResponse: string = response.event.status?.response as string;
const threadId = response.event.spec.state?.thread_id;
if (!threadId) {
return res.status(400).json({ error: "Thread ID not found" });
}
const thread = store.get(threadId);
if (!thread) {
return res.status(404).json({ error: "Thread not found" });
}
if (!thread.awaitingHumanResponse()) {
return res.status(400).json({ error: "Thread is not awaiting human response" });
}
});
const port = process.env.PORT || 3000;
app.listen(port, () => {
console.log(`Server running on port ${port}`);
});
export { app };

105
workshops/2025-05-17/walkthrough/12a-server.ts Normal file
View File

@@ -0,0 +1,105 @@
import express, { Request, Response } from 'express';
import { Thread, agentLoop, handleNextStep } from '../src/agent';
import { ThreadStore } from '../src/state';
import { humanlayer, V1Beta2HumanContactCompleted } from 'humanlayer';
const app = express();
app.use(express.json());
app.set('json spaces', 2);
const store = new ThreadStore();
const getHumanlayer = () => {
const HUMANLAYER_EMAIL = process.env.HUMANLAYER_EMAIL;
if (!HUMANLAYER_EMAIL) {
throw new Error("missing or invalid parameters: HUMANLAYER_EMAIL");
}
const HUMANLAYER_API_KEY = process.env.HUMANLAYER_API_KEY;
if (!HUMANLAYER_API_KEY) {
throw new Error("missing or invalid parameters: HUMANLAYER_API_KEY");
}
return humanlayer({
runId: `12fa-agent`,
contactChannel: {
email: { address: HUMANLAYER_EMAIL }
}
});
}
// POST /thread - Start new thread
app.post('/thread', async (req: Request, res: Response) => {
const thread = new Thread([{
type: "user_input",
data: req.body.message
}]);
// run agent loop asynchronously, return immediately
Promise.resolve().then(async () => {
const threadId = store.create(thread);
const newThread = await agentLoop(thread);
store.update(threadId, newThread);
const lastEvent = newThread.events[newThread.events.length - 1];
if (thread.awaitingHumanResponse()) {
const hl = getHumanlayer();
// create a human contact - returns immediately
hl.createHumanContact({
spec: {
msg: lastEvent.data.message,
state: {
thread_id: threadId,
}
}
});
}
});
res.json({ status: "processing" });
});
// GET /thread/:id - Get thread status
app.get('/thread/:id', (req: Request, res: Response) => {
const thread = store.get(req.params.id);
if (!thread) {
return res.status(404).json({ error: "Thread not found" });
}
res.json(thread);
});
type WebhookResponse = V1Beta2HumanContactCompleted;
const handleHumanResponse = async (req: Request, res: Response) => {
}
app.post('/webhook', async (req: Request, res: Response) => {
console.log("webhook response", req.body);
const response = req.body as WebhookResponse;
// response is guaranteed to be set on a webhook
const humanResponse: string = response.event.status?.response as string;
const threadId = response.event.spec.state?.thread_id;
if (!threadId) {
return res.status(400).json({ error: "Thread ID not found" });
}
const thread = store.get(threadId);
if (!thread) {
return res.status(404).json({ error: "Thread not found" });
}
if (!thread.awaitingHumanResponse()) {
return res.status(400).json({ error: "Thread is not awaiting human response" });
}
});
const port = process.env.PORT || 3000;
app.listen(port, () => {
console.log(`Server running on port ${port}`);
});
export { app };

106
workshops/2025-05-17/walkthrough/12aa-server.ts Normal file
View File

@@ -0,0 +1,106 @@
import express, { Request, Response } from 'express';
import { Thread, agentLoop, handleNextStep } from '../src/agent';
import { ThreadStore } from '../src/state';
import { humanlayer, V1Beta2HumanContactCompleted } from 'humanlayer';
const app = express();
app.use(express.json());
app.set('json spaces', 2);
const store = new ThreadStore();
const getHumanlayer = () => {
const HUMANLAYER_EMAIL = process.env.HUMANLAYER_EMAIL;
if (!HUMANLAYER_EMAIL) {
throw new Error("missing or invalid parameters: HUMANLAYER_EMAIL");
}
const HUMANLAYER_API_KEY = process.env.HUMANLAYER_API_KEY;
if (!HUMANLAYER_API_KEY) {
throw new Error("missing or invalid parameters: HUMANLAYER_API_KEY");
}
return humanlayer({
runId: `12fa-agent`,
contactChannel: {
email: { address: HUMANLAYER_EMAIL }
}
});
}
// POST /thread - Start new thread
app.post('/thread', async (req: Request, res: Response) => {
const thread = new Thread([{
type: "user_input",
data: req.body.message
}]);
// run agent loop asynchronously, return immediately
Promise.resolve().then(async () => {
const threadId = store.create(thread);
const newThread = await agentLoop(thread);
store.update(threadId, newThread);
const lastEvent = newThread.events[newThread.events.length - 1];
if (thread.awaitingHumanResponse()) {
const hl = getHumanlayer();
// create a human contact - returns immediately
hl.createHumanContact({
spec: {
msg: lastEvent.data.message,
state: {
thread_id: threadId,
}
}
});
}
});
res.json({ status: "processing" });
});
// GET /thread/:id - Get thread status
app.get('/thread/:id', (req: Request, res: Response) => {
const thread = store.get(req.params.id);
if (!thread) {
return res.status(404).json({ error: "Thread not found" });
}
res.json(thread);
});
type WebhookResponse = V1Beta2HumanContactCompleted;
const handleHumanResponse = async (req: Request, res: Response) => {
}
app.post('/webhook', async (req: Request, res: Response) => {
console.log("webhook response", req.body);
const response = req.body as WebhookResponse;
// response is guaranteed to be set on a webhook
const humanResponse: string = response.event.status?.response as string;
const threadId = response.event.spec.state?.thread_id;
if (!threadId) {
return res.status(400).json({ error: "Thread ID not found" });
}
const thread = store.get(threadId);
if (!thread) {
return res.status(404).json({ error: "Thread not found" });
}
if (!thread.awaitingHumanResponse()) {
return res.status(400).json({ error: "Thread is not awaiting human response" });
}
});
const port = process.env.PORT || 3000;
app.listen(port, () => {
console.log(`Server running on port ${port}`);
});
export { app };

143
workshops/2025-05-17/walkthrough/12b-server.ts Normal file
View File

@@ -0,0 +1,143 @@
import express from 'express';
import { Thread, agentLoop, handleNextStep } from '../src/agent';
import { ThreadStore } from '../src/state';
import { V1Beta2EmailEventReceived, V1Beta2FunctionCallCompleted, V1Beta2HumanContactCompleted } from 'humanlayer';
const app = express();
app.use(express.json());
app.set('json spaces', 2);
const store = new ThreadStore();
// POST /thread - Start new thread
app.post('/thread', async (req, res) => {
const thread = new Thread([{
type: "user_input",
data: req.body.message
}]);
const threadId = store.create(thread);
const newThread = await agentLoop(thread);
store.update(threadId, newThread);
const lastEvent = newThread.events[newThread.events.length - 1];
// If we exited the loop, include the response URL so the client can
// push a new message onto the thread
lastEvent.data.response_url = `/thread/${threadId}/response`;
console.log("returning last event from endpoint", lastEvent);
res.json({
thread_id: threadId,
...newThread
});
});
// GET /thread/:id - Get thread status
app.get('/thread/:id', (req, res) => {
const thread = store.get(req.params.id);
if (!thread) {
return res.status(404).json({ error: "Thread not found" });
}
res.json(thread);
});
type ApprovalPayload = {
type: "approval";
approved: boolean;
comment?: string;
}
type ResponsePayload = {
type: "response";
response: string;
}
type Payload = ApprovalPayload | ResponsePayload;
// POST /thread/:id/response - Handle clarification response
app.post('/thread/:id/response', async (req, res) => {
let thread = store.get(req.params.id);
if (!thread) {
return res.status(404).json({ error: "Thread not found" });
}
const body: Payload = req.body;
let lastEvent = thread.events[thread.events.length - 1];
if (thread.awaitingHumanResponse() && body.type === 'response') {
thread.events.push({
type: "human_response",
data: body.response
});
} else if (thread.awaitingHumanApproval() && body.type === 'approval' && !body.approved) {
// push feedback onto the thread
thread.events.push({
type: "tool_response",
data: `user denied the operation with feedback: "${body.comment}"`
});
} else if (thread.awaitingHumanApproval() && body.type === 'approval' && body.approved) {
// approved, run the tool, pushing results onto the thread
await handleNextStep(lastEvent.data, thread);
} else {
res.status(400).json({
error: "Invalid request: " + body.type,
awaitingHumanResponse: thread.awaitingHumanResponse(),
awaitingHumanApproval: thread.awaitingHumanApproval()
});
return;
}
// loop until stop event
const result = await agentLoop(thread);
store.update(req.params.id, result);
lastEvent = result.events[result.events.length - 1];
lastEvent.data.response_url = `/thread/${req.params.id}/response`;
console.log("returning last event from endpoint", lastEvent);
res.json(result);
});
type WebhookResponse = V1Beta2HumanContactCompleted;
app.post('/webhook/response', async (req, res) => {
console.log("webhook response", req.body);
const response = req.body as WebhookResponse;
// response is guaranteed to be set on a webhook
const humanResponse: string = response.event.status?.response as string;
const threadId = response.event.spec.state?.thread_id;
if (!threadId) {
return res.status(400).json({ error: "Thread ID not found" });
}
const thread = store.get(threadId);
if (!thread) {
return res.status(404).json({ error: "Thread not found" });
}
if (!thread.awaitingHumanResponse()) {
return res.status(400).json({ error: "Thread is not awaiting human response" });
}
thread.events.push({
type: "human_response",
data: response.event.status?.response
});
});
const port = process.env.PORT || 3000;
app.listen(port, () => {
console.log(`Server running on port ${port}`);
});
export { app };