504ad6f544
- Update all factor file names to use two-digit numbering (e.g., factor-01, factor-02) - Update all internal links to reflect new file naming convention - Update image references to match new naming pattern - Maintain consistent navigation links across all factor documents This change improves file organization and makes the documentation structure more maintainable.
3.0 KiB
4. Tools are just structured outputs
Tools don't need to be complex. At their core, they're just structured output from your LLM that triggers deterministic code.
For example, lets say you have two tools CreateIssue and SearchIssues. To ask an LLM to "use one of several tools" is just to ask it to output JSON we can parse into an object representing those tools.
class Issue:
title: str
description: str
team_id: str
assignee_id: str
class CreateIssue:
intent: "create_issue"
issue: Issue
class SearchIssues:
intent: "search_issues"
query: str
what_youre_looking_for: str
The pattern is simple:
- LLM outputs structured JSON
- Deterministic code executes the appropriate action (like calling an external API)
- Results are captured and fed back into the context
This creates a clean separation between the LLM's decision-making and your application's actions. The LLM decides what to do, but your code controls how it's done. Just because an LLM "called a tool" doesn't mean you have to go execute a specific corresponding function in the same way every time.
If you recall our switch statement from above
if nextStep.intent == 'create_payment_link':
stripe.paymentlinks.create(nextStep.parameters)
return # or whatever you want, see below
elif nextStep.intent == 'wait_for_a_while':
# do something monadic idk
else: #... the model didn't call a tool we know about
# do something else
Note: there has been a lot said about the benefits of "plain prompting" vs. "tool calling" vs. "JSON mode" and the performance tradeoffs of each. We'll link some resources to that stuff soon, but not gonna get into it here. See Prompting vs JSON Mode vs Function Calling vs Constrained Generation vs SAP, When should I use function calling, structured outputs, or JSON mode? and OpenAI JSON vs Function Calling.
The "next step" might not be as atomic as just "run a pure function and return the result". You unlock a lot of flexibility when you think of "tool calls" as just a model outputting JSON describing what deterministic code should do. Put this together with factor 8 own your control flow.