II · THE IDEA · ARTIFICIAL INTELLIGENCE
Tool Use and Function Calling
▶ Listen · narrated
A language model cannot multiply large numbers, fetch live data or send an email. But it can describe what it needs done, in a format your program can execute.
At a glance
- What it is
- Model emits a structured call; external code executes it and returns the result
- Model's role
- Decides which tool to invoke and what arguments to supply
- Code's role
- Validates, executes, handles errors, returns formatted output
- Security boundary
- Model never executes anything; all actions gated by your code
Imagine you are working with an assistant who is excellent at conversation but cannot leave the room, use a phone, or operate a computer. You can. So when the assistant needs a piece of information from outside, they write you a note describing exactly what they need: call this number, search for this term, calculate this sum. You read the note, do the task, write down the result, and hand it back. They read your answer and continue the conversation, now informed by what you found. Tool use works the same way. The model is the assistant, fluent and thoughtful but confined to generating text. Your code is you, able to execute operations in the real world. The model writes a structured request, your code performs it, and the result comes back as text the model can read and reason about.
The model emits a structured representation of a function call, typically JSON, containing a function identifier and a dictionary of named arguments. Your application parses this, validates the structure, checks that the function exists in a predefined registry and that the arguments match the expected schema, then invokes the corresponding implementation in a controlled execution environment. The function runs, returns a value or raises an exception, and your code serialises the result back into text, appends it to the conversation history, and prompts the model to continue generation. The model has no access to the execution environment, no ability to evaluate code, and no direct interaction with system resources. All it can do is emit text that your parser recognises as a tool call. Security depends entirely on your validation and sandboxing. The model learns which tool to invoke and when by training on datasets that include function signatures, docstrings, and example calling patterns, or by few-shot prompting with examples of correct tool use. It learns the syntax of tool calls the same way it learns any linguistic structure: by predicting text that matches the pattern. Errors are common. The model may hallucinate function names, supply arguments of the wrong type, call a tool when direct generation would suffice, or ignore a tool result and generate confidently incorrect output. Robust systems validate every call, return actionable error messages, and log all tool invocations for auditing.
Look closer
The model emits a structured call, not code that runs
When a model uses a tool, it produces a specially formatted string, often JSON, containing a function name and arguments. Your application parses that string, validates it, decides whether to permit the call, executes the actual operation in a controlled environment, and formats the result back into text for the next generation step. The model never touches the file system, never opens a socket, never evaluates its own output. This separation is the entire security model.
The model learned to request tools by seeing examples
Toolformer, an early demonstration, showed that models could learn tool use from a corpus augmented with synthetic examples of API calls and their results. The training data included passages where a calculator call appeared mid-sentence, followed by the numeric result, teaching the model both when to invoke a tool and how to continue reasoning with the answer. More recent approaches fine-tune on datasets of function signatures and calling patterns, or rely on in-context examples in the system prompt. Either way, the model is learning a linguistic pattern, not acquiring the ability to execute operations.
The model can call the wrong tool, or call the right one incorrectly
A model may request a database query when it should calculate locally, or format arguments in a way your parser rejects. It may hallucinate a function name that does not exist, or pass a string where an integer is required. Your code must handle these gracefully: validate the structure, check that the requested function is permitted, coerce or reject arguments as needed, catch execution errors, and return intelligible feedback. The model will often recover if you tell it what went wrong, but only if your error messages are clear enough to guide the next attempt.
The story
A language model is a text predictor. It cannot fetch a stock price, query a database, or multiply two thirty-digit numbers with reliable accuracy. But it can write a sentence that describes such an operation in a structured format, and if your code is listening for that format, you can execute the operation on the model's behalf and feed the result back as part of the conversation.
This is tool use, sometimes called function calling. The model generates a specially formatted request, your application parses it, your code runs the actual operation in a controlled environment, and the result is returned to the model as text so it can continue reasoning. The model never executes anything itself. It proposes; your code disposes.
The structured format is usually JSON, though some systems use XML or custom delimiters. A calculator call might look like this: {"function": "multiply", "arguments": {"a": 123456789, "b": 987654321}}. Your code recognises that structure, extracts the function name and arguments, performs the multiplication, and returns the result formatted as text. The model receives that text in the conversation history and continues generating, now able to refer to the product as if it had calculated it itself.
The model decides which tool to call and what arguments to supply based on the conversation so far and the tool descriptions you provided, usually in the system prompt. Those descriptions are just text: the function name, a summary of what it does, the parameter names and types it expects. The model has learned from training data that when it needs information or capability it does not possess, it should emit a structured call matching that format. It has also learned, if the training was effective, roughly when each type of tool is appropriate.
Your code sits outside the model, watching for these structured requests. When one appears, you parse it, validate that the requested function exists and is permitted, check that the arguments are well-formed, and execute the operation in whatever environment you control. If the call succeeds, you format the result and append it to the conversation. If it fails, you return an error message clear enough that the model can adjust and try again. Then you prompt the model to continue, and it generates the next portion of its response, informed by the tool's output.
This loop can repeat. The model might call a search tool to retrieve documents, read the results, realise it needs a calculation, call a calculator, then synthesise an answer from both. Each tool call is a round trip: model generates, your code executes, result comes back, model continues. The model is orchestrating a sequence of operations it cannot perform, using only its ability to predict what structured request would be useful next.
Why it mattered then
Toolformer, published in early 2023, demonstrated that language models could learn to invoke external APIs without task-specific fine-tuning for each tool, simply by training on a corpus where API calls and results had been inserted at useful points. The paper showed a model learning when to call a calculator for arithmetic, a question-answering system for factual retrieval, a search engine for recent information, a translation API for non-English text, and a calendar for date arithmetic. The model was not given explicit supervision about which tool to use in which context; it inferred the pattern from examples. This mattered because it suggested that tool use was a capability that scaled with general language understanding, not something that required separate engineering for every new function.
Why it matters now
Tool use has become the standard method for connecting language models to live data, internal systems, and operations they cannot perform through prediction alone. A model with access to a search tool can answer questions about events after its training cutoff. A model with database access can query your inventory, your customer records, your logs. A model with a code interpreter can perform reliable arithmetic, generate plots, process files. The boundary between what a model can do through prediction and what it can do through tool calls defines the practical scope of the system you build around it. The security model matters because a model with unrestricted tool access can do anything your code can do, and a model is not a trusted process. Every production system that uses tool calling must validate requests, enforce permissions, handle errors, and log actions. The model's role is to propose useful operations; your code's role is to decide whether to permit them.
The surprising detail
Gorilla, a model fine-tuned specifically for API calling, was trained on a synthetic dataset of function signatures and example invocations generated by scraping API documentation and using another language model to produce realistic call patterns. The training data included not just correct calls but also common errors and their corrections, teaching the model to recover from mistakes. The approach worked well enough that the model could invoke APIs it had never seen during training, as long as their documentation was provided in the prompt. This suggests that tool use is partly a matter of learning the grammar of function calls and partly a matter of reading and following instructions, both of which are tasks language models were already performing in other contexts.
Remember this
The model writes a structured request. Your code decides whether to execute it. The model never runs anything.
Test yourself
A model with calculator access still sometimes returns wrong arithmetic. Why might you see this even when the calculator itself is correct?
The model may attempt the calculation inline, generating the digits directly instead of emitting a tool call, especially for arithmetic that looks simple. It may also call the calculator but then ignore or misinterpret the result, continuing to generate as if it had worked the problem out itself. A third possibility: it may call the calculator with malformed arguments, receive an error, and proceed anyway rather than correcting the call. The model learned tool use as a pattern, not as a hard rule, so it will sometimes skip the tool when it predicts that the next token is a number. Reliable arithmetic requires not just providing a calculator but training or prompting the model to use it consistently, and validating that it incorporates the result rather than generating over it.
Go deeper
- Toolformer: Language Models Can Teach Themselves to Use Tools · arXiv · Timo Schick et al. · 2023-02-09
- Gorilla: Large Language Model Connected with Massive APIs · arXiv · Shishir G. Patil et al. · 2023-05-24
Image: Original diagram, The Daily Triptych. Licence: Original work. Source.