This guide helps you understand how to use tool calling, sometimes known as function calling, with chat completions. Tool calling allows you to extend the capabilities of chats with LLMs by enabling the LLM to call custom functions, or tools. Your custom tools can perform a wide range of tasks, such as querying databases, fetching real-time data from APIs, processing data, or executing business logic. You can then integrate the result of these tool calls back into the model’s output. Tool calling is available for Palmyra X4 and later models.
This guide discusses calling custom functions as tools. Writer also offers prebuilt tools that models can execute remotely:
You need an API key to access the Writer API. Get an API key by following the steps in the API quickstart.We recommend setting the API key as an environment variable in a .env file with the name WRITER_API_KEY.

​
Overview

To use tool calling, follow these steps:
  1. Define your functions in code
  2. Pass the functions to the model in a chat completion request
  3. Append the assistant’s response (containing tool calls) to the message history
  4. Check to see which functions the model wants to invoke and run the corresponding functions
  5. Append the tool results to the message history
  6. Pass the updated messages back to the model to get the final response
  7. Append the final response to maintain complete conversation history

Tool calling overview

Diagram of tool calling

Example: Calculate the mean of a list of numbers

Diagram of tool calling to calculate the mean of a list of numbers
Continue reading to learn more about each step.

​
Define your custom functions

First, define the custom functions in your code. Typical use cases for tool calling include calling an API, performing mathematical calculations, or running complex business logic. You can define these functions in your code as you would any other function. Here’s an example of a function to calculate the mean of a list of numbers. 
def calculate_mean(numbers: list) -> float:
    return sum(numbers) / len(numbers)

​
Describe functions as tools

After you’ve defined your functions, create a tools array to pass to the model. The tools array describes your functions as tools available to the model. You describe tools in the form of a JSON schema. Each tool should include a type of function and a function object that includes a name, description, and a dictionary of parameters.

​
Tool structure

The tools array contains an object with the following parameters:
ParameterTypeDescription
typestringThe type of tool, which is function for a custom function
functionobjectAn object containing the tool’s description and parameter definitions
function.namestringThe name of the tool
function.descriptionstringA description of what the tool does and when the model should use it
function.parametersobjectAn object containing the tool’s input parameters
function.parameters.typestringThe type of the parameter, which is object for a JSON schema
function.parameters.propertiesobjectAn object containing the tool’s parameters in the form of a JSON schema. See below for more details.
function.parameters.requiredarrayAn array of the tool’s required parameters
See the full tools object schema for more details. The function.parameters.properties object contains the tool’s parameter definitions as a JSON schema. The object’s keys should be the names of the parameters, and the values should be objects containing the parameter’s type and description. When the model decides you should use the tool to answer the user’s question, it returns the parameters that you should use when calling the function you’ve defined.

​
Example tool array

Here’s an example of a tools array for the calculate_mean function: 
tools = [
    {
        "type": "function",
        "function": {
            "name": "calculate_mean",
            "description": "A function that calculates the mean (average) of a list of numbers. Any user request asking for the mean of a list of numbers should use this tool.",
            "parameters": {
                "type": "object",
                "properties": {
                    "numbers": {
                        "type": "array",
                        "items": {"type": "number"},
                        "description": "List of numbers"
                    }
                },
                "required": ["numbers"]
            }
        }
    }
]
To help the model understand when to use the tool, follow these best practices for the function.description parameter:
  • Indicate that the tool is a function that invokes a no-code agent
  • Specify the function’s purpose and capabilities
  • Describe when the tool should be used
An example description for a tool that invokes a function to calculate the mean of a list of numbers:
“A function that calculates the mean of a list of numbers. Any user request asking for the mean of a list of numbers should use this tool.”

​
Pass tools to the model

Once the tools array is complete, pass it to the chat completions endpoint along with the chat messages.

​
tool_choice parameter

The chat completion endpoint has a tool_choice parameter that controls how the model decides when to use the tools you’ve defined.
ValueDescription
autoThe model decides which tools to use, if any.
noneThe model does not use tools and only returns a generated response.
requiredThe model must use at least one of the tools you’ve defined.
You can also use a JSON object to force the model to use a specific tool. For example, if you want the model to use the calculate_mean tool, you can set tool_choice to {"type": "function", "function": {"name": "calculate_mean"}}. In this example, tool_choice is auto, which means the model decides which tools to use, if any, based on the message and tool descriptions. 
import json
from writerai import Writer

# Initialize the Writer client. If you don't pass the `apiKey` parameter,
# the client looks for the `WRITER_API_KEY` environment variable.
client = Writer()

messages = [{"role": "user", "content": "what is the mean of [1,3,5,7,9]?"}]

response = client.chat.chat(
    model="palmyra-x5",
    messages=messages,
    tools=tools,
    tool_choice="auto"
)

​
Process tool calls

When the model identifies a need to call a tool based on the user’s input, it indicates it in the response and includes the necessary parameters to pass when calling the tool. You then execute the tool’s function and return the result to the model.
Proper conversation history management requires appending the assistant’s response, tool results, and final response to the message history.
The method for checking for tool calls and executing the tool’s function differs depending on whether you’re streaming the response or not. Each method is described below.

​
Streaming

When using streaming, the tool calls come back in chunks inside of the delta object of the choices array. To process the tool calls:
  1. Stream and collect tool calls from the response chunks
  2. Reconstruct the assistant’s response and append it to the message history
  3. Execute functions and append tool results to the message history
  4. Get the final response from the model and append it to the message history

​
Step 1: Stream and collect tool calls from the response chunks

First, stream and collect tool calls from the response chunks.
streaming_content = ""
function_calls = []

for chunk in response:
    choice = chunk.choices[0]

    if choice.delta:
        # Collect tool calls as they stream in
        if choice.delta.tool_calls:
            for tool_call in choice.delta.tool_calls:
                if tool_call.id:
                    # Start a new function call
                    function_calls.append({
                        "name": "",
                        "arguments": "",
                        "call_id": tool_call.id
                    })
                if tool_call.function:
                    # Append to the most recent function call
                    if function_calls:
                        function_calls[-1]["name"] += tool_call.function.name or ""
                        function_calls[-1]["arguments"] += tool_call.function.arguments or ""

        # Collect regular content (for cases where no tools are called)
        elif choice.delta.content:
            streaming_content += choice.delta.content

​
Step 2: Check finish reason and reconstruct the assistant’s response

Check the finish_reason to determine if tools were called, then reconstruct and append the assistant’s response to the conversation history.
# Inside the streaming loop, check for finish_reason
if choice.finish_reason:
    if choice.finish_reason == "stop":
        # No tools were called, just regular response
        messages.append({"role": "assistant", "content": streaming_content})
        break

    elif choice.finish_reason == "tool_calls":
        # Reconstruct and append assistant message with tool calls
        tool_calls_for_message = []
        for func_call in function_calls:
            tool_calls_for_message.append({
                "id": func_call["call_id"],
                "type": "function",
                "function": {
                    "name": func_call["name"],
                    "arguments": func_call["arguments"]
                }
            })

        assistant_message = {
            "role": "assistant",
            "content": None,
            "tool_calls": tool_calls_for_message
        }
        messages.append(assistant_message)

​
Step 3: Execute functions and append tool results

Execute each function in the function_calls list and append the results to the messages array.
# Inside the tool_calls finish_reason block
for function_call in function_calls:
    function_name = function_call["name"]

    if function_name == "calculate_mean":
        try:
            arguments_dict = json.loads(function_call["arguments"])
            function_response = calculate_mean(arguments_dict["numbers"])

            # Append tool result to conversation history
            messages.append({
                "role": "tool",
                "content": str(function_response),
                "tool_call_id": function_call["call_id"],
                "name": function_name,
            })
        except Exception as e:
            # Handle errors gracefully
            messages.append({
                "role": "tool",
                "content": f"Error: {str(e)}",
                "tool_call_id": function_call["call_id"],
                "name": function_name,
            })

​
Step 4: Get and append the final response

After appending tool results, get the final response from the model and append it to maintain complete conversation history.
# Inside the tool_calls finish_reason block, after processing all functions
final_response = client.chat.chat(
    model="palmyra-x5",
    messages=messages,
    stream=True
)

final_content = ""
for chunk in final_response:
    choice = chunk.choices[0]
    if choice.delta and choice.delta.content:
        final_content += choice.delta.content

# Append final response to conversation history
messages.append({
    "role": "assistant",
    "content": final_content
})

​
Complete streaming code example

import json
from writerai import Writer

# Initialize the Writer client. If you don't pass the `apiKey` parameter,
# the client looks for the `WRITER_API_KEY` environment variable.
client = Writer()

def calculate_mean(numbers: list) -> float:
    return sum(numbers) / len(numbers)

tools = [
    {
        "type": "function",
        "function": {
            "name": "calculate_mean",
            "description": "Calculate the mean (average) of a list of numbers.",
            "parameters": {
                "type": "object",
                "properties": {
                    "numbers": {
                        "type": "array",
                        "items": {"type": "number"},
                        "description": "List of numbers"
                    }
                },
                "required": ["numbers"]
            }
        }
    }
]

messages = [{"role": "user", "content": "what is the mean of [1,3,5,7,9]?"}]

# Step 1: Initial request with tools
response = client.chat.chat(
    model="palmyra-x5",
    messages=messages,
    tools=tools,
    tool_choice="auto",
    stream=True
)

# Step 2: Process streaming response to collect tool calls
streaming_content = ""
function_calls = []

for chunk in response:
    choice = chunk.choices[0]

    if choice.delta:
        # Collect tool calls as they stream in
        if choice.delta.tool_calls:
            for tool_call in choice.delta.tool_calls:
                if tool_call.id:
                    # Start a new function call
                    function_calls.append({
                        "name": "",
                        "arguments": "",
                        "call_id": tool_call.id
                    })
                if tool_call.function:
                    # Append to the most recent function call
                    if function_calls:
                        function_calls[-1]["name"] += tool_call.function.name or ""
                        function_calls[-1]["arguments"] += tool_call.function.arguments or ""

        # Collect regular content (for cases where no tools are called)
        elif choice.delta.content:
            streaming_content += choice.delta.content

    # Check if streaming is complete
    if choice.finish_reason:
        if choice.finish_reason == "stop":
            # No tools were called, just regular response
            print(f"Response: {streaming_content}")
            messages.append({"role": "assistant", "content": streaming_content})
            break

        elif choice.finish_reason == "tool_calls":
            # Step 3: Reconstruct and append assistant message with tool calls
            tool_calls_for_message = []
            for func_call in function_calls:
                tool_calls_for_message.append({
                    "id": func_call["call_id"],
                    "type": "function",
                    "function": {
                        "name": func_call["name"],
                        "arguments": func_call["arguments"]
                    }
                })

            assistant_message = {
                "role": "assistant",
                "content": None,
                "tool_calls": tool_calls_for_message
            }
            messages.append(assistant_message)  # Append assistant response

            # Step 4: Execute each function and add results to messages
            for function_call in function_calls:
                function_name = function_call["name"]

                if function_name == "calculate_mean":
                    try:
                        arguments_dict = json.loads(function_call["arguments"])
                        function_response = calculate_mean(arguments_dict["numbers"])

                        # Add tool response to messages
                        messages.append({
                            "role": "tool",
                            "content": str(function_response),
                            "tool_call_id": function_call["call_id"],
                            "name": function_name,
                        })
                    except Exception as e:
                        # Handle errors gracefully
                        messages.append({
                            "role": "tool",
                            "content": f"Error: {str(e)}",
                            "tool_call_id": function_call["call_id"],
                            "name": function_name,
                        })

            # Step 5: Get the final response from the model
            final_response = client.chat.chat(
                model="palmyra-x5",
                messages=messages,
                stream=True
            )

            final_content = ""
            for chunk in final_response:
                choice = chunk.choices[0]
                if choice.delta and choice.delta.content:
                    final_content += choice.delta.content
                    print(choice.delta.content, end="", flush=True)

            # Step 6: Add final response to message history
            messages.append({
                "role": "assistant",
                "content": final_content
            })
            break

​
Example: External API call

The following example covers a common use case for tool calling: calling an external API. The code uses a publicly available dictionary API to return information about an English word’s phonetic pronunciation. This example is using non-streaming; for streaming, refer to the preceding section’s streaming example to adjust the code.

​
Define function calling an API

First, define the function in your code. The examples below take in a word, call the dictionary API, and return the phonetic pronunciation of the word as a JSON-formatted string. 
import requests
def get_word_pronunciation(word):
    url = f"https://api.dictionaryapi.dev/api/v2/entries/en/{word}"
    response = requests.get(url)
    if response.status_code == 200:
        return json.dumps(response.json()[0]['phonetics'])
    else:
        return f"Failed to retrieve word pronunciation. Status code: {response.status_code}"

​
Define tools array

Next, define a tools array that describes the tool with a JSON schema. 
tools = [
    {
        "type": "function",
        "function": {
            "name": "get_word_pronunciation",
            "description": "A function that will return JSON containing the phonetic pronunciation of an English word",
            "parameters": {
                "type": "object",
                "properties": {
                    "word": {
                        "type": "string",
                        "description": "The word to get the phonetic pronunciation for",
                    }
                },
                "required": ["word"],
            },
        },
    }
]

​
Pass the tools to the model

Call the chat.chat method with the tools parameter set to the tools array and tool_choice set to auto. 
from writerai import Writer

# Initialize the Writer client. If you don't pass the `apiKey` parameter,
# the client looks for the `WRITER_API_KEY` environment variable.
client = Writer()

messages = [{"role": "user", "content": "what is the phonetic pronunciation of the word 'epitome' in English?"}]

response = client.chat.chat(
    model="palmyra-x5",
    messages=messages,
    tools=tools,
    tool_choice="auto",
    stream=False
)

​
Check response for tool calling

Loop through the tool_calls array to check for the invocation of the tool. Then, call the tool’s function with the arguments the model provided. 
response_message = response.choices[0].message
messages.append(response_message)
tool_calls = response_message.tool_calls
if tool_calls:
    tool_call = tool_calls[0]
    tool_call_id = tool_call.id
    function_name = tool_call.function.name
    function_args = json.loads(tool_call.function.arguments)

    if function_name == "get_word_pronunciation":
        function_response = get_word_pronunciation(function_args["word"])

​
Append the tool result to the conversation history

Finally, pass the result back to the model by appending it to the messages array, and get the final response. 
messages.append({
    "role": "tool",
    "tool_call_id": tool_call_id,
    "name": function_name,
    "content": function_response,
})

final_response = client.chat.chat(
    model="palmyra-x5",
    messages=messages,
    stream=False
)

final_content = final_response.choices[0].message.content

# Append final response to conversation history
messages.append({
    "role": "assistant",
    "content": final_content
})

print(f"Final response: {final_content}")
# Final response: The phonetic pronunciation of the word "epitome" in English is /əˈpɪt.ə.mi/...

​
Complete external API call example

import requests
import json
from writerai import Writer

# Initialize the Writer client. If you don't pass the `apiKey` parameter,
# the client looks for the `WRITER_API_KEY` environment variable.
client = Writer()

def get_word_pronunciation(word):
    url = f"https://api.dictionaryapi.dev/api/v2/entries/en/{word}"
    try:
        response = requests.get(url)
        if response.status_code == 200:
            return json.dumps(response.json()[0]['phonetics'])
        else:
            return f"Failed to retrieve word pronunciation. Status code: {response.status_code}"
    except Exception as e:
        return f"Error fetching word pronunciation: {str(e)}"

tools = [
    {
        "type": "function",
        "function": {
            "name": "get_word_pronunciation",
            "description": "A function that will return JSON containing the phonetic pronunciation of an English word",
            "parameters": {
                "type": "object",
                "properties": {
                    "word": {
                        "type": "string",
                        "description": "The word to get the phonetic pronunciation for",
                    }
                },
                "required": ["word"],
            },
        },
    }
]

messages = [{"role": "user", "content": "what is the phonetic pronunciation of the word 'epitome' in English?"}]

# Step 1: Initial request
response = client.chat.chat(
    model="palmyra-x5",
    messages=messages,
    tools=tools,
    tool_choice="auto",
    stream=False
)

# Step 2: Append assistant response
response_message = response.choices[0].message
messages.append(response_message)
tool_calls = response_message.tool_calls

# Step 3: Process tool calls
if tool_calls:
    for tool_call in tool_calls:
        tool_call_id = tool_call.id
        function_name = tool_call.function.name
        function_args = json.loads(tool_call.function.arguments)

        if function_name == "get_word_pronunciation":
            function_response = get_word_pronunciation(function_args["word"])

            # Append tool result
            messages.append({
                "role": "tool",
                "tool_call_id": tool_call_id,
                "name": function_name,
                "content": function_response,
            })

    # Step 4: Get final response
    final_response = client.chat.chat(
        model="palmyra-x5",
        messages=messages,
        stream=False
    )

    final_content = final_response.choices[0].message.content

    # Step 5: Append final response
    messages.append({
        "role": "assistant",
        "content": final_content
    })

    print(f"Final response: {final_content}")

# The conversation history is now complete and ready for additional turns

​
Next steps

By following this guide, you can incorporate tool calling into your application and augment the capabilities of a model with real-time data, math operations, business logic, and much more. For more examples, check out the tool calling cookbooks available on GitHub. Next, learn how to invoke no-code agents with tool calling. Or, explore prebuilt tools that Writer models can execute remotely: