Johannes Knoll

FunctionGemma

A tiny LLM for function calling.

FunctionGemma is a very special LLM. It only has 300MB for 270M parameters, which is very small by modern standards and compared to the frontier models. Unlike typicial chatbots it doesn’t give text answers, but only turns text inputs into structured function calls.

https://www.youtube.com/watch?v=UldqWmyUap4

What is “Function Calling”?

It is indeed not much rocket science. A function consists of a name, input parameters (the arguments), and an output. For example, a function that fetches the current temperature might look like this (in Python):

def get_weather(city: str) -> float: ...

Then calling this function, looks like this:

>>> get_weather("Augsburg")
4.0

To call this function I as a developer had to think about, well what was its name? I had to think about the right function name to call and know the parameters it takes and give them to the function. This needs a very precise understanding of the function and the syntax. With FunctionGemma this changes. It serves as a natural language interface for code. Instead of manually writing the function call, it is able to “translate” a sentence describing the action to perform into a function call. For example my natural language question:

“What’s the temperature in Augsburg?”

Outputs a structured object, that looks like this:

ToolCall(function=Function(name='get_weather', arguments={'city': 'Augsburg'}))

And just like that, it is possible to call a function using plain English!

Wiring it Up

To test FunctionGemma, I wanted to implement this weather calling in real code.

First of all I had to get the model running locally. Thanks to its small size, my laptop was capable of running this model without problems.

I used ollama to get the model easily running on my pc. With ollama pull functiongemma, I had the model ready to go. Then I setup a simple Python project with uv init, added the ollama Python Client with uv add ollama, and a http client with uv add httpx to call an external weather API.

The basic get_weather function looks like this:

def get_weather(city: str) -> float:
    """
    Get the current weather for a city.

    Args:
        city: The name of the city

    Returns:
        A float representing the current temperature in Celsius.
    """
    r = httpx.get(f"https://wttr.in/{city}?format=j1")
    data = r.json()
    return data["current_condition"][0]["FeelsLikeC"]

The model then needs to be able to inspect the function, which is easy in Python, because functions are first-class objects. Functions are passed as `tools` to the ollama client, where it can see its name and typed parameters and output. But also it sees the docstring, which provides a human-readable description of what the function does. This is not only helpful to humans, but also gives LLMs more context.

tools = [get_weather]

response = chat(
    "functiongemma",
    messages,
    tools=tools,
)

To make the function callable by the string of its name, I used a simple registry:

tool_registry = {f.__name__: f for f in tools}

# Retrieve and execute the function
func_name = tool_call.function.name
func = tool_registry[func_name]
result = func(**tool_call.function.arguments)
print(f"Result: {result} C")

After this I was able to run the script:

uv run examples/weather.py
Prompt: What's the weather in Augsburg?
Calling: get_weather({'city': 'Augsburg'})
Result: 6 C

Pretty neat! :)

The repo with the full code is available on Codeberg.

There are two other examples in the repo that show how to use FunctionGemma with different types of tools. One for controlling my linux desktop environment using ydotool, and another for automating git commands.

Conclusino

I was amazed that I could run this model locally on my laptop! No API fees, no cloud dependency or privacy issues, just plain old Python and ollama.

I also think this is where LLMs really shine. LLMs are really good at understanding natural language and also flexible enough to be trained on custom data to understand specific domain language, just like the weather.py script. But LLMs really suck (at least at the scale of functiongemma) in logic. Functions on the other hand are great at logic. Combining both allows us to have the best of both worlds, mapping human language to machine actions.

Really interesting to see the world of AI move in all directions. Maybe more parameters isn’t always necessary, as seen with frontier models. Maybe models could be specialized for certain tasks and one model orchestrates multiple specialized, smaller models, just like our brain does :)

References