Function Calling with Spring AI
A simple Spring application using Open AI's function calling feature
08/08/2024
Introduction
According to OpenAI, function calling is the ability to connect large language models (LLMs) to external functionalities such as real-time services, databases, etc, so the AI assistant can "call" functions or tools in order to provide answers based on external data. Since the LLM is not actually capable of calling functions, the idea is to get output from the LLM in JSON format, so your application can properly select and invoke a function. This function will either provide data to be fed back to the LLM or just perform an external action.
In this short tutorial, we will create a Spring application that answers the user's question about the cryptocurrency market using a chat model from OpenAI and an external API for retrieving quotation data. The LLM model will decide whether or not to call the external function based on the user's prompt and provide basic assessments about the market. The diagram below illustrates how this process work:
The project
Let's get started by creating a new Spring project with spring initializr. For this demo application I will be using Spring Web, OpenAI, and Lombok.
Then, let's set up a few properties in our application.properties file:
The OpenAI API key will be provided via the environment variable OPEN_API_KEY
. Let's create a configuration properties file for the chat parameters:
Don't forget to enable the configuration properties in the main application file:
A simple overview endpoint
It's always good to start simple. Let's create an endpoint that returns a basic overview of the specified cryptocurrency based on a fake quotation:
The path variable is of the CryptoSymbol type, which is an enum listing the cryptocurrencies supported by our application:
The core logic of our service will be implemented in AdviceService
, where we are going to inject an instance of the OpenAIChatModel
and perform a prompt call to the GPT model. There we create a simple user message asking for a short overview of the specified crypto based on its current quotation. When building the OpenAIChatOptions parameter that is passed to the prompt constructor, we specify the model, the temperature, and the quotation function. By doing so, we inform the model that the "CurrentQuotation" function is available for it to call if necessary (in this case, it will always be).
Now we need to implement the CurrentQuotation function and make it available in the application container. Let's create a class QuotationFunctionMock that implements the Function interface providing a Quotation object with mock data:
The Quotation class is a simple record with price and volume information:
Finally, we create a configuration class that provides a bean of the type FunctionCallback. This object will wrap an instance of our QuotationFunctionMock class and hold information about the function (name and description). The name attribute should match the one we used in the AdviceService ("CurrentQuotation") and the description should instruct the chat model about what the function does and how to call it.
Now we have everything set up to try our overview endpoint out. Remember to set the environment variable OPENAI_API_KEY to your OpenAI key. By calling the endpoint http://localhost/advice/overview/BTC
using curl, some HTTP client, or even your browser, you should get an (outdated) overview about Bitcoin based on the mock information our function provided:
Answering a question using real-time data
Now it's time to make our application a bit more interesting. Let's create a new service allowing the user to ask a question about the cryptomarket to the chat model. First, create a record called Question containing the single string field text. Then, create the following method in the AdviceController class:
Then we implement a method called answer in the AdviceService that works similarly to the overview method, but passing two messages to the model instead:
(i) A system message with instructions of how to properly answer the user about the cryptomarket using the available tools (our quotation function, in this case). The prompt is designed so that the model will (hopefully) only answer questions related to cryptocurrencies (ii) A user message containing the question from the client input
Connecting to an external API
At this point, everything should work fine using our fake data. In order to start working with real-time information, we need to wire our application up with some external API. For the sake of simplicity, we will use the RestTemplate class for that purpose, which is already included in the Spring Web Framework. You should opt for WebClient in your applications instead, since it's a more modern, non-blocking, and reactive solution.
For this demo application, I chose to use the DIA Data API, which provides simple crypto information for free and does not require the use of an authorization key (up to this date). Let's create a new service that will be responsible for retrieving the latest quotation info for a given cryptocurrency.
The QuotationProperties configuration class and the associated properties can be found on the project repository. Now we must create a new quotation function class that uses this service instead of just returning an object with mock data:
Then we must modify the BeanProvider configuration class to inject the QuotationService when creating the QuotationFunction object and also to build and provide the RestTemplate object:
Running and testing the new service
Let's run the application and perform a post request to our new endpoint:
By sending the question above, you should probably get a response based on two external API calls (one for each mentioned cryptocurrency). This is what I got in 29/07/2024:
Notice that the answer is based both on the data retrieved by the external API and its own knowledge about Ethereum and Solana.
Conclusion
In this short tutorial we created a Spring API for getting advice about cryptocurrencies using a chat model and up-to-date quotation data. Despite its simplicity, the application demonstrates an interesting interplay between the foundational knowledge of a large language model and external information. By integrating the model with more complex data, like price history, moving averages, etc, it should be possible to design powerful applications for providing insight into the market in real-time.
Resources
- Project source code on GitHub