When AI Doesn't Need to Generate Anything
Feat. Gemini Decisions API
When building applications with AI, we often ask models to generate text and then try to make sense of that text in our code.
For instance, imagine you’re building a feedback management application. You want to categorise incoming feedback as a bug report, a feature request, or general feedback. You might ask an LLM to classify the message and return the result as JSON.
Your application then parses that JSON, checks whether the expected fields exist, and uses the result to decide what to do next.
This works, but there’s an interesting question here: if all we need is a decision, why ask the model to generate a whole lot of text in the first place? That’s simply the waste of precious tokens and time. Instead, we could ask the model to evaluate a set of predefined questions and return the answers directly.
Google seems to have been thinking along similar lines. It recently introduced the Gemini Decisions API, a new API powered by the gemini-decision-1 model that’s designed specifically for making decisions rather than generating responses.
What is the Gemini Decisions API?
The Gemini Decisions API is designed for situations where you already know the possible outcomes and just need a model to evaluate which one applies.
Instead of generating a response token by token, the model evaluates predefined questions and returns probabilities for the possible answers.
It currently supports three types of questions:
- Yes/No (
noul): Determines whether a particular condition is true. - Choice (
choice): Picks one option from a predefined list. - Score (
score): Evaluates something against an ordered set of criteria.
For instance, you could ask whether a customer message describes a bug, which department should handle it, and how severe the issue is.
The interesting part is that you can ask all these questions about the same input in a single request. The API supports up to 1,000 independent questions in one call.
It also supports multimodal input, including text, images, audio, video, and PDFs.
A simple feedback classifier using the Decisions API
Let’s put this into practice with a small JavaScript example.
We’ll use the Gemini API’s official JavaScript SDK, @google/genai.
First, install the package:
npm install @google/genai
We’ll also need an API key from Google AI Studio.
Now, let’s initialise the client:
import { GoogleGenAI } from "@google/genai";
const ai = new GoogleGenAI({
apiKey: process.env.GEMINI_API_KEY,
apiVersion: "v1beta"
});
For this example, let’s say we receive the following feedback:
const feedback = `
The editor crashes whenever I try to export my notes to PDF.
`;
We want to determine three things:
- Is this a bug report?
- Which category does it belong to?
- How severe is the issue?
Here’s how we can ask all three questions in one request:
const result = await ai.decisions.create({
model: "gemini-decision-1",
input: feedback,
questions: {
is_bug: {
type: "noul",
instructions: "Is this message reporting a software bug?"
},
category: {
type: "choice",
instructions: "Which category best describes this feedback?",
criteria: {
bug: "A feature is broken or produces an unexpected result.",
feature_request: "The user is requesting new functionality.",
general_feedback: "General feedback or suggestions.",
other: "The message does not fit any of these categories."
}
},
severity: {
type: "score",
instructions: "How severe is the reported issue?",
criteria: [
"Minor inconvenience.",
"A feature is partially broken.",
"A core feature is completely unusable."
]
}
}
});
console.log(result.answers);
Notice that we’re not asking the model to explain its reasoning or generate a JSON object. We’re giving it a set of clearly defined questions and possible outcomes.
The response contains an answer for each question, along with its confidence value.
For instance, the response might look something like this:
{
"answers": {
"is_bug": {
"type": "noul",
"noul": 0.99,
"confidence": 0.99
},
"category": {
"type": "choice",
"choice": "bug",
"probabilities": {
"bug": 0.99,
"feature_request": 0.002,
"general_feedback": 0.005,
"other": 0.003
},
"confidence": 0.99
},
"severity": {
"type": "score",
"score": 1.82,
"confidence": 0.91
}
}
}
The values above are illustrative. Actual results will depend on the input and the model’s evaluation.
Use the result in your application
This is where things get interesting. Since the API returns structured answers, we can use them directly in our application logic.
For example, we could automatically route feedback to the appropriate team when the model is sufficiently confident about its classification.
const { is_bug, category, severity } = result.answers;
if (
is_bug?.type === "noul" &&
category?.type === "choice" &&
is_bug.noul >= 0.85 &&
category.confidence >= 0.85
) {
console.log("Route to:", category.choice);
} else {
console.log("Send for human review");
}
The confidence thresholds here are illustrative. In a real application, you’d want to determine them using labelled examples from your own data.
There’s also an important distinction here. The questions are evaluated independently. The severity question doesn’t know what the category question answered. If one decision depends on another, your application needs to combine the answers or make a separate request.
Also, a choice question always selects from the options you provide, even when none of them is a particularly good fit. That’s why including an other option and checking the confidence is useful.
The cost effectiveness is the icing on the cake
Another interesting aspect of the Gemini Decisions API is its pricing.
Google charges $0.10 per million input tokens, with no output-token charges. There’s also a free tier available for prototyping.
But the bigger advantage isn’t just the low input-token price. It’s how the API handles multiple questions in a single request.
Remember how we asked three different questions about the same feedback message earlier? The API processes the shared input only once, regardless of how many questions we ask. Each additional question adds only its own instructions and criteria tokens.
This makes it particularly cost-effective for applications that need to evaluate several conditions against the same input.
For example, imagine processing 100,000 customer feedback messages, with each message containing around 500 tokens. At the published rate, the shared input alone would cost approximately $5.
That’s before accounting for the additional tokens used by question instructions and criteria, but it gives you an idea of how inexpensive this can be.
Compare this with a conventional generative approach, where you’d potentially need to generate a response for every message and then parse it in your application.
There’s no output-token billing here, no generated paragraphs to process, and no need to ask separate questions in separate requests when they can be evaluated together.
For applications involving classification, routing, scoring, or content evaluation at scale, this pricing model is certainly worth considering.
When should you use it?
The Decisions API isn’t meant to replace generative models. It serves a different purpose.
For instance, consider these scenarios:
| Task | Suitable approach |
|---|---|
| Categorising customer feedback | Decisions API |
| Routing a support ticket | Decisions API |
| Checking whether content violates a rule | Decisions API |
| Scoring an application against a rubric | Decisions API |
| Summarising customer feedback | Generative model |
| Explaining why something was classified as a bug | Generative model |
| Generating a response to a customer | Generative model |
The distinction is fairly simple. If you already know the possible outcomes and need the model to choose between them, the Decisions API is worth considering.
If you need an explanation, a summary, or an open-ended response, a generative model is still the better fit.
There are a few other things worth knowing. The Decisions API currently uses the gemini-decision-1 model, and its questions are limited to the predefined types. It also doesn’t support the kind of multi-step reasoning or tool calling you’d expect from a general-purpose model.
Wrapping up
In my opinion, the Gemini Decisions API is an interesting addition to the tools available for building AI-powered applications.
We’ve become accustomed to using generative models for almost everything, including tasks where the application only needs a simple decision. That often means generating text, parsing it, validating its structure, and finally using it in an ordinary conditional statement.
The Decisions API offers a different approach. Instead of generating something that your code has to interpret, it gives your code a decision it can work with directly.
I think this is a useful way to look at AI APIs. We don’t always need a model to generate a response. Sometimes, we just need it to make a decision and let our application take it from there.
👋 Hi there! This is Amit, again. I write articles about all things web development. If you enjoy my work (the articles, the open-source projects, my general demeanour... anything really), consider leaving a tip & supporting the site. Your support is incredibly appreciated!