%3Aquality(100)%2F&w=3840&q=75)
The Grafana AI SDK for Go: a shared foundation for building AI applications
Starting an experiment with an LLM has never been easier. Keeping a growing collection of those experiments consistent is another matter.
Earlier this year, as more teams began exploring AI features here at Grafana Labs, we repeatedly encountered the same pattern: a new experiment would start, move quickly, and build its own client for whichever model provider it needed. The next experiment would do the same, with a slightly different abstraction for streaming, tools, errors, or provider configuration.
This was understandable, given the circumstances. Model providers were changing quickly, our teams were learning quickly, and coding agents made it possible to turn an idea into a working integration faster than ever. But that speed also made it easier for every integration to develop its own architecture.
Eventually, we were maintaining a collection of solutions to what was essentially the same problem.
And since most of our backend is written in Go, we built the Grafana AI SDK for Go to give our teams a shared foundation to work from. It provides common interfaces for calling models, streaming responses, executing tools, producing structured output, and running multi-step agents. It also speaks the protocol used by Vercel AI SDK frontend hooks, so a Go backend can stream directly to useChat, useCompletion, and useObject.
We built it because we needed it inside Grafana Labs, but we open sourced it last month (alongside a broader collection of tools we released for building, operating, and understanding AI systems during our first Grafana Labs AI Week) because we think other teams building AI applications in Go are likely to encounter many of the same problems.
We would like to build the next part together, so in this blog I'll tell you a bit more about the project, including how you can put it to use today, as well as how you can help us improve it.
What teams can build with it today
The SDK supports both simple model calls and larger application workflows:
- Generate a complete response or stream output as it arrives.
- Expose typed Go functions as tools and continue through multiple model and tool steps.
- Pause consequential tools for human or application approval.
- Generate schema-validated Go objects, arrays, and choices.
- Add timeouts, retries, model fallback, logging, metrics, and Agent Observability.
- Serve streaming Go endpoints to AI SDK React hooks.
The repository includes a full-stack agent chat example and a structured extraction example, along with guides for testing, security, error handling, and production operation.
This is the common foundation many Grafana Labs teams now rely on rather than reimplementing the same patterns for each new integration. It is also still a young project operating in an ecosystem that changes unusually quickly. We expect the interfaces, providers, and conformance baseline to keep evolving as we learn from more applications.
From fast experiments to common patterns
Next, I want to tell you a bit more about how we got here, because I suspect many of you have been living through similar experiences.
Our early LLM integrations were intentionally experimental. At that stage, local decisions and duplicated code were often the fastest way to learn. The problem appeared as those experiments multiplied.
Each provider has its own API, authentication model, streaming events, error behavior, and model-specific features. And once an application moves beyond a single prompt and response, it also needs to make decisions about tools, structured output, retries, timeouts, conversation history, and what happens when a stream fails halfway through.
When every team owns that entire stack, small differences accumulate:
- One integration retries at the provider layer, while another retries the whole operation.
- One streams custom events to the frontend, while another waits for a complete response.
- Each application has to learn a different set of conventions before it can add or change an AI-powered workflow.
None of those implementations were necessarily wrong. Collectively, however, they made it harder to share improvements and harder to establish consistent operational and safety practices.
The goal of the SDK is to move that repeated infrastructure into one place. Product teams still own their prompts, tools, authorization, workflows, and user experience, but they shouldn't have to invent a new provider abstraction and streaming protocol before they can begin.
Inspired by Vercel AI SDK, implemented for Go
We didn't want to invent a new mental model simply for the sake of having our own.
We've been fans of what Vercel built with its AI SDK: a common model interface, a clear orchestration layer, composable tools, and frontend primitives that make streaming AI experiences easier to build. Those abstractions were already familiar to developers and captured many of the patterns we wanted.
The Grafana AI SDK follows that model while expressing it through Go conventions: interfaces, channels, explicit errors, contexts, and functional options. The Vercel AI SDK is also our reference for behavior, naming, and the frontend wire protocol. We run conformance tests against pinned upstream versions and test Go endpoints with the real TypeScript frontend packages.
That doesn't mean this is a line-by-line port or a claim of complete feature-for-feature parity. Some upstream features do not yet exist in the Go SDK, and some concepts need a different shape in Go. We maintain a public compatibility baseline and a coverage map of known gaps and deviations so those differences are visible rather than hidden behind a broad compatibility claim.
One interface across providers
The Grafana AI SDK currently supports Anthropic, Amazon Bedrock, OpenAI's Responses API, and OpenAI-compatible APIs. Provider integrations live in separate Go modules, so an application installs only the dependencies it uses. Once a model is constructed, the surrounding generation and orchestration APIs remain the same.
A basic call looks like this:
model := anthropic.New(apiKey, modelID)
result, err := aisdk.GenerateText(ctx, model,
aisdk.WithModelMessages(
provider.UserText("Summarize this incident."),
),
)
The same model interface works with streaming, typed tools, structured output, agent loops, fallback, and middleware. Switching providers can still expose real differences—models don't universally support the same inputs, reasoning options, or tools—but applications don't need a completely different architecture for each one.
This shared provider boundary is what makes the rest of the SDK composable. It lets us add behavior around model calls once and apply it consistently across applications.
Adding the Grafana Labs flavor through middleware
For us, the most important example of that shared behavior is observability.
An AI application can turn one user request into several model calls. An agent may call tools, continue through multiple steps, retry a failed request, or fall back to another provider. If we observe only the outer HTTP request, much of the work—and many of the failure modes—remain invisible.
The SDK's Agent Observability middleware records model and agent activity in Grafana, including usage, errors, multi-step relationships, and which provider served a routed call. The SDK also includes optional middleware for structured logging and Prometheus metrics.
Instrumentation is the starting point, not the end goal. The larger challenge is using production signals to understand agent behavior, find failures, build better test cases, and evaluate changes before they reach users. For a deeper look at that progression—from monitoring live traffic to evaluations, test suites, experiments, and CI/CD—read our blog post on how to build a trust platform for your agent with Grafana Agent Observability.
Because middleware wraps the common model interface, application teams don't have to instrument every provider integration independently. They can start with shared conventions and spend their time on the behavior that makes their product useful.
Note: Observability is only one part of operating AI systems responsibly. We are also working on LLM security middleware, which we will share more about in the future. More broadly, applications still need to authenticate users, authorize tools, validate model-generated input, constrain resource use, and decide carefully which content may be stored or recorded. The SDK provides building blocks for those decisions; it does not make the decisions on an application's behalf.
A simpler path from Go to the frontend
The backend was not the only place where our implementations were diverging. Streaming a model response to a browser can involve its own event format, message representation, tool state, error handling, and conversation persistence.
The Grafana AI SDK implements the UI message stream protocol used by @ai-sdk/react. A Go handler can write server-sent events directly to useChat, while tools, reasoning, sources, files, and application data move through typed message parts.
Go backend React frontend
---------- --------------
StreamText(...) -- SSE --> useChat(...)
WriteUIMessageStream @ai-sdk/react
This lets backend teams stay in Go while frontend teams use an ecosystem and set of primitives they already know. It also means teams can reuse an existing AI SDK frontend when moving backend logic into a Go service, without creating a protocol adapter between them.
Let's build it together
Grafana AI SDK started with a practical internal need: keep our Go-based AI work from becoming a collection of disconnected provider clients and custom protocols. It now gives our teams a common way to build and observe those applications.
If your backend is written in Go, we hope it can give you the same kind of head start.
You can find the code, documentation, and runnable examples in the grafana/ai-sdk repository. Try it with an existing AI SDK frontend, build a provider or middleware, tell us where the abstractions do not fit, or open an issue for a capability you need.
Grafana AI SDK was one of several projects and features we shared during AI Week. Read the AI Week recap to explore the other announcements and the broader thinking behind our work with AI and observability.
We know this foundation is useful to us. Now we would like to find out what it can become with the Go and open source communities involved.
Grafana Assistant is the easiest way to get started with metrics, logs, traces, dashboards, and more in Grafana Cloud. We have a generous forever-free tier and plans for every use case. Sign up for free now!
