Documentation Grafana Cloud AI and machine learning Grafana Assistant (Preview) Investigations

Assistant Investigations (Preview)

Assistant Investigations lets you swarm multiple agents over a problem in the background to investigate issues and discover root causes.

How it works

Assistant Investigations coordinates multiple specialist agents over an issue. They concurrently investigate different parts of your system, following the intent and context provided in the prompt.

Specialist agents include:

• Lead investigator - coordinates the investigation
• Loki specialist - understands LogQL and digs into logs to find clues
• Prometheus specialist - understands PromQL and digs into metrics to find clues
• Tempo specialist - understands Tempo queries and digs into traces to find clues
• Pyroscope specialist - understands Pyroscope queries and digs into profiling data to find clues
• Reporter agent - observes agent activity and generates a report

The specialist agents are deployed in parallel and can investigate different parts of your system.

View the investigation report

The agents generate a report that you can use to review the investigation findings. The report includes the following sections:

• Summary - A high-level overview of the investigation
• Report - A detailed report of the investigation
• Timeline - A timeline of the investigation
• Activity - A list of the activities performed by the agents

Start an investigation

To start an investigation, provide a prompt:

1. Select the Deep Investigation option in the Assistant chat box.
2. Provide as much context as possible about the issue in your prompt.
3. The assistant uses multiple agents to investigate the problem and provide a report.

High-level prompts work. The agents break down the problem into smaller parts and investigate each part:

The shopping cart is very slow—investigate why.

However, it’s better to provide more context using the @ mention feature:

The shopping cart is very slow. See if the @catalogservice is struggling. Pay attention to cases where the ‘app’ label is set to @shopping-cart.

Ask the assistant

You can ask the assistant in natural language to investigate an issue more deeply. It creates an investigation using the current conversation context.

You can also select Launch investigation to create one from the current conversation.

View past investigations

To see a complete list of investigations, select Open investigations from the Assistant menu.

Guide an investigation

Agents are effective at discovery, but the more context and guidance you provide, the better and faster the investigation.

You can guide investigations by:

1. Use the thumbs-up and thumbs-down icons to provide quick feedback. The agents notice this and may change direction accordingly.
2. Use the assistant in natural language to tell it what to investigate and what to ignore.

For example, you can say to the assistant:

I’m not interested in the NGINX logs. Focus on the application metrics.

You can provide additional context using this technique:

Focus on the MySQL logs and look for ‘connection refused’ errors.