--- title: "Experiments API | Grafana Cloud documentation" description: "Manage saved conversations, collections, and test suites, submit experiment runs, and read results." --- > For a curated documentation index, see [llms.txt](/llms.txt). For the complete documentation index, see [llms-full.txt](/llms-full.txt). # Experiments API Manage saved conversations, collections, and test suites, submit experiment runs, and read results. ## Before you begin - Enable Agent Observability on your Grafana Cloud stack. Refer to [Set up Agent Observability in Grafana Cloud](/docs/grafana-cloud/observe-and-act/agent-observability/get-started/grafana-cloud). - For configuration changes, use a Grafana identity with permission to manage the relevant resource. Refer to [Security and access controls](/docs/grafana-cloud/observe-and-act/agent-observability/privacy-and-access/security). - For experiment runs or exported scores, create a Grafana Cloud Access Policy Token with the `sigil:write` scope. For credential requirements, refer to [Choose credentials](/docs/grafana-cloud/observe-and-act/agent-observability/reference/#choose-credentials). ## Manage saved conversations Use saved conversation endpoints to bookmark telemetry conversations or create manual conversations for review and experiments. Expand table | Method | Path | Description | |----------|-----------------------------------------------------|---------------------------------------------------| | `GET` | `/api/v1/eval/saved-conversations` | List saved conversations. | | `POST` | `/api/v1/eval/saved-conversations` | Save a telemetry conversation. | | `POST` | `/api/v1/eval/saved-conversations:manual` | Create a manual conversation. | | `GET` | `/api/v1/eval/saved-conversations/{id}` | Get a saved conversation. | | `DELETE` | `/api/v1/eval/saved-conversations/{id}` | Delete a saved conversation. | | `GET` | `/api/v1/eval/saved-conversations/{id}/collections` | List collections containing a saved conversation. | ## Manage collections Use collection endpoints to group saved conversations for review or promotion into test cases. Expand table | Method | Path | Description | |----------|----------------------------------------------------|-----------------------------| | `GET` | `/api/v1/eval/collections` | List collections. | | `POST` | `/api/v1/eval/collections` | Create a collection. | | `GET` | `/api/v1/eval/collections/{id}` | Get a collection. | | `PATCH` | `/api/v1/eval/collections/{id}` | Update a collection. | | `DELETE` | `/api/v1/eval/collections/{id}` | Delete a collection. | | `GET` | `/api/v1/eval/collections/{id}/members` | List collection members. | | `POST` | `/api/v1/eval/collections/{id}/members` | Add collection members. | | `DELETE` | `/api/v1/eval/collections/{id}/members/{saved_id}` | Remove a collection member. | ## Manage test suites Use test suite endpoints to manage versioned datasets for repeatable offline experiments. Add or edit cases in a draft, then publish the version to make it a stable input for experiment runs. Expand table | Method | Path | Description | |----------|------------------------------------------------------------------------------------|------------------------------------| | `GET` | `/api/v1/eval/test-suites` | List test suites. | | `POST` | `/api/v1/eval/test-suites` | Create a test suite. | | `GET` | `/api/v1/eval/test-suites/{suite_id}` | Get a test suite and its versions. | | `PATCH` | `/api/v1/eval/test-suites/{suite_id}` | Update test suite metadata. | | `POST` | `/api/v1/eval/test-suites/{suite_id}/versions` | Create a draft version. | | `GET` | `/api/v1/eval/test-suites/{suite_id}/versions/{version}/test-cases` | List test cases in a version. | | `POST` | `/api/v1/eval/test-suites/{suite_id}/versions/{version}/test-cases` | Create or update a test case. | | `GET` | `/api/v1/eval/test-suites/{suite_id}/versions/{version}/test-cases/{test_case_id}` | Get a test case. | | `PATCH` | `/api/v1/eval/test-suites/{suite_id}/versions/{version}/test-cases/{test_case_id}` | Update a test case. | | `DELETE` | `/api/v1/eval/test-suites/{suite_id}/versions/{version}/test-cases/{test_case_id}` | Delete a test case. | | `POST` | `/api/v1/eval/test-suites/{suite_id}/versions/{version}:publish` | Publish a suite version. | Published versions are immutable. Create a new draft version when you need to change the suite. The Agent Observability SDK exposes these operations through `TestSuitesClient` and can synchronize local YAML or JSON suites with stored suite versions. ## Ingest experiment runs Use experiment run ingest endpoints from SDKs, CI jobs, and benchmark processes that execute test cases. Prefer the Agent Observability SDK, which manages these calls and finalizes experiments automatically. These endpoints use tenant API credentials. When a process needs to set machine ownership for experiment runs, trials, or artifacts, use the `X-Agento11y-Ingest-Actor` request header. Some routes still accept the deprecated `X-Sigil-Ingest-Actor` spelling for compatibility, but new integrations should use the `X-Agento11y-*` header name. Expand table | Method | Path | Description | |---------|-----------------------------------------------------------------------------------------|--------------------------------------| | `POST` | `/api/v1/experiment-runs:upsert` | Create or resume an experiment. | | `POST` | `/api/v1/experiment-runs/{experiment_id}:finalize` | Finalize an experiment. | | `POST` | `/api/v1/experiment-runs/{experiment_id}/trials` | Create or update a test-case trial. | | `PATCH` | `/api/v1/experiment-runs/{experiment_id}/trials/{trial_id}` | Update a test-case trial. | | `POST` | `/api/v1/experiment-runs/{experiment_id}/trials/{trial_id}:evaluate` | Queue evaluator scoring for a trial. | | `GET` | `/api/v1/experiment-runs/{experiment_id}/trials/{trial_id}/evaluations/{evaluation_id}` | Get trial evaluation status. | | `POST` | `/api/v1/experiment-runs/{experiment_id}/trials/{trial_id}/artifacts:upload` | Upload a trial artifact. | After you finalize an experiment, Agent Observability builds its report in the background. Set `planned_trial_count` when you create the run if the total number of attempts is known. Agent Observability uses this value to show progress for an active experiment. Use trial evaluation endpoints when you want Agent Observability to run an existing evaluator against a recorded trial conversation on demand. The trigger request accepts `evaluator_id` and an optional `evaluator_version`. The response includes an `evaluation_id`, status, attempts, and any error details. Poll the status endpoint until the evaluation reaches a terminal state. You can also set `report_role` to `primary_verdict` or `diagnostic` in the trigger request. The role applies to this trial evaluation’s score, not to the evaluator definition. The response includes the role when you supply it. For evaluator-ID-only requests, if you omit `report_role`, the score has no report role. For headline selection and pass-rate behavior, refer to [Interpret trial verdicts](/docs/grafana-cloud/observe-and-act/agent-observability/evaluate-and-improve/experiments/#interpret-trial-verdicts). For experiment traffic isolation and the immutable `online_evaluations_enabled` setting, refer to [Keep experiment traffic separate](/docs/grafana-cloud/observe-and-act/agent-observability/evaluate-and-improve/experiments/#keep-experiment-traffic-separate). ## Read experiment results Use the Agent Observability SDK to read the scores and aggregated report for an offline experiment. Expand table | Method | Path | Description | |--------|----------------------------------------|---------------------------------------| | `GET` | `/api/v1/eval/experiments/{id}/scores` | List experiment scores. | | `GET` | `/api/v1/eval/experiments/{id}/report` | Get the aggregated experiment report. | These reads use the same tenant API credentials as experiment run ingest. In Python, use `experiment.report()` to retrieve the aggregated report and `experiment.client.list_scores()` to retrieve scores. ## Next steps - [Run an experiment with the SDK](/docs/grafana-cloud/observe-and-act/agent-observability/evaluate-and-improve/run-an-experiment) - [Export scores](/docs/grafana-cloud/observe-and-act/agent-observability/reference/evaluation/#export-scores)