Grafana Cloud

Use gcx to manage Machine Learning resources

gcx is a command-line interface (CLI) that you and coding agents can use to manage Grafana resources. With gcx ml, you and coding agents can list, inspect, create, update, and delete forecasts, holidays, outlier detectors, and their alerts. Use it to check what is configured on your stack and make changes from the command line.

Before you begin

  1. Install gcx.

  2. Authenticate gcx with the preferred method for your Grafana Cloud stack.

  3. To check which configuration gcx uses, run:

    Bash
    gcx config view

Manage resources with gcx ml

Use gcx ml to inspect and manage Machine Learning resources:

CommandPurposeOperations
gcx ml jobsManage forecasts and their alertslist, get, create, update, delete, list-alerts, get-alert, create-alert, update-alert, delete-alert
gcx ml holidaysManage holidayslist, get, create, update, delete
gcx ml outlier-detectorsManage outlier detectors and their alertslist, get, create, update, delete, list-alerts, get-alert, create-alert, update-alert, delete-alert

The following examples use forecasts. To run them on holidays or outlier detectors, replace jobs with holidays or outlier-detectors:

  • List all forecasts. By default, the list shows up to 50 records. Use --limit to change the number, or --limit 0 to show all records:

    Bash
    gcx ml jobs list
  • Get a single forecast by its ID. Use -o yaml or -o json to save it to a file you can read or edit:

    Bash
    gcx ml jobs get <ID> -o yaml > forecast.yaml
  • Create a forecast from a file. For example, save the following definition as forecast.yaml and replace the data source UID and the query with the values for your stack:

    YAML
    name: request-forecast
    metric: request_forecast
    algorithm: grafana_prophet_1_0_1
    datasourceType: prometheus
    datasourceUid: <DATA_SOURCE_UID>
    queryParams:
      expr: sum(rate(http_requests_total[5m]))
    interval: 300
    trainingWindow: 7776000
    trainingFrequency: 86400

    Then create the forecast:

    Bash
    gcx ml jobs create --filename forecast.yaml

    Creating a forecast can start model training. To check the file without making changes, add --dry-run. This validates the input locally only.

  • Update a forecast. Edit the file and run update with the forecast ID. Updating a forecast can also start model training:

    Bash
    gcx ml jobs update <ID> --filename forecast.yaml
  • Delete a forecast by its ID:

    Bash
    gcx ml jobs delete <ID>

    gcx asks for confirmation before it deletes the forecast. To skip the prompt, add --force.

Manage alerts

Forecasts and outlier detectors can have alerts. Alerts belong to a forecast or outlier detector, so every alert command needs the job or outlier detector ID:

Bash
gcx ml jobs list-alerts <JOB_ID>

The following operations are available on gcx ml jobs:

  • list-alerts <JOB_ID>
  • get-alert <JOB_ID> <ALERT_ID>
  • create-alert <JOB_ID> --filename alert.yaml
  • update-alert <JOB_ID> <ALERT_ID> --filename alert.yaml
  • delete-alert <JOB_ID> <ALERT_ID>

To run the alert commands on outlier detectors, run gcx ml outlier-detectors and replace <JOB_ID> with the <OUTLIER_DETECTOR_ID>.

Note

Alerts for outlier detectors need a non-zero window, for example 5m. Don’t set threshold or anomalyCondition on them.