Static mode on Grafana Labs

Set up Grafana Agent in static mode

Fri, 06 Mar 2026 07:23:54 +0000

Set up Grafana Agent in static mode

This section includes information that helps you get Grafana Agent in static mode installed and configured.

Configure static mode

Fri, 06 Mar 2026 07:23:54 +0000

Configure static mode

The configuration of static mode is split across two places:

A YAML file
Command-line flags

The YAML file is used to configure settings which are dynamic and can be changed at runtime. The command-line flags then configure things which cannot change at runtime, such as the listen port for the HTTP server.

This file describes the YAML configuration, which is usually in a file named config.yaml.

The configuration of Grafana Agent is “stable,” but subject to breaking changes as individual features change. Breaking changes to configuration will be well-documented.

Updating configuration

The configuration file can be reloaded at runtime using the /-/reload API endpoint or sending a SIGHUP signal to the process.

Variable substitution

You can use environment variables in the configuration file to set values that need to be configurable during deployment. To enable this functionality, you must pass -config.expand-env as a command-line flag to the Agent.

To refer to an environment variable in the config file, use:

${VAR}

Where VAR is the name of the environment variable.

Each variable reference is replaced at startup by the value of the environment variable. The replacement is case-sensitive and occurs before the YAML file is parsed. References to undefined variables are replaced by empty strings unless you specify a default value or custom error text.

To specify a default value, use:

${VAR:-default_value}

Where default_value is the value to use if the environment variable is undefined. The full list of supported syntax can be found at Drone’s envsubst repository.

Regex capture group references

When using -config.expand-env, VAR must be an alphanumeric string with at least one non-digit character. If VAR is a number, the expander will assume you’re trying to use a regex capture group reference, and will coerce the result to be one.

This means references in your config file like ${1} will remain untouched, but edge cases like ${1:-default} will also be coerced to ${1}, which may be slightly unexpected.

Reloading (beta)

The configuration file can be reloaded at runtime. Read the API documentation for more information.

This functionality is in beta, and may have issues. Please open GitHub issues for any problems you encounter.

File format

To specify which configuration file to load, pass the -config.file flag at the command line. The file is written in the YAML format, defined by the scheme below. Brackets indicate that a parameter is optional. For non-list parameters the value is set to the specified default.

Generic placeholders are defined as follows:

<boolean>: a boolean that can take the values true or false
<int>: any integer matching the regular expression [1-9]+[0-9]*
<duration>: a duration matching the regular expression [0-9]+(ns|us|µs|ms|[smh])
<labelname>: a string matching the regular expression [a-zA-Z_][a-zA-Z0-9_]*
<labelvalue>: a string of unicode characters
<filename>: a valid path relative to current working directory or an absolute path.
<host>: a valid string consisting of a hostname or IP followed by an optional port number
<string>: a regular string
<secret>: a regular string that is a secret, such as a password

Support contents and default values of config.yaml:

# Configures the server of the Agent used to enable self-scraping.
[server: <server_config>]

# Configures metric collection.
# In previous versions of the agent, this field was called "prometheus".
[metrics: <metrics_config>]

# Configures log collection.
# In previous versions of the agent, this field was called "loki".
[logs: <logs_config>]

# Configures Traces trace collection.
# In previous versions of the agent, this field was called "tempo".
[traces: <traces_config>]

# Configures integrations for the Agent.
[integrations: <integrations_config>]

Remote Configuration (Experimental)

An experimental feature for fetching remote configuration files over HTTP/S can be enabled by passing the -enable-features=remote-configs flag at the command line. With this feature enabled, you may pass an HTTP/S URL to the -config.file flag.

The following flags will configure basic auth for requests made to HTTP/S remote config URLs:

-config.url.basic-auth-user <user>: the basic auth username
-config.url.basic-auth-password-file <file>: path to a file containing the basic auth password

Note
This beta feature is subject to change in future releases.

Static mode APIs (Stable)

Fri, 06 Mar 2026 07:23:54 +0000

Static mode APIs (Stable)

The API for static mode is divided into several parts:

API endpoints are stable unless otherwise noted.

Config management API (Beta)

Grafana Agent exposes a configuration management REST API for managing instance configurations when it’s running in scraping service mode.

Note
The scraping service mode is a requirement for the configuration management API, however this isn’t a prerequisite for the Agent API or Ready/Healthy API.

The following endpoints are exposed:

List configs: GET /agent/api/v1/configs
Get config: GET /agent/api/v1/configs/{name}
Update config: PUT /agent/api/v1/config/{name}
Delete config: DELETE /agent/api/v1/config/{name}

Note
If you are running Grafana Agent in a Docker container and you want to expose the API outside the Docker container, you must change the default HTTP listen address from 127.0.0.1:12345 to a valid network interface address. You can change the HTTP listen address with the command-line flag: -server.http.address=0.0.0.0:12345. For more information, refer to the Server command-line flag documentation.

You must also publish the port in Docker. Refer to Published ports in the Docker documentation for more information.

API response

All Config Management API endpoints will return responses in the following form, unless an internal service error prevents the server from responding properly:

{
  "status": "success" | "error",
  "data": {}
}

Status will be either success or error. All 2xx responses will be accompanied by a success value for the status field. 4xx and 5xx responses will provide a value of error. All requests may potentially return 500 on an internal error. Other non-500 responses will be documented per API.

The data field may or may not be present, depending on the endpoint. It provides extra information for the query. The documentation for each endpoint will describe the full response provided.

List configs

GET /agent/api/v1/configs

List configs returns a list of the named configurations currently known by the underlying KV store.

Status code: 200 on success. Response:

{
  "status": "success",
  "data": {
    "configs": [
      // list of config names:
      "a",
      "b",
      "c",
      // ...
    ]
  }
}

Get config

GET /agent/api/v1/configs/{name}

Get config returns a single configuration by name. The configuration must exist or an error will be returned. URL-encoded names will be retrieved in decoded form. e.g., hello%2Fworld will represent the config named hello/world.

Status code: 200 on success, 400 on invalid config name. Response on success:

{
  "status": "success",
  "data": {
    "value": "/* YAML configuration */"
  }
}

Update config

PUT /agent/api/v1/config/{name}
POST /agent/api/v1/config/{name}

Update config updates or adds a new configuration by name. If a configuration with the same name already exists, then it will be completely overwritten.

URL-encoded names are stored in decoded form. e.g., hello%2Fworld will represent the config named hello/world.

The request body passed to this endpoint must match the format of metrics_instance_config defined in the Configuration Reference. The name field of the configuration is ignored and the name in the URL takes precedence. The request body must be formatted as YAML.

Warning
By default, all instance configuration files that read credentials from a file on disk will be rejected. This prevents malicious users from reading the contents of arbitrary files as passwords and sending their contents to fake remote_write endpoints. To change the behavior, set dangerous_allow_reading_files to true in the scraping_service block.

Status code: 201 with a new config, 200 on updated config. Response on success:

{
  "status": "success"
}

Delete config

DELETE /agent/api/v1/config/{name}

Delete config attempts to delete a configuration by name. The named configuration must exist; deleting a nonexistent config will result in an error.

URL-encoded names will be interpreted in decoded form. e.g., hello%2Fworld will represent the config named hello/world.

Status code: 200 on success, 400 with invalid config name. Response on success:

{
  "status": "success"
}

Agent API

List current running instances of metrics subsystem

GET /agent/api/v1/metrics/instances

Note
The deprecated alias is /agent/api/v1/instances

Status code: 200 on success. Response on success:

{
  "status": "success",
  "data": [
    <strings of instance names that are currently running>
  ]
}

List current scrape targets of metrics subsystem

GET /agent/api/v1/metrics/targets

Note
The deprecated alias is /agent/api/v1/targets

This endpoint collects all metrics subsystem targets known to the Agent across all running instances. Only targets being scraped from the local Agent will be returned. If running in scraping service mode, this endpoint must be invoked in all Agents separately to get the combined set of targets across the whole Agent cluster.

The labels fields shows the labels that will be added to metrics from the target, while the discovered_labels field shows all labels found during service discovery.

Status code: 200 on success. Response on success:

{
  "status": "success",
  "data": [
    {
      "instance": <string, instance config name>,
      "target_group": <string, scrape config group name>,
      "endpoint": <string, URL being scraped>
      "state": <string, one of up, down, unknown>,
      "discovered_labels": {
        "__address__": "<address>",
        ...
      },
      "labels": {
        "label_a": "value_a",
        ...
      },
      "last_scrape": <string, RFC 3339 timestamp of last scrape>,
      "scrape_duration_ms": <number, last scrape duration in milliseconds>,
      "scrape_error": <string, last error. empty if scrape succeeded>
    },
    ...
  ]
}

Accept remote_write requests

POST /agent/api/v1/metrics/instance/{instance}/write

This endpoint accepts Prometheus-compatible remote_write POST requests, and appends their contents into an instance’s WAL.

Replace {instance} with the name of the metrics instance from your config file. For example, this block defines the “dev” and “prod” instances:

metrics:
  configs:
  - name: dev     # /agent/api/v1/metrics/instance/dev/write
    ...
  - name: prod    # /agent/api/v1/metrics/instance/prod/write
    ...

Status code: 204 on success, 400 for bad requests related to the provided instance or POST payload format and content, 500 for cases where appending to the WAL failed.

List current running instances of logs subsystem

GET /agent/api/v1/logs/instances

Status code: 200 on success. Response on success:

{
  "status": "success",
  "data": [
    <strings of instance names that are currently running>
  ]
}

List current scrape targets of logs subsystem

GET /agent/api/v1/logs/targets

This endpoint collects all logs subsystem targets known to the Agent across all running instances. Only targets being scraped from Promtail will be returned.

The labels fields shows the labels that will be added to metrics from the target, while the discovered_labels field shows all labels found during service discovery.

Status code: 200 on success. Response on success:

{
  "status": "success",
  "data": [
    {
      "instance": "default",
      "target_group": "varlogs",
      "type": "File",
      "labels": {
        "job": "varlogs"
      },
      "discovered_labels": {
        "__address__": "localhost",
        "__path__": "/var/log/*log",
        "job": "varlogs"
      },
      "ready": true,
      "details": {
        "/var/log/alternatives.log": 13386,
        "/var/log/apport.log": 0,
        "/var/log/auth.log": 37009,
        "/var/log/bootstrap.log": 107347,
        "/var/log/dpkg.log": 374420,
        "/var/log/faillog": 0,
        "/var/log/fontconfig.log": 11629,
        "/var/log/gpu-manager.log": 1541,
        "/var/log/kern.log": 782582,
        "/var/log/lastlog": 0,
        "/var/log/syslog": 788450
      }
    }
  ]
}

Reload configuration file (beta)

This endpoint is currently in beta and may have issues. Please open any issues you encounter.

GET /-/reload
POST /-/reload

This endpoint will re-read the configuration file from disk and refresh the entire state of the Agent to reflect the new file on disk:

HTTP Server
Prometheus metrics subsystem
Loki logs subsystem
Tempo traces subsystem
Integrations

Valid configurations will be applied to each of the subsystems listed above, and /-/reload will return with a status code of 200 once all subsystems have been updated. Malformed configuration files (invalid YAML, failed validation checks) will be immediately rejected with a status code of 400.

Well-formed configuration files can still be invalid for various reasons, such as not having permissions to read the WAL directory. Issues such as these will cause per-subsystem problems while reloading the configuration, and will leave that subsystem in an undefined state. Specific errors encountered during reload will be logged, and should be fixed before calling /-/reload again.

Status code: 200 on success, 400 otherwise.

Show configuration file

GET /-/config

This endpoint prints out the currently loaded configuration the Agent is using. The returned YAML has defaults applied, and only shows changes to the state that validated successfully, so the results will not identically match the configuration file on disk.

Status code: 200 on success.

Generate support bundle

GET /-/support?duration=N

This endpoint returns a ‘support bundle’, a zip file that contains information about a running agent, and can be used as a baseline of information when trying to debug an issue.

The duration parameter is optional, must be less than or equal to the configured HTTP server write timeout, and if not provided, defaults to it. The endpoint is only exposed to the agent’s HTTP server listen address, which defaults to localhost:12345.

The support bundle contains all information in plain text, so that it can be inspected before sharing, to verify that no sensitive information has leaked.

In addition, you can inspect the supportbundle package to verify the code that is being used to generate these bundles.

A support bundle contains the following data:

agent-config.yaml contains the current agent configuration (when the -config.enable-read-api flag is passed).
agent-logs.txt contains the agent logs during the bundle generation.
agent-metadata.yaml contains the agent’s build version, operating system, architecture, uptime, plus a string payload defining which extra agent features have been enabled via command-line flags.
agent-metrics-instances.json and agent-metrics-targets.json contain the active metric subsystem instances and the discovered scrape targets for each one.
agent-logs-instances.json and agent-logs-targets.json contains the active logs subsystem instances and the discovered log targets for each one.
agent-metrics.txt contains a snapshot of the agent’s internal metrics.
The pprof/ directory contains Go runtime profiling data (CPU, heap, goroutine, mutex, block profiles) as exported by the pprof package.

Integrations API (Experimental)

WARNING: This API is currently only available when the experimental integrations revamp is enabled. Both the revamp and this API are subject to change while they are still experimental.

Integrations SD API

GET /agent/api/v1/metrics/integrations/sd

This endpoint returns all running metrics-based integrations. It conforms to the Prometheus http_sd_config API. Targets include integrations regardless of autoscrape being enabled; this allows for manually configuring scrape jobs to collect metrics from an integration running on an external agent.

The following labels will be present on all returned targets:

instance: The unique instance ID of the running integration.
job: integrations/<__meta_agent_integration_name>
agent_hostname: hostname:port of the agent running the integration.
__meta_agent_integration_name: The name of the integration.
__meta_agent_integration_instance: The unique instance ID for the running integration.
__meta_agent_integration_autoscrape: 1 if autoscrape is enabled for this integration, 0 otherwise.

To reduce the load on the agent’s HTTP server, the following query parameters may also be provided to the URL:

integrations: Comma-delimited list of integrations to return. i.e., agent,node_exporter.
instance: Return all integrations matching a specific value for instance.

Status code: 200 if successful. Response on success:

[
  {
    "targets": [ "<host>", ... ],
    "labels": {
      "<labelname>": "<labelvalue>", ...
    }
  },
  ...
]

Integrations autoscrape targets

GET /agent/api/v1/metrics/integrations/targets

This endpoint returns all integrations for which autoscrape is enabled. The response is identical to /agent/api/v1/metrics/targets.

Status code: 200 on success. Response on success:

{
  "status": "success",
  "data": [
    {
      "instance": <string, metrics instance where autoscraped metrics are sent>,
      "target_group": <string, scrape config group name>,
      "endpoint": <string, URL being scraped>
      "state": <string, one of up, down, unknown>,
      "discovered_labels": {
        "__address__": "<address>",
        ...
      },
      "labels": {
        "label_a": "value_a",
        ...
      },
      "last_scrape": <string, RFC 3339 timestamp of last scrape>,
      "scrape_duration_ms": <number, last scrape duration in milliseconds>,
      "scrape_error": <string, last error. empty if scrape succeeded>
    },
    ...
  ]
}

Ready / health API

Readiness check

GET /-/ready

Status code: 200 if ready.

Response:

Agent is Ready.

Healthiness check

GET /-/healthy

Status code: 200 if healthy.

Response:

Agent is Healthy.

Operation guide

Fri, 06 Mar 2026 07:23:54 +0000

Operation guide

This guide helps you operate Grafana Agent.

Horizontal Scaling

There are three options to horizontally scale your deployment of Grafana Agents:

Host filtering requires you to run one Agent on every machine you wish to collect metrics from. Agents will only collect metrics from the machines they run on.
Hashmod sharding allows you to roughly shard the discovered set of targets by using hashmod/keep relabel rules.
The scraping service allows you to cluster Grafana Agents and have them distribute per-tenant configs throughout the cluster.

Each has their own set of tradeoffs:

Host Filtering (Beta)
- Pros
  - Does not need specialized configs per agent
  - No external dependencies required to operate
- Cons
  - Can cause significant load on service discovery APIs
  - Requires each Agent to have the same list of scrape configs/remote_writes
Hashmod sharding (Stable)
- Pros
  - Exact control on the number of shards to run
  - Smaller load on SD compared to host filtering (as there are a smaller # of Agents)
  - No external dependencies required to operate
- Cons
  - Each Agent must have a specialized config with their shard number inserted into the hashmod/keep relabel rule pair.
  - Requires each Agent to have the same list of scrape configs/remote_writes, with the exception of the hashmod rule being different.
  - Hashmod is not consistent hashing, so up to 100% of jobs will move to a new machine when scaling shards.
Scraping service (Beta)
- Pros
  - Agents don’t have to have a synchronized set of scrape configs / remote_writes (they pull from a centralized location).
  - Exact control on the number of shards to run.
  - Uses consistent hashing, so only 1/N jobs will move to a new machine when scaling shards.
  - Smallest load on SD compared to host filtering, as only one Agent is responsible for a config.
- Cons
  - Centralized configs must discover a minimal set of targets to distribute evenly.
  - Requires running a separate KV store to store the centralized configs.
  - Managing centralized configs adds operational burden over managing a config file.

Host filtering (Beta)

Host filtering implements a form of “dumb sharding,” where operators may deploy one Grafana Agent instance per machine in a cluster, all using the same configuration, and the Grafana Agents will only scrape targets that are running on the same node as the Agent.

Running with host_filter: true means that if you have a target whose host machine is not also running a Grafana Agent process, that target will not be scraped!

Host filtering is usually paired with a dedicated Agent process that is used for scraping targets that are running outside of a given cluster. For example, when running the Grafana Agent on GKE, you would have a DaemonSet with host_filter for scraping in-cluster targets, and a single dedicated Deployment for scraping other targets that are not running on a cluster node, such as the Kubernetes control plane API.

If you want to scale your scrape load without host filtering, you can use the scraping service instead.

The host name of the Agent is determined by reading $HOSTNAME. If $HOSTNAME isn’t defined, the Agent will use Go’s os.Hostname to determine the hostname.

The following meta-labels are used to determine if a target is running on the same machine as the Agent:

__address__
__meta_consul_node
__meta_dockerswarm_node_id
__meta_dockerswarm_node_hostname
__meta_dockerswarm_node_address
__meta_kubernetes_pod_node_name
__meta_kubernetes_node_name
__host__

The final label, __host__, isn’t a label added by any Prometheus service discovery mechanism. Rather, __host__ can be generated by using host_filter_relabel_configs. This allows for custom relabeling rules to determine the hostname where the predefined ones fail. Relabeling rules added with host_filter_relabel_configs are temporary and just used for the host_filtering mechanism. Full relabeling rules should be applied in the appropriate scrape_config instead.

Note that scrape_config relabel_configs do not apply to the host filtering logic; only host_filter_relabel_configs will work.

If the determined hostname matches any of the meta labels, the discovered target is allowed. Otherwise, the target is ignored, and will not show up in the targets API.

Hashmod sharding (Stable)

Grafana Agents can be sharded by using a pair of hashmod/keep relabel rules. These rules will hash the address of a target and modulus it with the number of Agent shards that are running.

scrape_configs:
- job_name: some_job
  # Add usual service discovery here, such as static_configs
  relabel_configs:
  - source_labels: [__address__]
    modulus:       4    # 4 shards
    target_label:  __tmp_hash
    action:        hashmod
  - source_labels: [__tmp_hash]
    regex:         ^1$  # This is the 2nd shard
    action:        keep

Add the relabel_configs to all of your scrape_config blocks. Ensure that each running Agent shard has a different value for the regex; the first Agent shard should have ^0$, the second should have ^1$, and so on, up to ^3$.

This sharding mechanism means each Agent will ignore roughly 1/N of the total targets, where N is the number of shards. This allows for horizontal scaling the number of Agents and distributing load between them.

Note that the hashmod used here is not a consistent hashing algorithm; this means that changing the number of shards may cause any number of targets to move to a new shard, up to 100%. When moving to a new shard, any existing data in the WAL from the old machine is effectively discarded.

Prometheus instances

The Grafana Agent defines a concept of a Prometheus Instance, which is its own mini Prometheus-lite server. The instance runs a combination of Prometheus service discovery, scraping, a WAL for storage, and remote_write.

Instances allow for fine grained control of what data gets scraped and where it gets sent. Users can easily define two Instances that scrape different subsets of metrics and send them to two completely different remote_write systems.

Instances are especially relevant to the scraping service mode, where breaking up your scrape configs into multiple Instances is required for sharding and balancing scrape load across a cluster of Agents.

The v0.5.0 release of the Agent introduced the concept of instance sharing, which combines scrape_configs from compatible instance configs into a single, shared Instance. Instance configs are compatible when they have no differences in configuration with the exception of what they scrape. remote_write configs may also differ in the order which endpoints are declared, but the unsorted remote_writes must still be an exact match.

In the shared instances mode, the name field of remote_write configs is ignored. The resulting remote_write configs will have a name identical to the first six characters of the group name and the first six characters of the hash from that remote_write config separated by a -.

The shared instances mode is the new default, and the previous behavior is deprecated. If you wish to restore the old behavior, set instance_mode: distinct in the metrics_config block of your config file.

Shared instances are completely transparent to the user with the exception of exposed metrics. With instance_mode: shared, metrics for Prometheus components (WAL, service discovery, remote_write, etc.) have a instance_group_name label, which is the hash of all settings used to determine the shared instance. When instance_mode: distinct is set, the metrics for Prometheus components will instead have an instance_name label, which matches the name set on the individual Instance config. It is recommended to use the default of instance_mode: shared unless you don’t mind the performance hit and really need granular metrics.

Users can use the targets API to see all scraped targets, and the name of the shared instance they were assigned to.

Release notes

Fri, 06 Mar 2026 07:23:54 +0000

Release notes

The release notes provide information about deprecations and breaking changes in Grafana Agent static mode.

For a complete list of changes to Grafana Agent, with links to pull requests and related issues when available, refer to the Changelog.

Note: These release notes are specific to Grafana Agent static mode. Other release notes for the different Grafana Agent variants are contained on separate pages:

Static mode Kubernetes operator release notes

Flow mode release notes

v0.37

Breaking change: The default value of `retry_on_http_429` is overriden to `true` for the `queue_config` in `remote_write` in `metrics` config.

Note
The default set by Grafana Agent Static Mode is different than the default set by Prometheus.

The Prometheus default value for retry_on_http_429 is set to true for the queue_config in remote_write. This changed default setting allows the agent to retry sending data when it receives an HTTP 429 error and helps avoid losing data in metric pipelines.

If you explicitly set the retry_on_http_429, no action is required.
If you do not explicitly set retry_on_http_429 and you do not want to retry on HTTP 429, make sure you set it to false when you upgrade to this new version.

Breaking change: Renamed `non_indexed_labels` Loki processing stage to `structured_metadata`.

If you use the Loki processing stage in your Agent configuration, you must rename the non_indexed_labels pipeline stage definition to structured_metadata.

Old configuration example:

  pipeline_stages:
    - logfmt:
        mapping:
          app:
    - non_indexed_labels:
        app:

New configuration example:

  pipeline_stages:
    - logfmt:
        mapping:
          app:
    - structured_metadata:
        app:

v0.35

Breaking change: Jaeger remote sampling no longer configurable using the Jaeger receiver

Jaeger remote sampling used to be configured using the Jaeger receiver configuration. This receiver was updated to a new version, where support for remote sampling in the receiver was removed.

Jaeger remote sampling is available as a separate configuration field starting in v0.35.3.

Old configuration example:

receivers:
  jaeger:
    protocols:
      grpc:
    remote_sampling:
      strategy_file: /etc/agent/strategies.json
      strategy_file_reload_interval: 1s

New configuration example:

jaeger_remote_sampling:
  - source:
      file: /etc/agent/strategies.json
      reload_interval: 1s

Breaking change: `auth` and `version` attributes from `walk_params` block of SNMP integration have been removed

The SNMP integrations (both v1 and v2) wrap a new version of SNMP exporter which introduces a new configuration file format. This new format separates the walk and metric mappings from the connection and authentication settings. This allows for easier configuration of different auth params without having to duplicate the full walk and metric mapping.

Old configuration example:

    snmp_targets:
      - name: network_switch_1
        address: 192.168.1.2
        module: if_mib
        walk_params: public
        auth: public
    walk_params:
      public:
        retries: 2
        version: 2
        auth:
          community: public

New configuration example:

    snmp_targets:
      - name: network_switch_1
        address: 192.168.1.2
        module: if_mib
        walk_params: public
        auth: public
    walk_params:
      public:
        retries: 2

See Module and Auth Split Migration for more details.

Removal of Dynamic Configuration

The experimental feature Dynamic Configuration has been removed. The use case of dynamic configuration will be replaced with Modules in Grafana Agent Flow.

Breaking change: Removed and renamed tracing metrics

In the traces subsystem for Static mode some metrics are removed and others are renamed. The reason for the removal is a bug which caused the metrics to be incorrect if more than one instance of a traces configuration is specified.

Removed metrics:

“blackbox_exporter_config_last_reload_success_timestamp_seconds” (gauge)
“blackbox_exporter_config_last_reload_successful” (gauge)
“blackbox_module_unknown_total” (counter)
“traces_processor_tail_sampling_count_traces_sampled” (counter)
“traces_processor_tail_sampling_new_trace_id_received” (counter)
“traces_processor_tail_sampling_sampling_decision_latency” (histogram)
“traces_processor_tail_sampling_sampling_decision_timer_latency” (histogram)
“traces_processor_tail_sampling_sampling_policy_evaluation_error” (counter)
“traces_processor_tail_sampling_sampling_trace_dropped_too_early” (counter)
“traces_processor_tail_sampling_sampling_traces_on_memory” (gauge)
“traces_receiver_accepted_spans” (counter)
“traces_receiver_refused_spans” (counter)
“traces_exporter_enqueue_failed_log_records” (counter)
“traces_exporter_enqueue_failed_metric_points” (counter)
“traces_exporter_enqueue_failed_spans” (counter)
“traces_exporter_queue_capacity” (gauge)
“traces_exporter_queue_size” (gauge)

Renamed metrics:

“traces_receiver_refused_spans” is renamed to “traces_receiver_refused_spans_total”
“traces_receiver_accepted_spans” is renamed to “traces_receiver_refused_spans_total”
“traces_exporter_sent_metric_points” is renamed to “traces_exporter_sent_metric_points_total”

v0.33

Symbolic links in Docker containers removed

We’ve removed the deprecated symbolic links to /bin/agent* in Docker containers, as planned in v0.31. In case you’re setting a custom entrypoint, use the new binaries that are prefixed with /bin/grafana*.

Deprecation of Dynamic Configuration

Dynamic Configuration will be removed in v0.34. The use case of dynamic configuration will be replaced with Modules in Grafana Agent Flow.

v0.32

Breaking change: `node_exporter` configuration options changed

With the update of the node_exporter integration to use v1.5.0, configuration options for the diskstats collector have changed names:

diskstats_ignored_devices is now diskstats_device_exclude in the static mode configuration.
ignored_devices is now device_exclude in the Flow component configuration.

v0.31.1

Breaking change: all Windows executables are now zipped

All release Windows .exe files are now zipped. Prior to v0.31, only grafana-agent-installer.exe was unzipped.

This fixes an issue from v0.31.0 where all .exe files were accidentally left unzipped.

v0.31

Breaking change: binary names are now prefixed with `grafana-`

As first announced in v0.29, release binary names are now prefixed with grafana-:

agent is now grafana-agent.
agentctl is now grafana-agentctl.

For the grafana/agent Docker container, the entrypoint is now /bin/grafana-agent. A symbolic link from /bin/agent to the new binary has been added.

For the grafana/agentctl Docker container, the entrypoint is now /bin/grafana-agentctl. A symbolic link from /bin/agentctl to the new binary has been added.

These symbolic links will be removed in v0.33. Custom entrypoints must be updated prior to v0.33 to use the new binaries before the symbolic links get removed.

v0.30

Breaking change: `ebpf_exporter` integration removed

The ebpf_exporter version bundled in the Agent used bcc to compile eBPF programs at runtime. This made it hard to run successfully, as the dynamic linking approach required a compiler, the correct kernel headers, as well as an exact match of the libbpf toolchain on the host system. For these reasons, we’ve decided to remove the ebpf_exporter integration.

Running the ebpf_exporter integration is now deprecated and will result in configuration errors. To continue using the same configuration file, remove the ebpf block.

v0.29

Breaking change: JSON-encoded traces from OTLP versions below 0.16.0 are no longer supported

Grafana Agent’s OpenTelemetry Collector dependency has been updated from v0.55.0 to v0.61.0. OpenTelemetry Collector v0.58.0 no longer translates from InstrumentationLibrary to Scope.

This means that JSON-encoded traces that still use InstrumentationLibrary will be dropped. To work around this issue, either send traces using protobuf or update your OTLP protocol version to v0.16.0 or newer.

Deprecation: binary names will be prefixed with `grafana-` in v0.31.0

The binary names agent and agentctl have been deprecated and will be renamed to grafana-agent and grafana-agentctl respectively in the v0.31.0 release.

As part of this change, the Docker containers for the v0.31.0 release will include symbolic links from the old binary names to the new binary names.

There is no action to take at this time.

v0.26

Breaking change: Deprecated YAML fields in `server` block removed

The YAML fields which were first deprecated in the v0.24.0 release have now been removed, replaced by equivalent command line flags. Please refer to the original deprecation notice for instructions for how to migrate to the command line flags.

Breaking change: Reconcile sampling policies between Agent and OTel

Configuring sampling policies in the tail_sampling block of the traces block has been changed to be equal with the upstream configuration of the OTel processor. It now requires that the policy type is specified.

Old configuration:

traces:
  configs:
    - name: default
    ...
    tail_sampling:
      policies:
      - latency:
          threshold_ms: 100

New configuration:

traces:
  configs:
    - name: default
    ...
    tail_sampling:
      policies:
      - type: latency
        latency:
          threshold_ms: 100

v0.24

Breaking change: Integrations renamed when `integrations-next` feature flag is used

This change only applies to users utilizing the integrations-next feature flag. Nothing is changed for configuring integrations when the feature flag is not used.

Most integrations-next integrations have been renamed to describe what telemetry data they generate instead of the projects they are powered by.

consul_exporter is now consul
dnsmasq_exporter is now dnsmasq
elasticsearch_exporter is now elasticsearch
github_exporter is now github
kafka_exporter is now kafka
memcached_exporter is now memcached
mongodb_exporter is now mongodb
mysqld_exporter is now mysql
- Note that it is mysql and not mysqld
postgres_exporter is now postgres
process_exporter is now process
redis_exporter is now redis
statsd_exporter is now statsd
windows_exporter is now windows

Keys in the integrations config block have changed to match the above:

integrations.consul_exporter_configs is now integrations.consul_configs
integrations.dnsmasq_exporter_configs is now integrations.dnsmasq_configs
integrations.elasticsearch_exporter_configs is now integrations.elasticsearch_configs
integrations.github_exporter_configs is now integrations.github_configs
integrations.kafka_exporter_configs is now integrations.kafka_configs
integrations.memcached_exporter_configs is now integrations.memcached_configs
integrations.mongodb_exporter_configs is now integrations.mongodb_configs
integrations.mysqld_exporter_configs is now integrations.mysql_configs
integrations.postgres_exporter_configs is now integrations.postgres_configs
integrations.process_exporter is now integrations.process
integrations.redis_exporter_configs is now integrations.redis_configs
integrations.statsd_exporter is now integrations.statsd
integrations.windows_exporter is now integrations.windows

Integrations not listed here have not changed; node_exporter still has the same name.

This change propagates to the label values generated by these integrations. For example, job="integrations/redis_exporter will now be job="redis".

Change: Separating YAML and command line flags

As of this release, we are starting to separate what can be configured within the YAML file, and what can be configured by command line flag. Previously, there was a lot of overlap: many things could be set by both command line flag and configuration file, with command line flags taking precedence.

The configuration file will be used for settings that can be updated at runtime using the /-/reload endpoint or sending SIGHUP. Meanwhile, command line flags will be used for settings that must remain consistent throughout the process lifetime, such as the HTTP listen port.

This conceptual change will require some number of breaking changes. This release focuses on the server block of the YAML, which has historically caused the most issues with the /-/reload endpoint working correctly.

There may be more breaking changes in the future as we identify more settings that must be static and moved to flags. These changes will either be moving a YAML field to a flag or moving a flag to a YAML field. After we are done with this migration, there will be no overlap between flags and the YAML file.

Deprecation on YAML fields in `server` block that have flags

The server block is the most impacted by the separation of flags/fields. Instead of making a breaking change immediately, we are deprecating these fields.

NOTE: These deprecated fields will be removed in the v0.26.0 release. We will communicate when other deprecated features will be removed when a timeline is established.

The following fields are now deprecated in favor of command line flags:

server.register_instrumentation
server.graceful_shutdown_timeout
server.log_source_ips_enabled
server.log_source_ips_header
server.log_source_ips_regex
server.http_listen_network
server.http_listen_address
server.http_listen_port
server.http_listen_conn_limit
server.http_server_read_timeout
server.http_server_write_timout
server.http_server_idle_timeout
server.grpc_listen_network
server.grpc_listen_address
server.grpc_listen_port
server.grpc_listen_conn_limit
server.grpc_server_max_recv_msg_size
server.grpc_server_max_send_msg_size
server.grpc_server_max_concurrent_streams
server.grpc_server_max_connection_idle
server.grpc_server_max_connection_age
server.grpc_server_max_connection_age_grace
server.grpc_server_keepalive_time
server.grpc_server_keepalive_timeout
server.grpc_server_min_time_between_pings
server.grpc_server_ping_without_stream_allowed

This is most of the fields; the remaining non-deprecated fields are server.log_level, server.log_format, server.http_tls_config, and server.grpc_tls_config, which support dynamic updating.

Breaking change: Removing support for dynamically updating deprecated server fields

/-/reload will now fail if any of the deprecated server block fields have changed. It is still valid to change a non-deprecated field (i.e., changing the log level).

Breaking change: Server-specific command line flags have changed

The following flags are new:

-server.http.enable-tls
-server.grpc.enable-tls
-server.http.address
-server.grpc.address

The following flags have been removed:

-log.level (replacement: use YAML field server.log_level)
-log.format (replacement: use YAML field server.log_format)
-server.http-tls-cert-path (replacement: use YAML field server.http_tls_config)
-server.http-tls-key-path (replacement: use YAML field server.http_tls_config)
-server.http-tls-client-auth (replacement: use YAML field server.http_tls_config)
-server.http-tls-ca-path (replacement: use YAML field server.http_tls_config)
-server.grpc-tls-cert-path (replacement: use YAML field server.grpc_tls_config)
-server.grpc-tls-key-path (replacement: use YAML field server.grpc_tls_config)
-server.grpc-tls-client-auth (replacement: use YAML field server.grpc_tls_config)
-server.grpc-tls-ca-path (replacement: use YAML field server.grpc_tls_config)
-server.http-listen-address (replacement: use the new -server.http.address flag, which combines host and port)
-server.http-listen-port (replacement: use the new -server.http.address flag, which combines host and port)
-server.grpc-listen-address (replacement: use the new -server.grpc.address flag, which combines host and port)
-server.grpc-listen-port (replacement: use the new -server.grpc.address flag, which combines host and port)
-server.path-prefix (no replacement; this flag was unsupported and caused undefined behavior when set)

The following flags have been renamed:

-server.log-source-ips-enabled has been renamed to -server.log.source-ips.enabled
-server.log-source-ips-header has been renamed to -server.log.source-ips.header
-server.log-source-ips-regex has been renamed to -server.log.source-ips.regex
-server.http-listen-network has been renamed to -server.http.network
-server.http-conn-limit has been renamed to -server.http.conn-limit
-server.http-read-timeout has been renamed to -server.http.read-timeout
-server.http-write-timeout has been renamed to -server.http.write-timeout
-server.http-idle-timeout has been renamed to -server.http.idle-timeout
-server.grpc-listen-network has been renamed to -server.grpc.network
-server.grpc-conn-limit has been renamed to -server.grpc.conn-limit
-server.grpc-max-recv-msg-size-bytes has been renamed to -server.grpc.max-recv-msg-size-bytes
-server.grpc-max-send-msg-size-bytes has been renamed to -server.grpc.max-send-msg-size-bytes
-server.grpc-max-concurrent-streams has been renamed to -server.grpc.max-concurrent-streams

Breaking change: New TLS flags required for enabling TLS

The two new flags, -server.http.enable-tls and -server.grpc.enable-tls now must be provided for TLS support to be enabled.

This is a change over the previous behavior where TLS was automatically enabled when a certificate pair was provided.

Breaking change: Default HTTP/gRPC address changes

The HTTP and gRPC listen addresses now default to 127.0.0.1:12345 and 127.0.0.1:12346 respectively.

If running inside of a container, you must change these to 0.0.0.0 to externally communicate with the agent’s HTTP server.

The listen addresses may be changed via -server.http.address and -server.grpc.address respectively.

Breaking change: Removal of `-reload-addr` and `-reload-port` flags

The -reload-addr and -reload-port flags have been removed. They were initially added to workaround an issue where reloading a changed server block would cause the primary HTTP server to restart. As the HTTP server settings are now static, this can no longer happen, and as such the flags have been removed.

Change: In-memory autoscrape for integrations-next

This change is only relevant to those using the integrations-next feature flag.

In-memory connections will now be used for autoscraping-enabled integrations. This is a change over the previous behavior where autoscraping integrations would connect to themselves over the network. As a result of this change, the integrations.client_config field is no longer necessary and has been removed.

v0.22

`node_exporter` integration deprecated field names

The following field names for the node_exporter integration are now deprecated:

netdev_device_whitelist is deprecated in favor of netdev_device_include.
netdev_device_blacklist is deprecated in favor of netdev_device_exclude.
systemd_unit_whitelist is deprecated in favor of systemd_unit_include.
systemd_unit_blacklist is deprecated in favor of systemd_unit_exclude.
filesystem_ignored_mount_points is deprecated in favor of filesystem_mount_points_exclude.
filesystem_ignored_fs_types is deprecated in favor of filesystem_fs_types_exclude.

This change aligns with the equivalent flag names also being deprecated in the upstream node_exporter.

Support for the old field names will be removed in a future version. A warning will be logged if using the old field names when the integration is enabled.

v0.21.2, v0.20.1

Disabling of config retrieval endpoints

These two patch releases, as part of a fix for CVE-2021-41090, disable the /-/config and /agent/api/v1/configs/{name} endpoints by default. Pass the --config.enable-read-api flag at the command line to re-enable them.

v0.21

Integrations: Change in how instance labels are handled (Breaking change)

Integrations will now use a SUO-specific instance label value. Integrations that apply to a whole machine or agent will continue to use <agent machine hostname>:<agent listen port>, but integrations that connect to an external system will now infer an appropriate value based on the config for that specific integration. Please refer to the documentation for each integration for which defaults are used.

Note: In some cases, a default value for instance cannot be inferred. This is the case for mongodb_exporter and postgres_exporter if more than one SUO is being connected to. In these cases, the instance value can be manually set by configuring the instance field on the integration. This can also be useful if two agents infer the same value for instance for the same integration.

As part of this change, the agent_hostname label is permanently affixed to self-scraped integrations and cannot be disabled. This disambiguates multiple agents using the same instance label for an integration, and allows users to identify which agents need to be updated with an override for instance.

Both use_hostname_label and replace_instance_label are now both deprecated and ignored from the YAML file, permanently treated as true. A future release will remove these fields, causing YAML errors on load instead of being silently ignored.

v0.20

Traces: Changes to receiver’s TLS config (Breaking change).

Upgrading to OpenTelemetry v0.36.0 contains a change in the receivers TLS config. TLS params have been changed from being squashed to being in its own block. This affect the jaeger receiver’s remote_sampling config.

Example old config:

receivers:
  jaeger:
    protocols:
      grpc: null,
    remote_sampling:
      strategy_file: <file_path>
      insecure: true

Example new config:

receivers:
  jaeger:
    protocols:
      grpc: null,
    remote_sampling:
      strategy_file: <file_path>
      tls:
        insecure: true

Traces: push_config is no longer supported (Breaking change)

push_config was deprecated in favor of remote_write in v0.14.0, while maintaining backwards compatibility. Refer to the deprecation announcement for how to upgrade.

Traces: legacy OTLP gRPC port no longer default port

OTLP gRPC receivers listen at port 4317 by default, instead of at port 55680. This goes in line with OTLP legacy port deprecation.

To upgrade, point the client instrumentation push endpoint to :4317 if using the default OTLP gRPC endpoint.

v0.19

Traces: Deprecation of “tempo” in config and metrics. (Deprecation)

The term tempo in the config has been deprecated of favor of traces. This change is to make intent clearer.

Example old config:

tempo:
  configs:
    - name: default
      receivers:
        jaeger:
          protocols:
            thrift_http:

Example of new config:

traces:
  configs:
    - name: default
      receivers:
        jaeger:
          protocols:
            thrift_http:

Any tempo metrics have been renamed from tempo_* to traces_*.

Tempo: split grouping by trace from tail sampling config (Breaking change)

Load balancing traces between agent instances has been moved from an embedded functionality in tail sampling to its own configuration block. This is done due to more processor benefiting from receiving consistently receiving all spans for a trace in the same agent to be processed, such as service graphs.

As a consequence, tail_sampling.load_balancing has been deprecated in favor of a load_balancing block. Also, port has been renamed to receiver_port and moved to the new load_balancing block.

Example old config:

tail_sampling:
  policies:
    - always_sample:
  port: 4318
  load_balancing:
    exporter:
      insecure: true
    resolver:
      dns:
        hostname: agent
        port: 4318

Example new config:

tail_sampling:
  policies:
    - always_sample:
load_balancing:
  exporter:
    insecure: true
  resolver:
    dns:
      hostname: agent
      port: 4318
  receiver_port: 4318

Metrics: Deprecation of “prometheus” in config. (Deprecation)

The term prometheus in the config has been deprecated of favor of metrics. This change is to make it clearer when referring to Prometheus or another Prometheus-like database, and configuration of Grafana Agent to send metrics to one of those systems.

Old configs will continue to work for now, but support for the old format will eventually be removed. To migrate your config, change the prometheus key to metrics.

Example old config:

prometheus:
  configs:
    - name: default
      host_filter: false
      scrape_configs:
        - job_name: local_scrape
          static_configs:
            - targets: ['127.0.0.1:12345']
              labels:
                cluster: 'localhost'
      remote_write:
        - url: http://localhost:9009/api/prom/push

Example new config:

metrics:
  configs:
    - name: default
      host_filter: false
      scrape_configs:
        - job_name: local_scrape
          static_configs:
            - targets: ['127.0.0.1:12345']
              labels:
                cluster: 'localhost'
      remote_write:
        - url: http://localhost:9009/api/prom/push

Tempo: prom_instance rename (Breaking change)

As part of prometheus being renamed to metrics, the spanmetrics prom_instance field has been renamed to metrics_instance. This is a breaking change, and the old name will no longer work.

Example old config:

tempo:
  configs:
  - name: default
    spanmetrics:
      prom_instance: default

Example new config:

tempo:
  configs:
  - name: default
    spanmetrics:
      metrics_instance: default

Logs: Deprecation of “loki” in config. (Deprecation)

The term loki in the config has been deprecated of favor of logs. This change is to make it clearer when referring to Grafana Loki, and configuration of Grafana Agent to send logs to Grafana Loki.

Old configs will continue to work for now, but support for the old format will eventually be removed. To migrate your config, change the loki key to logs.

Example old config:

loki:
  positions_directory: /tmp/loki-positions
  configs:
  - name: default
    clients:
      - url: http://localhost:3100/loki/api/v1/push
    scrape_configs:
    - job_name: system
      static_configs:
      - targets: ['localhost']
        labels:
          job: varlogs
          __path__: /var/log/*log

Example new config:

logs:
  positions_directory: /tmp/loki-positions
  configs:
  - name: default
    clients:
      - url: http://localhost:3100/loki/api/v1/push
    scrape_configs:
    - job_name: system
      static_configs:
      - targets: ['localhost']
        labels:
          job: varlogs
          __path__: /var/log/*log

Tempo: Deprecation of “loki” in config. (Deprecation)

As part of the loki to logs rename, parts of the automatic_logging component in Tempo have been updated to refer to logs_instance instead.

Old configurations using loki_name, loki_tag, or backend: loki will continue to work as of this version, but support for the old config format will eventually be removed.

Example old config:

tempo:
  configs:
  - name: default
    automatic_logging:
      backend: loki
      loki_name: default
      spans: true
      processes: true
      roots: true
    overrides:
      loki_tag: tempo

Example new config:

tempo:
  configs:
  - name: default
    automatic_logging:
      backend: logs_instance
      logs_instance_name: default
      spans: true
      processes: true
      roots: true
    overrides:
      logs_instance_tag: tempo

v0.18

Tempo: Remote write TLS config

Tempo remote_write now supports configuring TLS settings in the trace exporter’s client. insecure_skip_verify is moved into this setting’s block.

Old configurations with insecure_skip_verify outside tls_config will continue to work as of this version, but support will eventually be removed. If both insecure_skip_verify and tls_config.insecure_skip_verify are used, then the latter take precedence.

Example old config:

tempo:
  configs:
    - name: default
      remote_write:
        - endpoint: otel-collector:55680
          insecure: true
          insecure_skip_verify: true

Example new config:

tempo:
  configs:
    - name: default
      remote_write:
        - endpoint: otel-collector:55680
          insecure: true
          tls_config:
            insecure_skip_verify: true

v0.15

Tempo: `automatic_logging` changes

Tempo automatic logging previously assumed that the operator wanted to log to a Loki instance. With the addition of an option to log to stdout a new field is required to maintain the old behavior.

Example old config:

tempo:
  configs:
  - name: default
    automatic_logging:
      loki_name: <some loki instance>

Example new config:

tempo:
  configs:
  - name: default
    automatic_logging:
      backend: loki
      loki_name: <some loki instance>

v0.14

Scraping Service security change

v0.14.0 changes the default behavior of the scraping service config management API to reject all configuration files that read credentials from a file on disk. This prevents malicious users from crafting an instance config file that read arbitrary files on disk and send their contents to remote endpoints.

To revert to the old behavior, add dangerous_allow_reading_files: true in your scraping_service config.

Example old config:

prometheus:
  scraping_service:
    # ...

Example new config:

prometheus:
  scraping_service:
    dangerous_allow_reading_files: true
    # ...

SigV4 config change

v0.14.0 updates the internal Prometheus dependency to 2.26.0, which includes native support for SigV4, but uses a slightly different configuration structure than the Grafana Agent did.

To migrate, remove the enabled key from your sigv4 configs. If enabled was the only key, define sigv4 as an empty object: sigv4: {}.

Example old config:

sigv4:
  enabled: true
  region: us-east-1

Example new config:

sigv4:
  region: us-east-1

Tempo: `push_config` deprecation

push_config is now deprecated in favor of a remote_write array which allows for sending spans to multiple endpoints. push_config will be removed in a future release, and it is recommended to migrate to remote_write as soon as possible.

To migrate, move the batch options outside the push_config block. Then, add a remote_write array and move the remaining of your push_config block inside it.

Example old config:

tempo:
  configs:
    - name: default
      receivers:
        otlp:
          protocols:
            gpc:
      push_config:
        endpoint: otel-collector:55680
        insecure: true
        batch:
          timeout: 5s
          send_batch_size: 100

Example migrated config:

tempo:
  configs:
    - name: default
      receivers:
        otlp:
          protocols:
            gpc:
      remote_write:
        - endpoint: otel-collector:55680
          insecure: true
      batch:
        timeout: 5s
        send_batch_size: 100

v0.12

v0.12 had two breaking changes: the tempo and loki sections have been changed to require a list of tempo/loki configs rather than just one.

Tempo Config Change

The Tempo config (tempo in the config file) has been changed to store configs within a configs list. This allows for defining multiple Tempo instances for collecting traces and forwarding them to different OTLP endpoints.

To migrate, add a configs: array and move your existing config inside of it. Give the element a name: default field.

Each config must have a unique non-empty name. default is recommended for users that don’t have other configs. The name of the config will be added as a tempo_config label for metrics.

Example old config:

tempo:
  receivers:
    jaeger:
      protocols:
        thrift_http:
  attributes:
    actions:
    - action: upsert
      key: env
      value: prod
  push_config:
    endpoint: otel-collector:55680
    insecure: true
    batch:
      timeout: 5s
      send_batch_size: 100

Example migrated config:

tempo:
  configs:
  - name: default
    receivers:
      jaeger:
        protocols:
          thrift_http:
    attributes:
      actions:
      - action: upsert
        key: env
        value: prod
    push_config:
      endpoint: otel-collector:55680
      insecure: true
      batch:
        timeout: 5s
        send_batch_size: 100

Loki Promtail Config Change

The Loki Promtail config (loki in the config file) has been changed to store configs within a configs list. This allows for defining multiple Loki Promtail instances for collecting logs and forwarding them to different Loki servers.

To migrate, add a configs: array and move your existing config inside of it. Give the element a name: default field.

Each config must have a unique non-empty name. default is recommended for users that don’t have other configs. The name of the config will be added as a loki_config label for Loki Promtail metrics.

Example old config:

loki:
  positions:
    filename: /tmp/positions.yaml
  clients:
    - url: http://loki:3100/loki/api/v1/push
  scrape_configs:
  - job_name: system
    static_configs:
      - targets:
        - localhost
        labels:
          job: varlogs
          __path__: /var/log/*log

Example migrated config:

loki:
  configs:
  - name: default
    positions:
      filename: /tmp/positions.yaml
    clients:
      - url: http://loki:3100/loki/api/v1/push
    scrape_configs:
    - job_name: system
      static_configs:
        - targets:
          - localhost
          labels:
            job: varlogs
            __path__: /var/log/*log

Static mode on Grafana Labs

Set up Grafana Agent in static mode

Set up Grafana Agent in static mode

Configure static mode

Configure static mode

Updating configuration

Variable substitution

Regex capture group references

Reloading (beta)

File format

Remote Configuration (Experimental)

Static mode APIs (Stable)

Static mode APIs (Stable)

Config management API (Beta)

API response

List configs

Get config

Update config

Delete config

Agent API

List current running instances of metrics subsystem

List current scrape targets of metrics subsystem

Accept remote_write requests

List current running instances of logs subsystem

List current scrape targets of logs subsystem

Reload configuration file (beta)

Show configuration file

Generate support bundle

Integrations API (Experimental)

Integrations SD API

Integrations autoscrape targets

Ready / health API

Readiness check

Healthiness check

Operation guide

Operation guide

Horizontal Scaling

Host filtering (Beta)

Hashmod sharding (Stable)

Prometheus instances

Instance sharing (Stable)

Release notes

Release notes

v0.37

Breaking change: The default value of retry_on_http_429 is overriden to true for the queue_config in remote_write in metrics config.

Breaking change: Renamed non_indexed_labels Loki processing stage to structured_metadata.

v0.35

Breaking change: Jaeger remote sampling no longer configurable using the Jaeger receiver

Breaking change: auth and version attributes from walk_params block of SNMP integration have been removed

Removal of Dynamic Configuration

Breaking change: Removed and renamed tracing metrics

v0.33

Symbolic links in Docker containers removed

Deprecation of Dynamic Configuration

v0.32

Breaking change: node_exporter configuration options changed

v0.31.1

Breaking change: all Windows executables are now zipped

v0.31

Breaking change: binary names are now prefixed with grafana-

v0.30

Breaking change: ebpf_exporter integration removed

v0.29

Breaking change: JSON-encoded traces from OTLP versions below 0.16.0 are no longer supported

Deprecation: binary names will be prefixed with grafana- in v0.31.0

v0.26

Breaking change: Deprecated YAML fields in server block removed

Breaking change: Reconcile sampling policies between Agent and OTel

v0.24

Breaking change: Integrations renamed when integrations-next feature flag is used

Change: Separating YAML and command line flags

Deprecation on YAML fields in server block that have flags

Breaking change: Removing support for dynamically updating deprecated server fields

Breaking change: Server-specific command line flags have changed

Breaking change: New TLS flags required for enabling TLS

Breaking change: Default HTTP/gRPC address changes

Breaking change: Removal of -reload-addr and -reload-port flags

Change: In-memory autoscrape for integrations-next

v0.22

node_exporter integration deprecated field names

Breaking change: The default value of `retry_on_http_429` is overriden to `true` for the `queue_config` in `remote_write` in `metrics` config.

Breaking change: Renamed `non_indexed_labels` Loki processing stage to `structured_metadata`.

Breaking change: `auth` and `version` attributes from `walk_params` block of SNMP integration have been removed

Breaking change: `node_exporter` configuration options changed

Breaking change: binary names are now prefixed with `grafana-`

Breaking change: `ebpf_exporter` integration removed

Deprecation: binary names will be prefixed with `grafana-` in v0.31.0

Breaking change: Deprecated YAML fields in `server` block removed

Breaking change: Integrations renamed when `integrations-next` feature flag is used

Deprecation on YAML fields in `server` block that have flags

Breaking change: Removal of `-reload-addr` and `-reload-port` flags

`node_exporter` integration deprecated field names

Tempo: `automatic_logging` changes

Tempo: `push_config` deprecation