Custom Resource Definition Reference

Root resource in the deployment.

Metrics resources discovered by Agent.

Logs resources discovered by Agent.

Integrations resources discovered by Agent.

The full list of Secrets referenced by resources in the Deployment.

Spec holds the specification of the desired behavior for the Grafana Agent cluster.

Name from the extract data to parse. If empty, uses the log message.

RE2 regular expression.

If source is provided, the regex attempts to match the source.

If no source is provided, then the regex attempts to attach the log line.

If the provided regex matches the log line or a provided source, the line is dropped.

Value can only be specified when source is specified. If the value provided is an exact match for the given source then the line will be dropped.

Mutually exclusive with expression.

OlderThan will be parsed as a Go duration. If the log line’s timestamp is older than the current time minus the provided duration, it will be dropped.

LongerThan will drop a log line if it its content is longer than this value (in bytes). Can be expressed as an integer (8192) or a number with a suffix (8kb).

Every time a log line is dropped, the metric logentry_dropped_lines_total is incremented. A “reason” label is added, and can be customized by providing a custom value here. Defaults to “drop_stage”.

LogLevel controls the log level of the generated pods. Defaults to “info” if not set.

LogFormat controls the logging format of the generated pods. Defaults to “logfmt” if not set.

APIServerConfig allows specifying a host and auth methods to access the Kubernetes API server. If left empty, the Agent will assume that it is running inside of the cluster and will discover API servers automatically and use the pod’s CA certificate and bearer token file at /var/run/secrets/kubernetes.io/serviceaccount.

PodMetadata configures Labels and Annotations which are propagated to created Grafana Agent pods.

Version of Grafana Agent to be deployed.

Paused prevents actions except for deletion to be performed on the underlying managed objects.

Image, when specified, overrides the image used to run the Agent. It should be specified along with a tag. Version must still be set to ensure the Grafana Agent Operator knows which version of Grafana Agent is being configured.

ImagePullSecrets holds an optional list of references to secrets within the same namespace to use for pulling the Grafana Agent image from registries. More info: https://kubernetes.io/docs/user-guide/images#specifying-imagepullsecrets-on-a-pod

Storage spec to specify how storage will be used.

Volumes allows configuration of additional volumes on the output StatefulSet definition. Volumes specified will be appended to other volumes that are generated as a result of StorageSpec objects.

VolumeMounts allows configuration of additional VolumeMounts on the output StatefulSet definition. VolumEMounts specified will be appended to other VolumeMounts in the Grafana Agent container that are generated as a result of StorageSpec objects.

Resources holds requests and limits for individual pods.

NodeSelector defines which nodes pods should be scheduling on.

ServiceAccountName is the name of the ServiceAccount to use for running Grafana Agent pods.

Secrets is a list of secrets in the same namespace as the GrafanaAgent object which will be mounted into each running Grafana Agent pod. The secrets are mounted into /etc/grafana-agent/extra-secrets/.

ConfigMaps is a liset of config maps in the same namespace as the GrafanaAgent object which will be mounted into each running Grafana Agent pod. The ConfigMaps are mounted into /etc/grafana-agent/extra-configmaps/.

Affinity, if specified, controls pod scheduling constraints.

Tolerations, if specified, controls the pod’s tolerations.

TopologySpreadConstraints, if specified, controls the pod’s topology spread constraints.

SecurityContext holds pod-level security attributes and common container settings. When unspecified, defaults to the default PodSecurityContext.

Containers allows injecting additional containers or modifying operator generated containers. This can be used to allow adding an authentication proxy to a Grafana Agent pod or to change the behavior of an operator-generated container. Containers described here modify an operator generated container if they share the same name and modifications are done via a strategic merge patch. The current container names are: grafana-agent and config-reloader. Overriding containers is entirely outside the scope of what the Grafana Agent team will support and by doing so, you accept that this behavior may break at any time without notice.

InitContainers allows adding initContainers to the pod definition. These can be used to, for example, fetch secrets for injection into the Grafana Agent configuration from external sources. Any errors during the execution of an initContainer will lead to a restart of the pod. More info: https://kubernetes.io/docs/concepts/workloads/pods/init-containers/ Using initContainers for any use case other than secret fetching is entirely outside the scope of what the Grafana Agent maintainers will support and by doing so, you accept that this behavior may break at any time without notice.

PriorityClassName is the priority class assigned to pods.

Port name used for the pods and governing service. This defaults to agent-metrics.

Metrics controls the metrics subsystem of the Agent and settings unique to metrics-specific pods that are deployed.

Logs controls the logging subsystem of the Agent and settings unique to logging-specific pods that are deployed.

Integrations controls the integration subsystem of the Agent and settings unique to integration-specific pods that are deployed.

enableConfigReadAPI enables the read API for viewing currently running config port 8080 on the agent.

disableReporting disable reporting of enabled feature flags to Grafana.

disableSupportBundle disables the generation of support bundles.

Specifies the desired behavior of the Integration.

Name of the integration to run (e.g., “node_exporter”, “mysqld_exporter”).

Type informs Grafana Agent Operator about how to manage the integration being configured.

The configuration for the named integration. Note that Integrations are deployed with the integrations-next feature flag, which has different common settings:

https://grafana.com/docs/agent/latest/configuration/integrations/integrations-next/

An extra list of Volumes to be associated with the Grafana Agent pods running this integration. Volume names are mutated to be unique across all Integrations. Note that the specified volumes should be able to tolerate existing on multiple pods at once when type is daemonset.

Don’t use volumes for loading Secrets or ConfigMaps from the same namespace as the Integration; use the Secrets and ConfigMaps fields instead.

An extra list of VolumeMounts to be associated with the Grafana Agent pods running this integration. VolumeMount names are mutated to be unique across all used IntegrationSpecs.

Mount paths should include the namespace/name of the Integration CR to avoid potentially colliding with other resources.

An extra list of keys from Secrets in the same namespace as the Integration which will be mounted into the Grafana Agent pod running this Integration.

Secrets will be mounted at /etc/grafana-agent/integrations/secrets///.

An extra list of keys from ConfigMaps in the same namespace as the Integration which will be mounted into the Grafana Agent pod running this Integration.

ConfigMaps are mounted at /etc/grafana-agent/integrations/configMaps///.

When true, the configured integration should be run on every Node in the cluster. This is required for Integrations that generate Node-specific metrics like node_exporter, otherwise it must be false to avoid generating duplicate metrics.

Whether this integration can only be defined once for a Grafana Agent process, such as statsd_exporter. It is invalid for a GrafanaAgent to discover multiple unique Integrations with the same Integration name (i.e., a single GrafanaAgent cannot deploy two statsd_exporters).

Label selector to find Integration resources to run. When nil, no integration resources will be defined.

Label selector for namespaces to search when discovering integration resources. If nil, integration resources are only discovered in the namespace of the GrafanaAgent resource.

Set to {} to search all namespaces.

Name from the extracted data to parse as JSON. If empty, uses entire log message.

Set of the key/value pairs of JMESPath expressions. The key will be the key in the extracted data while the expression will be the value, evaluated as a JMESPath from the source data.

Literal JMESPath expressions can be used by wrapping a key in double quotes, which then must be wrapped again in single quotes in YAML so they get passed to the JMESPath parser.

Initial backoff time between retries. Time between retries is increased exponentially.

Maximum backoff time between retries.

Maximum number of retries to perform before giving up a request.

URL is the URL where Loki is listening. Must be a full HTTP URL, including protocol. Required. Example: https://logs-prod-us-central1.grafana.net/loki/api/v1/push.

Tenant ID used by default to push logs to Loki. If omitted assumes remote Loki is running in single-tenant mode or an authentication layer is used to inject an X-Scope-OrgID header.

Maximum amount of time to wait before sending a batch, even if that batch isn’t full.

Maximum batch size (in bytes) of logs to accumulate before sending the batch to Loki.

BasicAuth for the Loki server.

BearerToken used for remote_write.

BearerTokenFile used to read bearer token.

ProxyURL to proxy requests through. Optional.

TLSConfig to use for the client. Only used when the protocol of the URL is https.

Configures how to retry requests to Loki when a request fails. Defaults to a minPeriod of 500ms, maxPeriod of 5m, and maxRetries of 10.

ExternalLabels are labels to add to any time series when sending data to Loki.

Maximum time to wait for a server to respond to a request.

Spec holds the specification of the desired behavior for the logs instance.

Clients controls where logs are written to for this instance.

Determines which PodLogs should be selected for including in this instance.

Set of labels to determine which namespaces should be watched for PodLogs. If not provided, checks only namespace of the instance.

AdditionalScrapeConfigs allows specifying a key of a Secret containing additional Grafana Agent logging scrape configurations. Scrape configurations specified are appended to the configurations generated by the Grafana Agent Operator.

Job configurations specified must have the form as specified in the official Promtail documentation:

https://grafana.com/docs/loki/latest/clients/promtail/configuration/#scrape_configs

As scrape configs are appended, the user is responsible to make sure it is valid. Note that using this feature may expose the possibility to break upgrades of Grafana Agent. It is advised to review both Grafana Agent and Promtail release notes to ensure that no incompatible scrape configs are going to break Grafana Agent after the upgrade.

Configures how tailed targets are watched.

A global set of clients to use when a discovered LogsInstance does not have any clients defined.

LogsExternalLabelName is the name of the external label used to denote Grafana Agent cluster. Defaults to “cluster.” External label will not be added when value is set to the empty string.

InstanceSelector determines which LogInstances should be selected for running. Each instance runs its own set of Prometheus components, including service discovery, scraping, and remote_write.

InstanceNamespaceSelector are the set of labels to determine which namespaces to watch for LogInstances. If not provided, only checks own namespace.

IgnoreNamespaceSelectors, if true, will ignore NamespaceSelector settings from the PodLogs configs, and they will only discover endpoints within their current namespace.

EnforcedNamespaceLabel enforces adding a namespace label of origin for each metric that is user-created. The label value will always be the namespace of the object that is being created.

Period to resync directories being watched and files being tailed to discover new ones or stop watching removed ones.

LogQL stream selector and filter expressions. Required.

Names the pipeline. When defined, creates an additional label in the pipeline_duration_seconds histogram, where the value is concatenated with job_name using an underscore.

Determines what action is taken when the selector matches the log line. Can be keep or drop. Defaults to keep. When set to drop, entries are dropped and no later metrics are recorded. Stages must be empty when dropping metrics.

Every time a log line is dropped, the metric logentry_dropped_lines_total is incremented. A “reason” label is added, and can be customized by providing a custom value here. Defaults to “match_stage.”

Nested set of pipeline stages to execute when action is keep and the log line matches selector.

An example value for stages may be:

stages: | - json: {} - labelAllow: [foo, bar]

Note that stages is a string because SIG API Machinery does not support recursive types, and so it cannot be validated for correctness. Be careful not to mistype anything.

Send enables metric metadata to be sent to remote storage.

SendInterval controls how frequently metric metadata is sent to remote storage.

Spec holds the specification of the desired behavior for the Metrics instance.

WALTruncateFrequency specifies how frequently the WAL truncation process should run. Higher values causes the WAL to increase and for old series to stay in the WAL for longer, but reduces the chances of data loss when remote_write is failing for longer than the given frequency.

MinWALTime is the minimum amount of time series and samples may exist in the WAL before being considered for deletion.

MaxWALTime is the maximum amount of time series and asmples may exist in the WAL before being forcibly deleted.

RemoteFlushDeadline is the deadline for flushing data when an instance shuts down.

WriteStaleOnShutdown writes staleness markers on shutdown for all series.

ServiceMonitorSelector determines which ServiceMonitors should be selected for target discovery.

ServiceMonitorNamespaceSelector are the set of labels to determine which namespaces to watch for ServiceMonitor discovery. If nil, only checks own namespace.

PodMonitorSelector determines which PodMonitors should be selected for target discovery. Experimental.

PodMonitorNamespaceSelector are the set of labels to determine which namespaces to watch for PodMonitor discovery. If nil, only checks own namespace.

ProbeSelector determines which Probes should be selected for target discovery.

ProbeNamespaceSelector are the set of labels to determine which namespaces to watch for Probe discovery. If nil, only checks own namespace.

RemoteWrite controls remote_write settings for this instance.

AdditionalScrapeConfigs allows specifying a key of a Secret containing additional Grafana Agent Prometheus scrape configurations. SCrape configurations specified are appended to the configurations generated by the Grafana Agent Operator. Job configurations specified must have the form as specified in the official Prometheus documentation: https://prometheus.io/docs/prometheus/latest/configuration/configuration/#scrape_config. As scrape configs are appended, the user is responsible to make sure it is valid. Note that using this feature may expose the possibility to break upgrades of Grafana Agent. It is advised to review both Grafana Agent and Prometheus release notes to ensure that no incompatible scrape configs are going to break Grafana Agent after the upgrade.

The metric type to create. Must be one of counter, gauge, histogram. Required.

Sets the description for the created metric.

Sets the custom prefix name for the metric. Defaults to “promtailcustom”.

Key from the extracted data map to use for the metric. Defaults to the metrics name if not present.

Label values on metrics are dynamic which can cause exported metrics to go stale. To prevent unbounded cardinality, any metrics not updated within MaxIdleDuration are removed.

Must be greater or equal to 1s. Defaults to 5m.

If true, all log lines are counted without attempting to match the source to the extracted map. Mutually exclusive with value.

Only valid for type: counter.

If true all log line bytes are counted. Can only be set with matchAll: true and action: add.

Only valid for type: counter.

Filters down source data and only changes the metric if the targeted value matches the provided string exactly. If not present, all data matches.

The action to take against the metric. Required.

Must be either “inc” or “add” for type: counter or type: histogram. When type: gauge, must be one of “set”, “inc”, “dec”, “add”, or “sub”.

“add”, “set”, or “sub” requires the extracted value to be convertible to a positive float.

Buckets to create. Bucket values must be convertible to float64s. Extremely large or small numbers are subject to some loss of precision. Only valid for type: histogram.

RemoteWrite controls default remote_write settings for all instances. If an instance does not provide its own remoteWrite settings, these will be used instead.

Replicas of each shard to deploy for metrics pods. Number of replicas multiplied by the number of shards is the total number of pods created.

Shards to distribute targets onto. Number of replicas multiplied by the number of shards is the total number of pods created. Note that scaling down shards will not reshard data onto remaining instances, it must be manually moved. Increasing shards will not reshard data either but it will continue to be available from the same instances. Sharding is performed on the content of the address target meta-label.

ReplicaExternalLabelName is the name of the metrics external label used to denote replica name. Defaults to replica. External label will not be added when value is set to the empty string.

MetricsExternalLabelName is the name of the external label used to denote Grafana Agent cluster. Defaults to “cluster.” External label will not be added when value is set to the empty string.

ScrapeInterval is the time between consecutive scrapes.

ScrapeTimeout is the time to wait for a target to respond before marking a scrape as failed.

ExternalLabels are labels to add to any time series when sending data over remote_write.

ArbitraryFSAccessThroughSMs configures whether configuration based on a ServiceMonitor can access arbitrary files on the file system of the Grafana Agent container e.g. bearer token files.

OverrideHonorLabels, if true, overrides all configured honor_labels read from ServiceMonitor or PodMonitor to false.

OverrideHonorTimestamps allows to globally enforce honoring timestamps in all scrape configs.

IgnoreNamespaceSelectors, if true, will ignore NamespaceSelector settings from the PodMonitor and ServiceMonitor configs, and they will only discover endpoints within their current namespace.

EnforcedNamespaceLabel enforces adding a namespace label of origin for each metric that is user-created. The label value will always be the namespace of the object that is being created.

EnforcedSampleLimit defines global limit on the number of scraped samples that will be accepted. This overrides any SampleLimit set per ServiceMonitor and/or PodMonitor. It is meant to be used by admins to enforce the SampleLimit to keep the overall number of samples and series under the desired limit. Note that if a SampleLimit from a ServiceMonitor or PodMonitor is lower, that value will be used instead.

EnforcedTargetLimit defines a global limit on the number of scraped targets. This overrides any TargetLimit set per ServiceMonitor and/or PodMonitor. It is meant to be used by admins to enforce the TargetLimit to keep the overall number of targets under the desired limit. Note that if a TargetLimit from a ServiceMonitor or PodMonitor is higher, that value will be used instead.

InstanceSelector determines which MetricsInstances should be selected for running. Each instance runs its own set of Metrics components, including service discovery, scraping, and remote_write.

InstanceNamespaceSelector are the set of labels to determine which namespaces to watch for MetricsInstances. If not provided, only checks own namespace.

RE2 regular expression. Creates a new multiline block when matched. Required.

Maximum time to wait before passing on the multiline block to the next stage if no new lines are received. Defaults to 3s.

Maximum number of lines a block can have. A new block is started if the number of lines surpasses this value. Defaults to 128.

Name from extract data to use for the log entry. Required.

Name from extracted data or line labels. Required. Labels provided here are automatically removed from output labels.

If the resulting log line should use any existing timestamp or use time.Now() when the line was created. Set to true when combining several log streams from different containers to avoid out of order errors.

CRI is a parsing stage that reads log lines using the standard CRI logging format. Supply cri: {} to enable.

Docker is a parsing stage that reads log lines using the standard Docker logging format. Supply docker: {} to enable.

Drop is a filtering stage that lets you drop certain logs.

JSON is a parsing stage that reads the log line as JSON and accepts JMESPath expressions to extract data.

Information on JMESPath: http://jmespath.org/

LabelAllow is an action stage that only allows the provided labels to be included in the label set that is sent to Loki with the log entry.

LabelDrop is an action stage that drops labels from the label set that is sent to Loki with the log entry.

Labels is an action stage that takes data from the extracted map and modifies the label set that is sent to Loki with the log entry.

The key is REQUIRED and represents the name for the label that will be created. Value is optional and will be the name from extracted data to use for the value of the label. If the value is not provided, it defaults to match the key.

Match is a filtering stage that conditionally applies a set of stages or drop entries when a log entry matches a configurable LogQL stream selector and filter expressions.

Metrics is an action stage that supports defining and updating metrics based on data from the extracted map. Created metrics are not pushed to Loki or Prometheus and are instead exposed via the /metrics endpoint of the Grafana Agent pod. The Grafana Agent Operator should be configured with a MetricsInstance that discovers the logging DaemonSet to collect metrics created by this stage.

Multiline stage merges multiple lines into a multiline block before passing it on to the next stage in the pipeline.

Output stage is an action stage that takes data from the extracted map and changes the log line that will be sent to Loki.

Pack is a transform stage that lets you embed extracted values and labels into the log line by packing the log line and labels inside of a JSON object.

Regex is a parsing stage that parses a log line using a regular expression. Named capture groups in the regex allows for adding data into the extracted map.

Replace is a parsing stage that parses a log line using a regular expression and replaces the log line. Named capture groups in the regex allows for adding data into the extracted map.

Template is a transform stage that manipulates the values in the extracted map using Go’s template syntax.

Tenant is an action stage that sets the tenant ID for the log entry picking it from a field in the extracted data map. If the field is missing, the default LogsClientSpec.tenantId will be used.

Timestamp is an action stage that can change the timestamp of a log line before it is sent to Loki. If not present, the timestamp of a log line defaults to the time when the log line was read.

Spec holds the specification of the desired behavior for the PodLogs.

The label to use to retrieve the job name from.

PodTargetLabels transfers labels on the Kubernetes Pod onto the target.

Selector to select Pod objects. Required.

Selector to select which namespaces the Pod objects are discovered from.

Pipeline stages for this pod. Pipeline stages support transforming and filtering log lines.

RelabelConfigs to apply to logs before delivering. Grafana Agent Operator automatically adds relabelings for a few standard Kubernetes fields and replaces original scrape job name with __tmp_logs_job_name.

More info: https://grafana.com/docs/loki/latest/clients/promtail/configuration/#relabel_configs

Capacity is the number of samples to buffer per shard before we start dropping them.

MinShards is the minimum number of shards, i.e. amount of concurrency.

MaxShards is the maximum number of shards, i.e. amount of concurrency.

MaxSamplesPerSend is the maximum number of samples per send.

BatchSendDeadline is the maximum time a sample will wait in buffer.

MaxRetries is the maximum number of times to retry a batch on recoverable errors.

MinBackoff is the initial retry delay. Gets doubled for every retry.

MaxBackoff is the maximum retry delay.

RetryOnRateLimit retries requests when encountering rate limits.

Name from extracted data to parse. If empty, defaults to using the log message.

RE2 regular expression. Each capture group MUST be named. Required.

Name of the remote_write queue. Must be unique if specified. The name is used in metrics and logging in order to differentiate queues.

URL of the endpoint to send samples to.

RemoteTimeout is the timeout for requests to the remote_write endpoint.

Headers is a set of custom HTTP headers to be sent along with each remote_write request. Be aware that any headers set by Grafana Agent itself can’t be overwritten.

WriteRelabelConfigs holds relabel_configs to relabel samples before they are sent to the remote_write endpoint.

BasicAuth for the URL.

BearerToken used for remote_write.

BearerTokenFile used to read bearer token.

SigV4 configures SigV4-based authentication to the remote_write endpoint. Will be used if SigV4 is defined, even with an empty object.

TLSConfig to use for remote_write.

ProxyURL to proxy requests through. Optional.

QueueConfig allows tuning of the remote_write queue parameters.

MetadataConfig configures the sending of series metadata to remote storage.

Name from extracted data to parse. If empty, defaults to using the log message.

RE2 regular expression. Each capture group MUST be named. Required.

Value to replace the captured group with.

Region of the AWS endpoint. If blank, the region from the default credentials chain is used.

AccessKey holds the secret of the AWS API access key to use for signing. If not provided, The environment variable AWS_ACCESS_KEY_ID is used.

SecretKey of the AWS API to use for signing. If blank, the environment variable AWS_SECRET_ACCESS_KEY is used.

Profile is the named AWS profile to use for authentication.

RoleARN is the AWS Role ARN to use for authentication, as an alternative for using the AWS API keys.

Name from extracted data to parse. Required. If empty, defaults to using the log message.

Go template string to use. Required. In addition to normal template functions, ToLower, ToUpper, Replace, Trim, TrimLeft, TrimRight, TrimPrefix, and TrimSpace are also available.

Name from labels whose value should be set as tenant ID. Mutually exclusive with source and value.

Name from extracted data to use as the tenant ID. Mutually exclusive with label and value.

Value to use for the template ID. Useful when this stage is used within a conditional pipeline such as match. Mutually exclusive with label and source.

Name from extracted data to use as the timestamp. Required.

Determines format of the time string. Required. Can be one of: ANSIC, UnixDate, RubyDate, RFC822, RFC822Z, RFC850, RFC1123, RFC1123Z, RFC3339, RFC3339Nano, Unix, UnixMs, UnixUs, UnixNs.

Fallback formats to try if format fails.

IANA Timezone Database string.

Action to take when the timestamp can’t be extracted or parsed. Can be skip or fudge. Defaults to fudge.