Introduction to Adaptive Logs
Documentationbreadcrumb arrow Grafana Cloudbreadcrumb arrow Cost management and billingbreadcrumb arrow Adaptive Telemetrybreadcrumb arrow Adaptive Logsbreadcrumb arrow Introduction
Grafana Cloud RSS

Introduction to Adaptive Logs

Learn more about the fundamentals and available features that help you optimize your use of Adaptive Logs.

Adaptive Logs is a cost optimization feature in Grafana Cloud that lets you identify and reduce unneeded log volume. Most companies generate more logs than they need. By not storing logs that are not valuable to you, you can significantly reduce the number of logs you need to look through to find what you are looking for.

Adaptive Logs makes recommendations, which identify a percentage of logs that can be dropped based on query usage within your Grafana Cloud Logs environment.

The following diagram gives you an overview of Adaptive Logs and introduces you to some of the fundamental features that are the principles of how Adaptive Logs works.

An image showing how Adaptive Logs works.

How it works at a glance

  1. Adaptive Logs analyzes your logs

    It looks at your incoming logs and groups log lines into patterns.

  2. Adaptive Logs checks how you use your logs

    It checks which log patterns you actually query over the past 15 days.

  3. Adaptive Logs suggests which logs to drop

    Based on what it finds, Adaptive Logs generates recommendations suggesting a drop rate, or percentage of those logs that can safely be dropped without impacting your observability needs.

  4. You review, apply, or adjust the recommendations.

    Using the Adaptive Logs user interface, you review the recommendations and suggested drop rates. You can also choose to keep certain logs that Adaptive Logs suggests dropping.

  5. After reviewing the suggestions, you tell Adaptive Logs to start dropping those logs.

  6. Grafana Cloud drops the unneeded logs

    Grafana Cloud automatically gets rid of the unneeded logs, filtering them out and discarding them before they are indexed and stored, reducing your log ingest and associated costs.

Example

Each of the lines in the chart below represents a hypothetical pattern generated by Adaptive Logs.

An image showing an example of how Adaptive Logs works.

In the middle example, you can see the logs are queried frequently, so the recommendation is to keep them all. For patterns that are queried infrequently or not at all, Adaptive Logs recommends dropping a certain percentage of those logs.

Fundamentals

The following concepts are key to your understanding of how Adaptive Logs works.

Patterns

A pattern is a representation of log lines that includes both fixed and variable content. Patterns are grouped by the parts of the logs that do not change, that is, the fixed content.

Here is an example of a pattern.

console 
level=debug ts=<TIMESTAMP> caller=metrics.go:<NUM> traceID=<HEX> duration=<DURATION> <*>

This pattern can be broken down as follows.

  • level=debug: This is a constant string appearing in all log lines matching this pattern.

  • ts=<TIMESTAMP>: The ’ts=’ string is fixed, and the <TIMESTAMP> string is variable, but always a TIMESTAMP data type. Everything inside an angular bracket represents variable content. Possible types are:

    • <BYTESIZE>
    • <DURATION>
    • <HEX>
    • <IP>
    • <NUM>
    • <TIMESTAMP>
    • <UUID>

  • caller=metrics.go:<NUM>: This is a mix of fixed and variable content.

  • <*>: This represents variable alphanumeric string content.

Drop rate

A drop rate is the percentage of logs that you choose not to ingest into Grafana Cloud after they have been sent.

An 80% drop rate means roughly 8 out of 10 log lines are not ingested. This helps reduce ingest costs and clutter by eliminating log entries deemed less valuable or unnecessary for monitoring and troubleshooting and also creates savings in your Cloud Logs spending.

When drop rates are applied, dropping occurs on a go-forward basis - Adaptive Logs does not drop logs that have already been ingested. Each log pattern has its own recommended drop rate based on its ingest volume and query behavior. You can manage each recommendation separately on the Adaptive Logs screen.

Recommendations

Adaptive Logs generates recommendations suggesting a drop rate, or percentage of logs, that can safely be dropped without impacting your observability needs.

Adaptive Logs must wait some time before it can collect enough query data to analyze and make recommendations. The default time period is 7 days. After Adaptive Logs has collected enough data to analyze, it evaluates up to the previous 15 days of query activity to make recommendations.

After Adaptive Logs starts making recommendations, they are updated daily, and reflect any changes in your ingest and query behavior.

A recommendation includes the following information.

  • Query Frequency - how often patterns are queried. Query frequency reflects the ratio of the number log lines returned in query results to the number of log lines ingested which match a particular pattern. For example, if there are 100 ingested lines matching a pattern, and each of these lines has been queried five times, the query to ingest ratio would be 500:100, or 500%.
    • Never: Query:ingest ratio of 0%
    • Rarely: Between 0% and 1%
    • Sometimes: Between 1% and 40%
    • Often: Between 40% and 100%
    • Always: Greater than 100%
  • Volume - the volume of logs ingested for this pattern over the previous 15 days.
  • Current drop rate - the drop rate currently being applied to the recommendation. By default, this rate is 0.
  • Recommended drop rate - the drop rate the recommendation engine suggests applying to your logs.
  • Projected savings - projected savings over the following 15 day period with the recommended drop rate. This value is extrapolated from analyzing the previous 15 days if the Adaptive Logs recommendations been applied.
  • Pattern - the pattern being used to power the recommendation.

Adaptive Logs evaluates queries on a given data source regardless of where they originate. It ignores queries that are too broad or not typically used for observability needs. Recommendations for logs that can be dropped include logs that are not included in current query patterns. As query patterns change over time, recommendations are updated.

Exemptions

Exemptions allow you to exclude specific logs from being dropped, even if they are rarely queried. This gives you full control to ensure that, even as you optimize costs, your most important data is always available when you need it.

Using exemptions, you override the automated recommendations using stream-based exemptions, excluding certain logs from being dropped.

As an example, suppose you do not want to drop logs related to your login service. To prevent logs from your login service from being dropped, exclude the {service="login"} selector.

For more information on how to construct a stream selector statement, refer to the log stream selector documentation.