Grafana Cloud

Set up with Instrumentation Hub

Note

Instrumentation Hub is in public preview.

Instrumentation Hub is an initial setup and configuration tool within Grafana Cloud. With Instrumentation Hub, you can:

  • Have all your application and infrastructure services discovered
  • View what is discovered
  • Control the data you want to gather and what you want to instrument
  • View the status of the instrumentation process

Note

Currently, Instrumentation Hub supports Kubernetes Linux environments.

Access Instrumentation Hub

To access the Instrumentation Hub:

  1. On the main menu, expand Connections.
  2. Click Instrumentation Hub.

The next steps for setup and configuration are to:

  • Install the Grafana Alloy collector with the Helm chart, which installs Beyla to discover your Kubernetes clusters and services, and instruments your code.
  • Choose what services you want to instrument.
  • Activate application monitoring or Kubernetes Monitoring.
  • Build the Knowledge Graph, which creates a correlated view of your data.

Install Grafana Alloy

Click Install Alloy to open the Collector Setup page. This is the same wizard used to configure Kubernetes Monitoring with the Helm chart. For more information about the Helm chart, refer to Configuration steps for Kubernetes Monitoring with Helm chart.

  1. Distribution: Select your Kubernetes platform. Instrumentation Hub currently supports Kubernetes environments only.

  2. Access token: Create a new access policy token (recommended), paste an existing token, or use a Kubernetes Secret that already exists in your Cluster. If you use an existing token, it must grant the ability to send metrics, logs, and traces to Grafana Cloud.

    If you create a new token, save it, because it isn’t shown again. For more information, refer to Grafana Cloud access policies.

  3. Choose setup: Select a preconfigured Alloy deployment based on the level of observability you need. Expand Learn what this deployment includes on any option to compare the details.

  4. Monitoring: Select the remote monitoring features and managed pipelines you want. The settings of the two selections are optional and independent, so you can modify one, both, or neither, and you can change them later in Instrumentation Hub without a Helm upgrade.

    Both selections use recommended defaults that collect the signals Instrumentation Hub needs, so you can usually continue without changing anything. If you want to limit what Alloy collects, for example to control costs or to exclude telemetry you don’t use, click the menu icon in the corner of a card to open its advanced options.

    • Advanced settings: Use this setting to skip the set up of either of the monitoring selections, but continue with instrumentation. The two options are:
      • Recommended: The default, and recommended for auto-instrumentation.
      • Not recommended: To skip setup. If you skip Kubernetes Monitoring, you need to create your own dashboards to monitor this Cluster.
    • Advanced metrics tuning: Switch categories of Kubernetes metrics on or off, such as cost metrics, energy metrics, or Prometheus Operator objects. These settings are configured locally on your Cluster and aren’t managed from Grafana Cloud.
    • Advanced logs and events tuning: Switch collection of Cluster events, Pod logs, Node logs, and Kubernetes manifests on or off. For example, manifest collection powers the View Manifest and Changes tabs on workloads and Pods.

    Any signal you switch off isn’t collected, so dashboards and views that rely on it won’t have data.

    Note

    If you don’t select Auto-discover services on my cluster, Instrumentation Hub can’t discover services in your Cluster and one-click instrumentation isn’t available.

    The Monitoring step, showing the Enable Kubernetes monitoring and Auto-discover services on my cluster selections
    Monitoring step with Kubernetes monitoring and service discovery options
  5. Deployment: Deploys Alloy to your new Cluster.

    1. Enter the name of your Cluster.
    2. Enter the namespace where Alloy will be deployed.
    3. Custom labels (optional). Add labels, also known as attributes, for Fleet Management. For more information, refer to Fleet Management.
    4. Copy the generated deployment code and run it in your terminal. Make sure your kubectl context points to the Cluster you want to monitor.
    5. Click Test connection to ensure the collector is running.

    If the connection test returns an error, verify the output of the Helm command. If it completed successfully, verify that Alloy is listed in the Fleet Management interface. If it’s not listed, verify it was installed in the target cluster’s monitoring namespace by running kubectl --namespace monitoring get daemonset.

    The Deployment step, showing the Cluster name, Namespace, and Custom labels fields
    Deployment step with Cluster name, namespace, and custom labels

Begin instrumentation

On the main page, a message indicates what has been discovered. Choose to begin instrumentation for applications and services or Kubernetes infrastructure.

The Instrumentation Hub landing page, showing the Application instrumentation and Kubernetes instrumentation tiles with the number of discovered and uninstrumented items in each
Instrumentation Hub, with tiles for instrumenting applications and Kubernetes infrastructure

Instrument applications and services

Complete the following steps to instrument application services.

  1. In the Instrumentation Hub, click View & instrument services in the Application instrumentation tile.

  2. Select a Cluster from the list on the left.

    The Cluster shows its namespaces, with the total number of services in each and how many of them are instrumented.

  3. To choose what to instrument, complete any of the following:

    • Instrument every service in a Cluster: Click Select all. You can repeat this for another Cluster if you want to instrument multiple Clusters.
    • Instrument an entire namespace: Select the checkbox next to the namespace.
    • Instrument an individual service: Click the name of the namespace, then select the checkbox next to the service.
    The Application instrumentation page with every uninstrumented service in a Cluster selected, showing the counts of detected, uninstrumented, and selected services
    All uninstrumented services in a Cluster selected for instrumentation

    To find a specific item, search for a service, namespace, or technology type.

    To remove your instrumentation choice, click Clear selection.

  4. Click Instrument services.

    Instrument services is only available when your selection includes at least one service that is not instrumented.

    Note

    If you have not activated application services, you are interrupted after you click Instrument services. You cannot continue instrumentation until you activate the product. To continue, click Activate to activate Application services.

  5. Confirm your selection on the Review Selection page, then click Continue.

  6. View the settings on the Pipeline Configuration page.

    These settings apply to the whole namespace. They affect every instrumented service in the selected namespaces, not only the services you add in this run. Any service that’s already instrumented in those namespaces is updated to match.

    The following are included as part of instrumentation:

    • R.E.D metrics: Turns traces into metrics like request rate, error rate, and latency
    • Service to service links: Visualizes how your services interact, with CPU and memory usage at each node
    The Pipeline Configuration page, showing R.E.D metrics and Service to service links as included, and the six optional settings below them
    Pipeline Configuration page with the included and optional settings

    Switch on any of these optional selections:

    • Ongoing instrumentation: To instrument new services automatically as they’re discovered in the selected namespaces
    • Traces: To capture distributed traces that track request flows across services
    • Application logs: To capture application logs for centralized searching, analysis, and correlation with traces and metrics
    • Profiling: To capture CPU profiles that identify performance problems.
    • Process metrics: To capture metrics about processes that are running the instrumented application
    • Extended application metrics: To capture additional application metrics using OpenTelemetry semantic conventions for custom visualizations
  7. Click Instrument now.

    Instrumentation can take a few minutes, depending on the amount of items you have chosen to instrument.

Instrument Kubernetes Clusters

Complete the following steps to instrument Kubernetes Clusters.

  1. Click View & instrument clusters in the Kubernetes instrumentation tile.

  2. To choose which Clusters to instrument, complete either of the following:

    • Select the checkboxes next to the Clusters you want to instrument.
    • Click a Cluster in the list, then click Select for instrumentation.
    The Kubernetes instrumentation page with an uninstrumented Cluster selected, listing its namespaces with a status of Not Instrumented
    A Cluster selected for instrumentation

    To remove your instrumentation choice, click Clear selection.

  3. Click Instrument selected.

    Instrument selected is only available when your selection includes at least one Cluster that is not instrumented.

    Note

    If you have not activated Kubernetes Monitoring, you are interrupted after you click Instrument selected or Select for instrumentation. You cannot continue instrumentation until you activate the product. To continue, click Activate to activate Kubernetes Monitoring.

  4. Confirm your selection on the Review Selection page, then click Continue.

  5. View the settings on the Advanced Options page.

    These settings apply to every Cluster you instrument. You can change them later for individual Clusters.

    Cluster-wide metrics are included as part of instrumentation, and collect performance and resource metrics for the entire Cluster.

    Switch on any of these optional selections:

    • Cluster events: To capture Cluster events for debugging and auditing
    • Node logs: To capture journald logs from the Nodes
  6. Click Instrument now.

    Instrumentation can take a few minutes, depending on the amount of items you have chosen to instrument.

Manage instrumentation and settings

From the Application instrumentation and Kubernetes instrumentation pages, you can:

  • View and change the settings you chose for any Cluster or namespace
  • Remove instrumentation
  • Go directly to a Kubernetes namespace or application service by clicking its name

Change application namespace settings

Pipeline settings apply to one namespace and affect all the instrumented services within it.

  1. On the Application instrumentation page, select the Cluster that contains the namespace.

  2. To view the settings for a namespace, complete either of the following:

    • Click the name of the namespace to show a list of services within it, then click Namespace settings.
    • In the row of the namespace, click the gear icon. This shows the settings of the namespace without showing the list of services.

    Settings are only available for a namespace with at least one instrumented service.

  3. To make changes, switch on or off any optional settings.

  4. Click Apply.

    To leave without saving, click Back. If you have unsaved changes, confirm that you want to discard them.

Remove instrumentation from application services

You remove instrumentation from the same view you use to instrument.

  1. To choose what to remove from instrumentation, complete any of the following:

    • To remove every service in a Cluster, select the Cluster, then click Select all.
    • To remove an entire namespace, select the Cluster, then select the checkbox next to the namespace.
    • To remove an individual service, click the Cluster, click the namespace, then select the checkbox next to the service.
    The Application instrumentation page with instrumented services selected and Remove Instrumentation available
    Services selected for instrumentation removal

    To remove your choice, click Clear selection.

  2. Click Remove Instrumentation.

    Remove Instrumentation is only available when your selection includes at least one instrumented service.

  3. Review your selection at the confirmation screen, then click Uninstrument.

    Caution

    If you uninstrument every service in a namespace, the pipeline is deleted and no future services are auto-instrumented until you instrument that namespace again with at least one service.

Change Kubernetes Cluster settings

Cluster settings apply to one Cluster and affect data collection for all its namespaces.

  1. On the Kubernetes instrumentation page, select the Cluster from the list.

  2. Click Cluster settings.

    Cluster settings is only available while the Cluster has instrumented namespaces, and is unavailable while uninstrumentation is in progress.

  3. To make changes, switch on or off any optional settings.

  4. Click Apply.

    To leave without saving, click Cancel. If you have unsaved changes, confirm that you want to discard them.

Remove instrumentation from Kubernetes

You remove instrumentation from the same view you use to instrument.

  1. Select the Cluster you want to remove from instrumentation.

    Select the checkbox next to the Cluster, or click the Cluster in the list and then click Select for uninstrumentation.

    To remove your choice, click Clear selection.

    The Kubernetes instrumentation page with an instrumented Cluster selected and Remove instrumentation available
    Cluster selected for instrumentation removal
  2. Click Remove instrumentation.

    Remove instrumentation is only available when your selection includes at least one instrumented Cluster.

  3. Review your selection at the confirmation screen, then click Uninstrument.

Activate an observability product

To use a Grafana Cloud observability product for Kubernetes or application monitoring, you must first activate it. Activation allows your use of the product and begins host-hours billing. Even with an activated product, you are not billed until metrics are sent to Grafana Cloud.

To learn more about billing, refer to:

To deactivate, refer to:

Build the Grafana Knowledge Graph

You can build a correlated view of your data with the Knowledge Graph so you don’t need to document your inventory and its entity connections, or guess the health and performance of entities in your system.

Knowledge graph is an underlying technology that:

  • Provides an inventory of all system entities and their connections
  • Rates the performance and health of connections between services
  • Allows you to navigate between infrastructure and application services without losing context