Documentationbreadcrumb arrow Grafana Cloudbreadcrumb arrow Connect to data sourcesbreadcrumb arrow Private data source connect (PDC)breadcrumb arrow PDC scalability and security
Grafana Cloud
Last reviewed: April 2, 2026

PDC scalability and security

The PDC agent is designed to scale with your infrastructure and protect data in transit. On the scalability side, it supports parallel SSH connections within a single agent and horizontal scaling across multiple agent replicas, with automatic load balancing and failover handled by the PDC service in Grafana Cloud. On the security side, all traffic is encrypted through SSH tunnels with short-lived certificate-based authentication, access can be restricted to specific hosts and ports using PermitRemoteOpen, and cloud provider private networking options keep traffic off the public internet entirely.

Scalability

The PDC agent scales horizontally. To increase throughput, you can open parallel SSH connections within a single agent or deploy additional agent replicas. The PDC service in Grafana Cloud load balances connections across all available agents and scales horizontally to ensure high availability.

You do not need to set up a separate agent for each data source. A single agent connects a hosted Grafana instance to multiple data sources as long as they are in the same network. SSH allows multiplexing multiple SOCKS connections over a single tunnel.

Resource requirements

Memory and CPU requirements for the PDC agent vary based on request volume and data source query response sizes. As a baseline, allocate 512Mi of memory and 500m CPU, then adjust based on observed usage. You can monitor agent resource consumption using the PDC agent metrics, particularly CPU utilization and open SSH channel counts.

The PDC agent is limited to single-core CPU usage due to OpenSSH’s single-threaded architecture. CPU allocation doesn’t exceed 1 core.

Scale with parallel connections

Each PDC agent opens a single SSH connection by default. Use the -connections flag to open multiple parallel SSH connections within a single agent, increasing the total bandwidth to your network without deploying additional agents.

The total number of connections across all PDC agents for a single PDC network is limited to 50. Contact Grafana Support to request an increase.

Scale with additional agents

Deploy additional PDC agents with the same configuration to distribute load. The PDC service load balances requests across all connected agents automatically.

Note

For high availability, Grafana recommends running a minimum of 3 PDC agents in production environments. These can be deployed to different regions, data centers, or providers as long as they can reach the same data sources and are connected to the same PDC network in Grafana Cloud.

PDC supports seamless failover. For example, if you deploy three agents in each of two data centers, PDC load balances across all six replicas. If connectivity is lost with one data center, the PDC service automatically fails over to agents in the other.

When to add more agents

CPU usage is a core performance indicator for PDC agents. Because OpenSSH is single-threaded, performance degrades as that core becomes saturated. Add more agents before your existing agents reach this threshold. If your PDC agents run in Kubernetes, consider using Horizontal Pod Autoscaling to scale agents based on CPU usage.

Security

SSH tunnels are widely accepted by the security community as secure. You control the SSH tunnel initiated by the PDC agent and can stop it at any time if there is any suspicion of compromise.

The Grafana PDC agent is open source, and the source code is available at the pdc-agent repository. The agent is a small wrapper around OpenSSH that ensures the correct SSH flags are set.

Grafana Labs conducted an external penetration test and found no vulnerabilities. The full penetration test report is available at the Grafana Labs Trust Center.

Encryption

Traffic between the PDC agent in your private network and the PDC service in Grafana Cloud is encrypted through the SSH tunnel. If the private data source itself supports encryption (for example, HTTPS), traffic is encrypted end-to-end.

Grafana Cloud also uses mutual TLS (mTLS) between the hosted Grafana instance and the PDC proxy. These certificates are managed internally and require no user configuration.

Restrict access with PermitRemoteOpen

You can restrict which destinations Grafana Cloud can reach through the tunnel by configuring the PermitRemoteOpen SSH option on the PDC agent. This limits traffic to specific hosts and ports that you explicitly allow.

For example, you can configure the PDC agent to only allow traffic to two endpoints by passing the following command line argument:

-ssh-flag='-o PermitRemoteOpen=mysql.example.com:3306 prometheus.example.com:443'

For more information about the -ssh-flag option, refer to the PDC agent CLI reference.

SSH key pairs and certificate rotation

The PDC agent generates an SSH key pair on first run and stores it at ~/.ssh/grafana_pdc by default. On subsequent startups, the agent reuses the existing key pair. You can provide your own custom SSH key pair using the -ssh-key-file flag.

At startup, the PDC agent uploads its public key to Grafana Cloud, which returns a short-lived (1 hour) SSH certificate for authentication. The agent automatically renews the certificate before it expires. You can tune renewal timing with the following flags:

  • -cert-expiry-window (default 5m): How long before certificate expiry to initiate renewal.
  • -cert-check-expiry-period (default 1m): How often to check certificate validity.

To rotate the SSH key pair, either delete the existing key file and restart the agent, or restart with the -force-key-file-overwrite flag. How you restart the agent depends on your installation method.

Private network connectivity

By default, the SSH connection between the PDC agent and Grafana Cloud travels over the public internet (encrypted within the tunnel). To keep this traffic entirely within your cloud provider’s private backbone, you can use one of the following private networking options:

Audit and logging

The PDC agent logs every connection attempt, whether successful, failed, or denied. You can use these logs to audit traffic and identify unauthorized access attempts. The agent writes logs to stdout/stderr, so the method you use to collect and store logs depends on where the agent is deployed and your logging infrastructure.

For Prometheus-based monitoring, the PDC agent also exposes metrics on connection counts, signing request latency, and TCP connection status. Refer to PDC agent metrics for details.

When the PDC agent successfully connects to Grafana Cloud, you see the following message in the logs:

This is Grafana Private Data Source Connect!

Successful connections with -log.level=debug enabled contain lines like the following:

debug1: client_input_channel_open: ctype forwarded-tcpip rchan 1 win 2097152 max 32768
debug1: client_request_forwarded_tcpip: listen localhost port 1234, originator ::1 port 61779
debug1: channel 1: new [::1]
debug1: confirm forwarded-tcpip
debug1: connect_next: host 34.205.150.168 ([34.205.150.168]:443) in progress, fd=10
debug1: channel 1: connected to 34.205.150.168 port 443
debug1: channel 1: free: ::1, nchannels 2

Failed connections may look like this. In this example, requested forward not permitted indicates that PermitRemoteOpen blocked the connection to the requested destination:

debug1: client_input_channel_open: ctype forwarded-tcpip rchan 1 win 2097152 max 32768
debug1: client_request_forwarded_tcpip: listen localhost port 1234, originator ::1 port 61917
debug1: channel 1: new [::1]
debug1: confirm forwarded-tcpip
debug1: rdynamic_connect_finish: requested forward not permitted
debug1: channel 1: free: ::1, nchannels 2

Note

The specific reason logged for a failed connection varies. Common causes include unreachable hosts, denied forwards, and connection timeouts. To resolve these errors, verify that the data source is reachable from the agent’s network and that PermitRemoteOpen allows the target host and port. For detailed guidance, refer to Troubleshooting PDC.

Change the log verbosity

By default, the PDC agent does not print debug logs from OpenSSH. Change the -log.level flag to debug to see OpenSSH debug messages.

Running with debug verbosity produces verbose output that includes SSH message numbers. The meanings of these message numbers are defined in RFC 4250.

OpenSSH 9.2+ requirement

OpenSSH versions 8.7 through 9.1 contain a bug affecting the PermitRemoteOpen SSH option, which is used to restrict endpoint access through PDC. This issue was resolved in OpenSSH 9.2. From the OpenSSH 9.2 release notes:

ssh(8): in OpenSSH releases after 8.7, the PermitRemoteOpen option
   would ignore its first argument unless it was one of the special
   keywords "any" or "none", causing the permission list to fail open
   if only one permission was specified.

When upgrading OpenSSH is not feasible, use the PDC agent Docker image, which includes a supported OpenSSH version. Alternatively, the PDC agent provides a -skip-ssh-validation flag to bypass version validation.

Warning

Using the -skip-ssh-validation flag with an affected OpenSSH version means that PermitRemoteOpen restrictions may not work correctly, potentially allowing Grafana Cloud to reach any host on your private network. This is not recommended for production environments.