Gateway

Gateway

The Grafana Enterprise Traces (GET) gateway is a service target. It can proxy requests to other GET microservices. You can also use it for client-side load balancing of requests proxied to the distributors.

Configuration

There is more than one way to configure the gateway: you can configure it via a code block in the YAML configuration file for GET, or you can use command-line flags.

Configuration code block

The gateway has its own configuration block in the GET configuration files:

gateway:
  proxy:
    default: <backend_proxy_config>
    [ admin_api: <backend_proxy_config> ]
    [ compactor: <backend_proxy_config> ]
    [ distributor: <backend_proxy_config> ]
    [ ingester: <backend_proxy_config> ]
    [ querier: <backend_proxy_config> ]
    [ query_frontend: <backend_proxy_config> ]

Configuration flags

To configure the gateway via command-line flags, make sure that each flag contains the path to the equivalent configuration field.

To configure the distributor backend proxy URL, use the flag -gateway.proxy.distributor.url=<distributor URL>.

Likewise, given a configuration file field path of gateway.proxy.query_frontend.url, the flag is gateway.proxy.query-frontend.url.

Note: A path is joined by a period (.) and dashes (-). Underscores (_) are invalid within these command-line flags.

<backend_proxy_config>

A backend_proxy section specifies the URL and configuration of the backend that you want proxied. A URL is only required for the default proxy configuration. All other proxy configurations use the default URL if it is not explicitly configured. Other proxy configurations do not inherit any other configuration from the default proxy configuration.

url: <url> | default = <gateway.proxy.default.url>
[ enable_keepalive: <boolean> | default = true ]
[ tls_enabled: <boolean> | default = false ]
[ tls_ca_path: <string> | default = "" ]
[ tls_server_name: <string> | default = "" ]
[ tls_cert_path: <string> | default = "" ]
[ tls_key_path: <string> | default = "" ]
[ tls_insecure_skip_verify: <boolean> | default = false ]
[ read_timeout: <duration> | default = "120s" ]
[ write_timeout: <duration> | default = "30s" ]

Client-side load balancing

If you use a backend proxy URL that beings with dns:///, the gateway creates a gRPC proxy with client-side round-robin load balancing instead of the default HTTP reverse proxy. To configure client-side load balancing for requests to the distributors only, set the gateway.proxy.distributor.url to dns:///<distributor service>.

Note: There are three / characters in the preceding DNS URL, which means that you are using the default DNS authority. For details about DNS URLs, refer to RFC 4501.

Client-side load balancing is useful in ensuring that distributors are evenly loaded with requests.

Kubernetes Services are not load balancers; initial TCP connections are made using a random endpoint. After the connection is established, the same remote-write client talks to the same distributor server for its lifetime. This can mean an uneven load for your distributors and worse cluster performance overall.

The GET gateway solves this problem by exposing an HTTP server for receiving the client requests while using gRPC to talk to the distributors. The gRPC proxy maintains a list of endpoints that are returned from the DNS lookup, and keeps persistent connections to each endpoint. The proxies are also configured to perform per-request, client-side load balancing across the endpoints.

Ring endpoints

The Gateway exposes the ring endpoints for distributors, ingesters, and compactors. These endpoints require admin scope permissions.

  • /distributor/ring for the distributors
  • /ingester/ring for the ingesters
  • /compactor/ring for the compactors