Important: This documentation is about an older version. It's relevant only to the release noted, many of the features and functions have been updated or replaced. Please view the current version.

Documentation Grafana Enterprise Logs Configure Configure the GEL gateway

Configure the Grafana Enterprise Logs gateway

The Grafana Enterprise Logs (GEL) gateway is a service target. It can proxy requests to other GEL microservices. You can also use it for client-side load balancing of requests proxied to the distributors.

Configuration

There is more than one way to configure the gateway: you can configure it via a code block in the YAML configuration file for GEL, or you can use command-line flags.

Configuration code block

The gateway has its own configuration block in the GEL configuration files:

yaml Copy

gateway:
  proxy:
    default: <backend_proxy_config>
    [ admin_api: <backend_proxy_config> ]
    [ compactor: <backend_proxy_config> ]
    [ distributor: <backend_proxy_config> ]
    [ ingester: <backend_proxy_config> ]
    [ query_frontend: <backend_proxy_config> ]
    [ ruler: <backend_proxy_config> ]

Configuration flags

To configure the gateway via command-line flags, make sure that each flag contains the path to the equivalent configuration field.

To configure the distributor backend proxy URL, use the flag -gateway.proxy.distributor.url=<distributor URL>.

Likewise, given a configuration file field path of gateway.proxy.query_frontend.url, the flag is gateway.proxy.query-frontend.url.

Configure the backend proxy

A backend_proxy section specifies the URL and configuration of the backend that you want proxied. A URL is only required for the default proxy configuration. All other proxy configurations use the default URL if it is not explicitly configured. Other proxy configurations do not inherit any other configuration from the default proxy configuration.

yaml Copy

url: <url> | default = <gateway.proxy.default.url>
[ enable_keepalive: <boolean> | default = true ]
[ tls_enabled: <boolean> | default = false ]
[ tls_ca_path: <string> | default = "" ]
[ tls_cert_path: <string> | default = "" ]
[ tls_key_path: <string> | default = "" ]
[ tls_insecure_skip_verify: <boolean> | default = false ]
[ read_timeout: <duration> | default = "120s" ]
[ write_timeout: <duration> | default = "30s" ]

Client-side load balancing

If you use a backend proxy URL that beings with dns:///, the gateway creates a gRPC proxy with client-side round-robin load balancing instead of the default HTTP reverse proxy. To configure client-side load balancing for requests to the distributors only, set the gateway.proxy.distributor.url to dns:///<distributor service>.

Client-side load balancing is useful in ensuring that distributors are evenly loaded with requests. Prometheus remote-write clients use HTTP persistent connections, also known as HTTP keep-alive, the clients re-use a single TCP connection for multiple requests and responses, which results in reduced latency for subsequent requests.

Kubernetes Services are not load balancers; initial TCP connections are made using a random endpoint. After the connection is established, the same remote-write client talks to the same distributor server for its lifetime. This can mean an uneven load for your distributors and worse cluster performance overall.

The GEL gateway solves this problem by exposing an HTTP server for receiving the client requests while using gRPC to talk to the distributors. The gRPC proxy maintains a list of endpoints that are returned from the DNS lookup, and keeps persistent connections to each endpoint. The proxies are also configured to perform per-request, client-side load balancing across the endpoints.

Ring endpoints

The Gateway exposes the ring endpoints for the rulers, distributors, ingesters and compactors. These endpoints require admin scope permissions.

• /ring for the ingesters
• /ruler/ring for the rulers
• /compactor/ring for the compactors
• /distributor/ring for the distributors