K8S Control Plane

Metrics for API Server, Controller and Scheduler.

Based on this very helpful blog

https://blog.freshtracks.io/a-deep-dive-into-kubernetes-metrics-part-4-the-kubernetes-api-server-72f1e1210770

It contains below

Enable API Server Prometheus scraping
Assuming your prometheus scrapes based on service annotations, create services for Scheduler and Controller as outlined below

Promethus Configuration for API Server

- job_name: kubernetes-apiservers
      scrape_interval: 10s
      scrape_timeout: 10s
      metrics_path: /metrics
      scheme: https
      kubernetes_sd_configs:
      - api_server: null
        role: endpoints
        namespaces:
          names: []
      bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token
      tls_config:
        ca_file: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
        insecure_skip_verify: true
      relabel_configs:
      - source_labels: [__meta_kubernetes_namespace, __meta_kubernetes_service_name, __meta_kubernetes_endpoint_port_name]
        separator: ;
        regex: default;kubernetes;https
        replacement: $1
        action: keep

Service for kube-scheduler

apiVersion: v1
kind: Service
metadata:
  annotations:
    prometheus.io/scrape: "true"
    prometheus.io/scrape_port: "10251"
  namespace: kube-system
  name: kube-scheduler-prometheus-discovery
  labels:
    component: kube-scheduler
spec:
  selector:
    component: kube-scheduler
  type: ClusterIP
  clusterIP: None
  ports:
  - name: http-metrics
    port: 10251
    targetPort: 10251
    protocol: TCP

Service for kube-controller

apiVersion: v1
kind: Service
metadata:
  annotations:
    prometheus.io/scrape: "true"
    prometheus.io/scrape_port: "10252"
  namespace: kube-system
  name: kube-controller-manager-prometheus-discovery
  labels:
    component: kube-controller-manager
spec:
  selector:
    component: kube-controller-manager
  type: ClusterIP
  clusterIP: None
  ports:
  - name: http-metrics
    port: 10252
    targetPort: 10252
    protocol: TCP

Revisions

Revision	Description	Created
			Download

Kubernetes

Grafana Labs solution

Monitor your Kubernetes deployment with prebuilt visualizations that allow you to drill down from a high-level cluster overview to pod-specific details in minutes.

Get this dashboard

Import the dashboard template

Download JSON

Resources

Docs: Importing dashboards Webinar: Getting started with Grafana dashboard design Webinar: Building advanced Grafana dashboards

Datasource

Dependencies

Feedback

Relevant sources: