GitLab integration for Grafana Cloud
GitLab is an open-source DevOps platform for software development. The GitLab integration uses the Grafana agent to collect metrics for monitoring a GitLab EE (Enterprise Edition) instance, including aspects such as HTTP request rates, HTTP request latencies, and CI pipeline creation rates. The integration also supports the rails error logs being scraped by the agent using Promtail. An accompanying dashboard is provided to visualize these metrics and logs.
This integration supports GitLab EE 15.3.3+.
Pre-install configuration for the GitLab integration
This integration monitors a GitLab EE instance that exposes its metrics through GitLab’s built in prometheus metrics server.
In order for the integration to work, you must enable the prometheus metrics as described in the GitLab documentation.
In order to collect metrics using the agent, the local IP must be added to the monitoring whitelist in /etc/gitlab/gitlab.rb
:
gitlab_rails['monitoring_whitelist'] = ['127.0.0.0/8']
Then, GitLab must be restarted for the above change to take effect:
sudo gitlab-ctl restart
Install GitLab integration for Grafana Cloud
- In your Grafana Cloud instance, click Integrations and Connections (lightning bolt icon).
- Navigate to the GitLab tile and review the prerequisites. Then click Install integration.
- Once the integration is installed, follow the steps on the Configuration Details page to setup Grafana Agent and start sending GitLab metrics to your Grafana Cloud instance.
Post-install configuration for the GitLab integration
This integration supports metrics and logs from a GitLab EE instance. In order to see these signals correlated on the same dashboard, the job
and instance
labels must match for the metrics
scrape config and the logs
scrape config in the Agent configuration file (replace <your-instance-name>
with an identifier for your instance):
metrics:
wal_directory: /tmp/wal
configs:
- name: integrations
scrape_configs:
- job_name: integrations/gitlab
metrics_path: "/-/metrics"
relabel_configs:
- replacement: "<your-instance-name>"
target_label: instance
static_configs:
- targets: ['localhost']
remote_write:
- url: http://cortex:9009/api/prom/push
logs:
configs:
- name: integrations
clients:
- url: http://loki:3100/loki/api/v1/push
positions:
filename: /var/lib/grafana-agent/logs/positions.yaml
scrape_configs:
- job_name: integrations/gitlab
static_configs:
- targets:
- localhost
labels:
instance: "<your-instance-name>"
job: integrations/gitlab
__path__: /var/log/gitlab/gitlab-rails/exceptions_json.log
pipeline_stages:
- match:
selector: '{job="integrations/gitlab"} |~ "^#"'
action: drop
drop_counter_reason: gitlab_comment_line
By default, the agent scrapes the rails error logs at this path: /var/log/gitlab/gitlab-rails/exceptions_json.log
.
Dashboards
The GitLab integration installs the following dashboards in your Grafana Cloud instance to help monitor your metrics.
- GitLab Overview
GitLab overview dashboard 1
GitLab overview dashboard 2
GitLab overview dashboard 3
Alerts
The GitLab integration includes the following useful alerts:
Group: GitLabAlerts
Alert | Description |
---|---|
GitLabHighJobRegistrationFailures | Warning: Large percentage of failed attempts to register a job. |
GitLabHighRunnerAuthFailure | Warning: Large percentage of runner authentication failures. |
GitLabHigh5xxResponses | Critical: Large rate of HTTP 5XX errors. |
GitLabHigh4xxResponses | Warning: Large rate of HTTP 4XX errors. |
Metrics
The following metrics are automatically written to your Grafana Cloud instance by connecting your GitLab instance through this integration:
- gitlab_ci_active_jobs_sum
- gitlab_ci_pipeline_size_builds_sum
- gitlab_ci_runner_authentication_failure_total
- gitlab_ci_runner_authentication_success_total
- gitlab_ci_trace_operations_total
- gitlab_rails_boot_time_seconds
- http_request_duration_seconds_count
- http_request_duration_seconds_sum
- http_requests_total
- job_register_attempts_failed_total
- job_register_attempts_total
- pipelines_created_total
- user_session_logins_total
Changelog
# 0.0.1 - November 2022
* Initial Release
Cost
By connecting your GitLab instance to Grafana Cloud you might incur charges. To view information on the number of active series that your Grafana Cloud account uses for metrics included in each Cloud tier, see Active series and dpm usage and Cloud tier pricing.