Push metrics from Influx Telegraf to Prometheus
We now have support to ingest Influx Line protocol into Grafana Cloud. You can now point your Telegraf and other services pushing Line protocol metrics to GrafanaCloud via HTTP.
For example, if you are using Telegraf to push metrics, the following configuration is required:
[[outputs.influxdb]]
urls = ["<Modified metrics instance remote_write endpoint>/api/v1/push/influx"]
## HTTP Basic Auth
username = "Your Metrics instance ID"
password = "Your Grafana.com API Key"
You can find the URL, username, and password for your metrics endpoint by clicking on Details in the Prometheus card of the Cloud Portal.
The URL is based on the Remote Write Endpoint URL but is changed slightly.
You need to replace prometheus
with influx
and change the path from /api/prom/push
to api/v1/push/influx
.
For example if the remote_write endpoint is https://prometheus-prod-03-prod-us-central-0.grafana.net/api/prom/push
, your influx endpoint will be https://influx-prod-03-prod-us-central-0.grafana.net/api/v1/push/influx
.
NOTE: There is one special case exception to the new endpoint form. If your endpoint url was prometheus-us-central1.grafana.net
, your new influx endpoint will be https://influx-prod-06-prod-us-central-0.grafana.net/api/v1/push/influx
.
Influx line protocol has the following structure:
metric_name (set of k=v tags) (set of k=v fields) timestamp
We convert the above into the following series in Prometheus:
for each (field, value) in fields:
metric_name_field{tags...} value @timestamp
For example:
diskio,host=work,name=dm-1 write_bytes=651264i,read_time=29i 1661369405000000000
Will be converted to:
diskio_write_bytes{host="work", name="dm-1"} 651264 1661369405000000000
diskio_read_time{host="work", name="dm-1"} 29 1661369405000000000
The timestamp is optional. If specified, we expect the timestamp to be encoded in Nanoseconds. If no timestamp is specified, we will use the timestamp millisecond that our server receives the metric.
Pushing from applications directly
NOTE: The actual destination path for this API is /api/v1/push/influx/write
.
The telegraf plugin automatically appends “/write” to the URL provided, so it shouldn’t be specified in the configuration.
If you’re using a different plugin or pushing directly, you may need to specify the full path of /api/v1/push/influx/write
in your endpoint URL.
For example: https://influx-prod-03-prod-us-central-0.grafana.net/api/v1/push/influx/write
Additional examples using cURL, Node.js, Python, and Ruby:
cURL
API_KEY="<user_id>:<api-key>"
URL="https://influx[...].grafana.net/api/v1/push/influx/write"
curl -X \
POST -H \
"Authorization: Bearer $API_KEY" -H \
"Content-Type: application/json" "$URL" -d "test,bar_label=abc,source=grafana_cloud_docs metric=35.2"
Node.js
import fetch from 'node-fetch';
const USER_ID = '123456';
const API_KEY = 'your-api-key-123-!!!';
const body = 'test,bar_label=abc,source=grafana_cloud_docs metric=35.2';
// Endpoint is on the same page as the userid / API Key
const response = await fetch('https://influx[...].grafana.net/api/v1/push/influx/write', {
method: 'post',
body,
headers: {
'Authorization': `Bearer ${USER_ID}:${API_KEY}`,
'Content-Type': 'application/json',
},
method: 'post',
body,
headers: {
'Authorization': `Bearer ${USER_ID}:${API_KEY}`,
'Content-Type': 'text/plain',
},
});
const data = await response.json();
console.log(data);
Python
import requests
import base64
USER_ID = '123456'
API_KEY = 'your-api-key-123-!!!'
body = 'test,bar_label=abc,source=grafana_cloud_docs metric=35.2'
response = requests.post('https://influx[...].grafana.net/api/v1/push/influx/write',
headers = {
'Content-Type': 'text/plain',
},
data = str(body),
auth = (USER_ID, API_KEY)
)
status_code = response.status_code
print(status_code)
Ruby
require 'net/http'
require 'uri'
require 'base64'
USER_ID = '123456'
API_KEY = 'your-api-key-123-!!!'
BASE64_ENCODED_AUTH = Base64.strict_encode64(USER_ID + ':' + API_KEY)
uri = "https://influx[...].grafana.net/api/v1/push/influx/write"
uri = URI.parse(uri)
body = 'test,bar_label=abc,source=grafana_cloud_docs metric=35.2'
response = Net::HTTP.start(uri.host, uri.port, use_ssl: true) do |client|
request = Net::HTTP::Post.new(uri.path)
request.body = body
request["Authorization"] = "Basic #{BASE64_ENCODED_AUTH}"
request["Content-Type"] = "text/plain"
client.request(request)
end
data = response.code
puts data
Pushing using the remote-write output
Telegraf also supports using a native Prometheus remote-write endpoint.
You can use the same Prometheus URL given in your cloud portal for this. No need to change the URL in anyway.
Current limitations:
- No matter what the precision you send the data in, we will store it in millisecond precision. We currently don’t have support in our platform to do higher than millisecond precision.
- We support only float64 on our platform. This means all the integer and boolean values will be cast into floating point before storage. True becomes 1 and false becomes 0. We currently don’t ingest string values.
- We don’t support queries via Flux, you will need to use PromQL, but we have it part of our roadmap to support Flux.