Dataset data source for Grafana

The Dataset Grafana data source plugin allows you to create and visualize graphs and dashboards in Grafana using data in Dataset. You may want to use this plugin to allow you to visualize Dataset data next to other data sources, for instance when you want to monitor many feeds on a single dashboard.

With the Dataset plugin, you will be able to create and visualize your log-based metrics along side all of your other data sources. It's a great way to have a single pane of glass for today's complex systems. You can leverage Grafana alerts based on Dataset data to notify you when there are possible issues. More importantly, you'll soon be able to jump to Dataset's fast, easy and intuitive platform to quickly identify the underlying causes of issues that may arise.

Prerequisites

• An installed Grafana server instance with write access: This document assumes that an existing instance of Grafana already exists. If you need help bringing up a Grafana instance, please refer to the documentation provided by Grafana.
• A Dataset read log API Key: A Dataset API key is required for Grafana to pull data from Dataset. You can obtain one by going to your account in the Dataset product and selecting the “API Keys” from the menu in the top right corner. You can find documentation on API Keys here.

Installation

Using grafana-cli: grafana-cli plugins install sentinelone-dataset-datasource

Alternatively can download it here and unzip it manually into the Grafana plugins directory (eg /var/lib/grafana/plugins). A restart of the Grafana server is required afterwards.

Configuration

1. Log in to your grafana instance and navigate to Configuration Settings -> Data sources.

   

2. This will take you into the configuration page. If you already have other data sources installed, you will see them show up here. Click on the Add data source button:

   

3. If you enter "Dataset" in the search bar on the resulting page you should see "Dataset" grafana plugin show up as an option.

   

4. Click on Select. This will take you to a configuration page where you insert your API key mentioned in the prerequisite section.

   

5. Enter these settings:

   Field Name	Value
   Dataset API Key	Your Scalyr Read Logs API Key
   Dataset URL	https://app.scalyr.com or https://app.eu.scalyr.com for EU users.

6. Click Save & Test to verify these settings are correct.

Using the Dataset Datasource

Now that you’ve completed installing and configuring the Dataset data source plugin, lets go through an example of how you can start using it to create a dashboard using Scalyr data.

1. Create a new dashboard by click Create -> Dashboard.

   

2. In the New dashboard box, select the Add a new panel icon.

3. From the Data source dropdown, select Dataset.

   

4. A Query Type field allows to choose the type of query you wanted to search for.

   

5. Standard Query - A standard query allows to search on Graph view. You can enter graph functions into the expression box and visualize the results. You can even enter and visualize complex expressions. This is a good resource to see the list of supported functions.

   Enter an expression and click the save button. In the image below, we've added a query to graph the number of log messages that contain the word "error".

   

6. Power Query - Works similar to PQ search in Dataset app. You can enter rich set of commands for transforming and manipulating data. Data can be viewed in table format. Visit this page for more information on building Power Queries.

   Note PowerQuery results are unsorted by default. If using a PowerQuery to populate a timeseries graph, make sure to sort results by appending | sort -timestamp to the query.

   

7. Cross Team Search - The Team Emails field is used to specify which teams the query should be run against. Works the same as Cross Team Search in the DataSet app. More information on teams can be found here. Note that the owner of the Log Read API key used to setup the DataSet Datasource must be a member of all teams being queried.

   

You’ve successfully installed, configured and created a graph in Grafana using Dataset data!

Note: you can add multiple queries to a visualization to plot multiple series on the same graph.

Variables

For general information on Grafana variables see the documentation on variable syntax and variable format options

Queries support all Grafana variable substitution syntaxes, for example:

$varname
${varname}
${varname:<format>}

For multi-value variables use the singlequote or doublequote formatting option:

${varname:singlequote} => 'value1','value2','value3'
${varname:doublequote} => "value1","value2","value3"

The expected use of multi-value variables is for in queries, for example:

$serverHost in (${host:singlequote})