Grafana Enterprise ServiceNow Datasource

This is the ServiceNow Datasource that is used to connect to ServiceNow instances.

Table of contents

1. Screenshots
2. Installation
   • Prerequisites
   • Adding Plugin to Grafana
3. Usage
   • TableAPI queries
   • Time series queries
   • Templating
   • Alerting
   • Using an HTTP Proxy
   • Annotations

Features

• Queries
  • Stat API Queries
  • Table API Queries
    • Incidents, Changes, and any other table
• Alerts
• Annotations (beta feature)
• Template Variables

Screenshots

This shows an example dashboard that queries 3 different tables and displays the results as a graph, table, and pie-chart.

Configuration

Datasource List with ServiceNow Configured

Incident Dashboard

Example Incident (INC) Query

Example Incident (INC) Table

Change Dashboard

Example Change Management (CHG) Query

Example Change Management (CHG) Table

Installation

This plugin should be extracted to /var/lib/grafana/plugins, which will create the directory /var/lib/grafana/plugins/grafana-servicenow-datasource.

Prerequisites

Permissions

You will need a user with permissions for reading data from relevant tables.

The following roles are recommended:

The following permissions are recommended, however these can be optional depending on the tables you plan on accessing.

The sn_incident_write permission is required if using the Notification Channel described in the alerting section.

Install the plugin

Adding a plugin to Grafana

Restart grafana-server

The grafana-server process will need to be restarted to pick up the new plugin. For Linux, typically:

service grafana-server restart

or

systemctl restart grafana-server

Validate Grafana Plugin Loads

The /var/log/grafana/grafana.log should contain this message:

grafana_1  | t=2019-12-11T07:58:42+0000 lvl=info msg="Registering plugin" logger=plugins name=ServiceNow
grafana_1  | t=2019-12-11T07:58:42+0000 lvl=dbug msg="plugin started" logger=plugins plugin-id=grafana-servicenow-datasource path=/var/lib/grafana/plugins/grafana-servicenow-datasource/dist/grafana-servicenow-datasource_linux_amd64 pid=23
grafana_1  | t=2019-12-11T07:58:42+0000 lvl=dbug msg="waiting for RPC address" logger=plugins plugin-id=grafana-servicenow-datasource path=/var/lib/grafana/plugins/grafana-servicenow-datasource/dist/grafana-servicenow-datasource_linux_amd64
grafana_1  | t=2019-12-11T07:58:42+0000 lvl=dbug msg="cleanup of expired auth tokens done" logger=auth count=0
grafana_1  | t=2019-12-11T07:58:42+0000 lvl=dbug msg="2019-12-11T07:58:42.236Z [DEBUG] grafana-servicenow-datasource: Running GRPC server" logger=plugins plugin-id=grafana-servicenow-datasource
grafana_1  | t=2019-12-11T07:58:42+0000 lvl=dbug msg="plugin address" logger=plugins plugin-id=grafana-servicenow-datasource address=/tmp/plugin039170772 network=unix timestamp=2019-12-11T07:58:42.236Z
grafana_1  | t=2019-12-11T07:58:42+0000 lvl=dbug msg="using plugin" logger=plugins plugin-id=grafana-servicenow-datasource version=1
g

Configuration

Select datasources on the left panel of Grafana.

Select Add Datasource:

Type in servicenow to find the datasource plugin:

Enter ServiceNow URL:

Click on "Save & Test" and you should see a green message with "ServiceNow Connection OK".

Example Provisioning File

Note: update the two fields to below with your username and password.

# config file version
apiVersion: 1

deleteDatasources:
  - name: ServiceNow
    orgId: 1

datasources:
- name: ServiceNow
  type: grafana-servicenow-datasource
  enabled: true
  access: proxy
  url: https://dev59952.service-now.com
  basicAuth: true
  basicAuthUser: ADD_YOUR_USER_HERE
  jsonData:
    basicAuthUser: ADD_YOUR_USER_HERE
    tlsSkipVerify: true
  secureJsonData:
    basicAuthPassword: ADD_YOUR_PASSWORD_HERE
  version: 1
  editable: true
  isDefault: true

Example Dashboards

Pre-made dashboards are included with the plugin and can be imported through the datasource configuration page, under the dashboards tab.

Usage

There are two "methods" of returning data in the query editor.

• TableAPI
• AggregateAPI

Users can currently choose between querying pre-defined tables, like:

• Changes
• Incidents

Or, as of v1.4.0, an API driven list of tables and fields using the Other (Custom Table) option. This option will allow you to query data that is in any table available to the user used to set up the ServiceNow datasource.

The Custom Table option should support all of the same features as the pre-defined table lists.

TableAPI queries

The TableAPI returns data suitable for displaying in a table panel. It allows for an ordered selection of fields to display plus filtering options. The query editor also provides a field to limit the number of rows returned by a query.

Example table panel showing results from query above:

Show

The Show row provides a selector for a field to be displayed. Multiple fields can be also be specified. The fields will be returned in the exact order specified.

Display Values

The Display Values flag will cause the query to return human-friendly values, or "Display Values", instead of numeric values.

For example, a severity of 1 without this flag would only display 1. If the flag is enabled, the value displayed will be 1 - High.

According to the ServiceNow API documentation, this can have a negative performance impact.

[...] specifying the display value may cause performance issues since it is not reading directly from the database and may include referencing other fields and records.

Filters (general)

The Filters row provides the ability to narrow down the displayed rows based on multiple field and value criteria.

All filters are combined with an AND or an OR operation.

The following fields are available when not using a custom table (this list will expand in the future):

Active
Asset
Group
Assigned To
Escalation
Issue Number
Description
Priority
State
Type
Change Risk
Change State
Start Date
End Date
On Hold

When selecting a custom table, fields are automatically populated from the Service Now API.

Date Filters

For additional date values, see: https://developer.servicenow.com/app.do#!/api_doc?v=newyork&id=r_SGSYS-dateGenerate_S_S

Operators (general/string-based)

• Starts With
• Ends With
• Like
• Not Like
• Equals
• Not Equals
• Is Empty

Operators (time-based)

• Today
• Not Today
• Before
• At or Before
• After
• At or After

Values

Value selection depends on the type of filter selected.

• Boolean filters have True/False options
• Text filters will allow typing any value
• Escalation, Priority has a fixed set of numerical values

Sort By

The Sort By row provides the ability to narrow down the displayed rows based on multiple field and value criteria.

All filters are combined with an AND operation. Support for additional operators will be added.

Limit

A row limit can be specified to prevent returning too much data. The default value is 25.

Time Field

The Time Field is what turns your queried data into a time series. Your data being handled as a time series means that values in your selected "time field" that do not fall within your dashboard / panel's time range will not be displayed.

The default time field used is "Opened At", but can be changed to any available field that holds a time value.

A special value "Ignore Time" is provided to allow results "up until now" and also to enable the filters to control what data is displayed.

AggregateAPI queries (Stats)

The AggregateAPI will always return metrics, with the following aggregations: avg, min, max, sum. Filtering is also available to narrow queries.

Show

The Show row provides a selector for a metric to be displayed. Multiple metrics can be also be specified.

Filters (general)

Aggregate Filters provide the ability to narrow down the displayed metrics based on field and value criteria, similar to the table option.

All filters are combined with an AND operation. Support for additional operators will be added.

Stat filter options are the same as the TableAPI.

Aggregation

There are four types of metric aggregations, plus a "count":

• Average
• Minimum
• Maximum
• Sum
• Count - this returns the "number" of metrics returned by a query

Group By

This selector provides the ability to split metrics into lesser aggregates. Grouping by "priority" would return the metrics with a "tag" of priority and the unique values separated.

Templating

Instead of hard-coding names in your queries, you can use variables in their place. Variables are shown as dropdown select boxes at the top of the dashboard. These dropdowns make it easy to change the data being displayed on your dashboard.

See the example below in the Query Variable section on how to add a query variable and reference that with a Template value.

Query Variable

If you add a template variable of the type Query, you can write a query that can return items like category names, key names, or key values that are shown as a dropdown select box.

For example, you can have a variable that contains all values for categories by specifying a query like this in the templating variable Query setting.

When choosing the Query setting, a Filter section is displayed, allowing you to choose a Type and Field. Currently, Type is limited to Incidents and Changes. When selecting a type, you are provided with a list of fields applicable to that Type. Once a Type and Field are selected, a preview of values will be displayed at the bottom showing the options available for that Type/Field. Those values will be displayed in a drop down on the Dashboard, which you can use along with Templating to filter data on your Dashboard Panels.

For example, if you add a Variable named category then select Type = Incidents and Field = Category, you will see a list of options for Category. If you then add a Filter to a panel, and select Category Equals ${category}, the panel data will show only data for that Category that is selected from the Dashboard drop down.

Import the Incidents By Category dashboard to see an example.

Using Variables in Queries

There are two syntaxes:

$<varname> Example with a template variable named hostname:

[[varname]] Example with a template variable named hostname:

Alerting

Standard Grafana alerting is supported. Any queries defined in a graph panel can be used to generate alerts.

The following is an example query and an alert. This query will return a graph of all open critical high priority incidents:

This alert will trigger when there are more than 5 open critical high priority incidents:

Testing the alert rule will display output from the alert rule, and selecting the state history will show the alert transitioning from ok to pending to alerting.

The graph view will show a vertical line and the "heart" at the top will turn orange while the alert is pending.

Once the criteria for alerting has been met, the rule transitions to red, and the state history will look like this:

In the graph view, the red vertical line will appear and the "heart" at the top will turn red.

Writing Incidents for Alerts

Beta feature

• Configure a Notification Channel for your ServiceNow datasource.

This will configure a Grafana Notification Channel which uses your configured user to create incidents on the ServiceNow instance for this datasource.

This action requires that the ServiceNow datasource user has permissions for writing incidents.

Using an HTTP Proxy

When using an HTTP Proxy, Grafana will need the following environment variable(s) set to the location of the proxy:

• HTTP_PROXY (or http_proxy)

  • Full path - http://host:port
  • or just: host:port

• HTTPS_PROXY (or https_proxy):

  • Full path - https://host:port
  • or just: host:port

Annotations

Grafana Annotations are a beta feature as of v1.4.0 of this plugin. Annotations give you the ability to overlay events on graphs.

The Annotations query supports the same options as the standard query editor with a few minor differences:

• Only one "Show" column is selectable. This is likely going to be fixed in a future improvement.
• The time field is required.