Palo Alto Networks

Firewall health, capacity and updates for Palo Alto Networks firewalls. The data source reads each firewall's own XML API with one API key. Nothing is installed on the firewalls.

Requirements

• Grafana 12.4.0 or later.
• Firewalls on PAN-OS 11.1 or later.
• HTTPS from Grafana to each firewall's management interface, or to a data plane interface whose interface management profile permits HTTPS.
• An API key for an administrator whose role allows XML API Operational Requests and nothing else. Credentials, below, has the steps.

Getting started

1. Create the role, the administrator and the API key. Follow Credentials, below.

2. Add the data source. In Grafana, go to Connections, then Data sources, and add Palo Alto Networks. Enter the firewalls' addresses in Firewall list, separated by commas, and paste the key into API key. A firewall keeps a self-signed certificate until it is given another. Paste each firewall's certificate into CA certificates, or turn on Skip TLS certificate validation. Save & test reports how many firewalls answered.

3. Import the dashboards. Go to Connections, then Data sources, and open the data source. Open its Dashboards tab and select Import beside each dashboard. Start with Firewall health.

Credentials

The data source holds one API key and sends it to every firewall in Firewall list. Every firewall in the list must accept that key. A firewall accepts it when it has the same administrator, with the same password, as the firewall the key was generated on. It must also have the same master key, or the same API key certificate. An API key certificate is a certificate the firewall encrypts its API keys with. It is set in Device, then Setup, Management, Authentication Settings. Firewalls with the same API key certificate accept one key even when their master keys differ. Setting the certificate makes every existing key invalid. Generate the key after setting it. Firewalls that cannot share a key go in data sources of their own.

On each firewall the data source reads, in the web interface:

1. Create the role. Go to Device, then Admin Roles, and select Add. Enter a name, such as grafana-readonly. On a firewall with multiple virtual systems, the administrator must be device-wide. There, the role has a Role field. Set it to Device, not Virtual System.

   • On the Web UI tab, set every item to Disable.
   • On the XML API tab, set Operational Requests to Enable and every other item to Disable.
   • On the Command Line tab, select None.
   • On the REST API tab, set every item to Disable.

2. Create the administrator. Go to Device, then Administrators, and select Add. Enter a name, such as grafana, and a password. Set Administrator Type to Role Based and Profile to the role you created in step 1.

3. Commit the change.

Then generate the key once, on any one of the firewalls, with the administrator's name and password. Run this from a machine that reaches the firewall over HTTPS. It asks for the password:

printf 'Password: '; stty -echo; read -r PASSWORD; stty echo; echo
curl 'https://fw-dc-01.example.com/api/?type=keygen' \
  --data-urlencode 'user=grafana' --data-urlencode "password=$PASSWORD"
unset PASSWORD

Add -k to the curl command if the firewall has a self-signed certificate. The response contains the API key, as in <response status = 'success'><result><key>LUFRPT…</key></result></response>. Paste the text between <key> and </key> into the data source's API key field.

A key that expires or is revoked is replaced by generating a new one and saving it in API key.

What the role can read

Operational Requests lets the key send any operational command, not only the plugin's own. The key can read more than the plugin needs, including the firewall's IPsec keys in clear through show system state.

With the key	Result
show commands	Answered, apart from a few such as show device-certificate status
request commands, such as request license info	Answered
show config running	Answered with the whole running configuration. Password hashes and private keys are masked as ********
show system state	Answered with the firewall's internal state, including its IPsec keys in clear
Configuration requests, type=config	Refused

From the running configuration, the plugin reads only the configured certificates and the firewall's time zone. Raw operational command sends only the commands listed under Raw operational command, below.

Data source settings

Setting	Default	Behavior
Firewall list	None	1 to 50 addresses, separated by commas. An address is a hostname, an IPv4 address or an IPv6 address in square brackets, with an optional :port, and no scheme or path. Each address appears once. Dashboards list the firewalls in this order
API key	None	Sent to every firewall in the list
Skip TLS certificate validation	Off	Connects to every firewall in the list without verifying its certificate
CA certificates	None	One or more PEM certificates, one after another. A firewall is trusted when its certificate chains to one of them and names the address it is listed by. A firewall whose own certificate is one of them is trusted whatever address it is listed by
Timeout	30 seconds	For each attempt, from opening the connection to the end of the answer
Request attempts	3	1 to 5. Only an answer of HTTP 429 or a 5xx is sent again. The wait is the answer's Retry-After, or else 1 second. It doubles for each further attempt. A firewall that does not answer, as on a timeout or a refused connection, is asked again on the next refresh
Concurrent calls per firewall	5	1 to 5. The most requests sent to one firewall at once. The firewall's management plane also serves its web interface. Palo Alto Networks recommends no more than five

An address in Firewall list with no port is tried on port 443, then 4443. Port 4443 is used when 443 answers with something other than the XML API, as it does when GlobalProtect takes 443 on that interface. An address with a port is read on that port only.

Two addresses that reach the same firewall are read as one. The later address gets the status Already listed. Save & test fails and asks for it to be removed from Firewall list.

The data source reads up to ten firewalls at once.

To copy a firewall's certificate for CA certificates, give the port the data source reads it on. That is 4443 where GlobalProtect takes 443:

openssl s_client -connect fw-dc-01.example.com:443 </dev/null | openssl x509

Save & test sends show system info to every firewall in the list. It succeeds when every firewall answers. Otherwise it names each address that did not, with its status and the reason.

Provisioning

On a Grafana you run yourself, the data source can be created from a file in Grafana's provisioning/datasources directory:

apiVersion: 1
datasources:
  - name: Palo Alto Networks
    type: integrationplumbers-paloaltofirewall-datasource
    uid: palo-alto-networks
    access: proxy
    jsonData:
      firewallList: fw-dc-01.example.com, fw-dc-02.example.com, 192.0.2.10:8443
      timeout: 30
      requestAttempts: 3
      concurrentCalls: 5
      tlsSkipVerify: false
    secureJsonData:
      apiKey: $PANFW_API_KEY
      tlsCACert: $__file{/etc/grafana/palo-alto-networks-ca.pem}

Grafana reads $PANFW_API_KEY from its environment, and $__file{...} from that file. Settings left out take their defaults. enableSecureSocksProxy: true sends the requests through Grafana's secure SOCKS proxy. Settings that break a rule of the settings page fail Save & test with that rule's message, and no firewall is contacted.

Dashboards

Dashboard	Shows
Firewall health	Which firewall needs attention
Firewall detail	One firewall: sessions, CPU, memory, disk, interfaces, HA, sensors, drops and GlobalProtect
Expiry and updates	Certificate and license expiry, content age and PAN-OS version
Packet drops	What the firewalls are dropping, and why

Import them from the data source's Dashboards tab, as in Getting started. Each dashboard has a Data source selector.

Panels show the latest reading. The resource monitor panels on Firewall detail show the firewall's own history. A traffic, error or drop rate needs two readings of a firewall, at least 10 seconds apart. When a dashboard opens, its rate panels are empty until the next refresh. They show rates at once when the data source has read the same counters in the past hour.

A plugin update may overwrite the imported dashboards. To customize one, save a copy and edit that.

Query editor

Query type	Returns
Firewall inventory	Each firewall in the list with its status, hostname, serial, model, PAN-OS version and uptime
Content versions	Applications, Threats, Antivirus, WildFire and other content, with release dates and age
Sessions	Session table use, sessions by protocol, and connection and packet rates
CPU and memory	Management plane and data plane CPU, memory and swap
Disk	Filesystem use
Resource monitor	Data plane CPU per core, CPU by function and resource use over the chosen window: second, minute, hour, day or week
Interfaces	State, zone, virtual system and speed of each interface
Interface counters	Traffic, error and discard rates of chosen interfaces
Global counters	The global counters, filtered by category, severity, aspect and name
High availability	HA mode, local and peer state, configuration synchronization and the HA links
Licenses	Each license and its expiry
Certificates	Each configured certificate and its expiry
GlobalProtect	Current and previous GlobalProtect users, in total and per gateway
Environmentals	Temperature, fan and power readings and their alarms
Raw operational command	The answer to a command from the list below, as a table

Each query reads the firewalls chosen in Firewalls, or all of them. Only Resource monitor reads history. Every other query type returns the latest reading, whatever the dashboard's time range. Interface counters reads at most 100 interfaces per firewall. On Global counters, Row limit sets the counters kept from each firewall: 100 by default, and 1,000 at most. They are the ones with the highest rate.

Raw operational command

Raw operational command sends only these commands, written as XML as shown. Any user who can query the data source can send them. Any other command is refused and sent to no firewall.

Command	XML
show system info	<show><system><info></info></system></show>
show session info	<show><session><info></info></session></show>
show system resources	<show><system><resources></resources></system></show>
show system disk-space	<show><system><disk-space></disk-space></system></show>
show system environmentals	<show><system><environmentals></environmentals></system></show>
show counter global	<show><counter><global></global></counter></show>
show global-protect-gateway statistics	<show><global-protect-gateway><statistics></statistics></global-protect-gateway></show>
show high-availability all	<show><high-availability><all></all></high-availability></show>
show interface all	<show><interface>all</interface></show>
show interface {name}	<show><interface>ethernet1/1</interface></show>, with an interface name of letters, digits and / . : _ - that starts with a letter or a digit
show running resource-monitor {window}	<show><running><resource-monitor><minute></minute></resource-monitor></running></show>, with second, minute, hour, day or week
show system masterkey-properties	<show><system><masterkey-properties></masterkey-properties></system></show>
show sslmgr-store config-certificate-info	<show><sslmgr-store><config-certificate-info></config-certificate-info></sslmgr-store></show>
show sslmgr-store config-ca-certificate	<show><sslmgr-store><config-ca-certificate></config-ca-certificate></sslmgr-store></show>

A command must match the list exactly. Its elements are lowercase, as shown, with no attributes and no further arguments. A command is at most 512 characters long and nests at most 16 elements. Row limit sets the rows kept from each firewall: 1,000 by default, and 10,000 at most.

Template variables

For building your own dashboards. Create each as a Query variable on a Palo Alto Networks data source.

Query type	Lists	Settings
Firewall inventory	Each firewall in the list, by hostname, or by address until it has answered. Its value is the address	For Multi-value, turn on Include All option and set Custom all value to *
Interfaces	The interfaces of the firewalls in Firewalls, such as $firewall	For Multi-value, turn on Include All option and set Custom all value to *

Private Data Source Connect and the secure SOCKS proxy

When Grafana has its secure SOCKS proxy enabled, the data source's settings page shows a Secure Socks Proxy section. Turn on Enabled there to send every request to every firewall in Firewall list through the proxy. On Grafana Cloud, this is how Private Data Source Connect reaches firewalls on a private network.

The Private Data Source Connect agent must reach every firewall in the list, on the port each one is read on. That is the port an address gives, or 443 for an address with no port. An address with no port is also tried on 4443.

Alert rules

The plugin includes the 15 alert rules below as one Grafana provisioning file, alerting/palo-alto-networks.yaml in the plugin's package.

Rule	Fires when	Checked every	Pending period
Firewall not answering	A firewall in Firewall list does not answer, or refuses the API key or its role	1m	5m
Collection failing	Commands sent to a firewall that answers are failing	5m	15m
Session table filling	The session table is more than 90% full	1m	5m
Management plane CPU high	Management plane CPU is above 95%	1m	15m
Data plane CPU high	A data plane's CPU is above 95%	1m	5m
Interface down	An interface is down. An interface down on the passive member of an HA pair does not count	1m	5m
Interface errors or discards	An interface's receive errors, transmit errors or receive discards are above 1 a second	5m	15m
HA unhealthy	A firewall in an HA pair reports a state other than active, passive, active-primary or active-secondary	1m	5m
HA configuration not synchronized	The configuration of an HA pair is not synchronized	5m	15m
License expiring	A license expires within 30 days, or has expired	1h	0s
Certificate expiring	A certificate expires within 30 days, or has expired	1h	0s
Applications and Threats content out of date	The installed Applications or Threats content was released more than 14 days ago	1h	0s
Antivirus content out of date	The installed Antivirus content was released more than 7 days ago	1h	0s
WildFire content out of date	The installed WildFire content was released more than 2 days ago	1h	0s
Sensor alarm	A temperature, fan or power sensor is in alarm	5m	5m

Support

Email helpdesk@integrationplumbers.io.

Use of the plugin is subject to the Integration Plumbers acceptable use policy.