Azure Activity Log

https://github.com/bhozar/grafana-dashboards Azure Activity Logs dashboard.

Azure Activity Log Dashboard

Dashboard to visualize Azure Activity Logs data from Logstash Event Hubs plugin

Setup Azure Activity Log to stream data in an Azure EventHub so the ES plugin can pickup the data. Microsoft provide documentation: Export Azure Activity log to storage or Azure Event Hubs

Dashboard is setup to filter based on a subscription name by mapping the subscription GUID to a friendly name. The Logstash filter files have been provided on GitHub .

Bugs, suggestions and feedback

Bug reports, suggestions and feedback to GitHub please!

Logstash Filters

00-input-azureeventhubs.conf

input {
   azure_event_hubs {
   event_hub_connections => ["Endpoint=sb://XXXXXX.servicebus.windows.net/;SharedAccessKeyName=...."]
   decorate_events => true
   consumer_group => "logstash-opslogs"
   storage_connection => "DefaultEndpointsProtocol=https;AccountName=..."]
   tags => ["azureopslogs"]
   }
}

01-filter-azure-ops-logs.conf

filter {
if [@metadata][azure_event_hubs][consumer_group] == "logstash-opslogs" {
json {
    source => "message"
  }

  if "_jsonparsefailure" not in [tags] {
    split {
      field => "records"
      target => "data"
    }

    if "_split_type_failure" not in [tags] {
      ruby{
        code => "
          event.get('data').each { |k,v|
            event.set(k, v)
          }
          event.remove('data')
        "
        remove_field => ["message", "records"]
      }

      if "_rubyexception" not in [tags] {
        azure_event{}

        if "_azure_event_failure" not in [tags] {
          if [callerIpAddress]{
            geoip {
              source => "callerIpAddress"
            }
          }
          date {
              match => [ "time", "ISO8601" ]
          }
          mutate {
            add_field => { "azure.metadata.azure_event_hubs.name" => "%{[@metadata][azure_event_hubs][name]}" }
            add_field => { "azure.metadata.azure_event_hubs.consumer_group" => "%{[@metadata][azure_event_hubs][consumer_group]}" }
          }
          translate {
            field => "[azure][subscription]"
            destination => "azure_subscription_name"
             dictionary => {
               "XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX" => "Production"
               "YYYYYYYY-YYYY-YYYY-YYYY-YYYYYYYYYYYY" => "Non-Production"
               }
          }
        }
      }
    }
  }
}
}

To Do

Enrich data by including a Logstash dictionary of LocalizedValue fields or operationName as they are not supplied by the Azure Log streaming system and they will make the logs more readable.

    "operationName": {
        "value": "Microsoft.Resourcehealth/healthevent/Activated/action",
        "localizedValue": "Health Event Activated"
    },