This one is long overdue, but finally here: Butler SOS v2.0
The new version is an almost complete re-write of v1.0. Changes are plentiful and include
All warnings and errors stored by Sense in its log database are now pulled into Butler SOS, from where it can be graphed and acted upon. This is a big deal, as it was previously not possible to get notifications or alerts when errors or warnings started to pile up in the logs.
Operational health metrics are still pulled from Qlik Sense, but this is now done directly from the QIX engine rather than via a hard-to-secure virtual proxy.
Using certificates for authentication with Sense removes potential security issues with v1.0.
Config file is now YAML instead of JSON. More human readable and with inline comments.
Config file now allows for more fine-grained control of Butler SOS.
Several bugs fixed, especially around sending metrics to MQTT.
The readme file on GitHub has all the details, here are some screen shots to get you started though:
I got somewhat distracted from the idea of breaking up the existing Butler software into smaller, stand-alone micro services.
Or rather, an idea came to mind. An idea too good not to explore…
The healthcheck API of Qlik Sense provides basic metrics for both the Qlik Sense engine itself, and the server it is running on. Things like CPU load, available RAM, number of connected users and what apps are loaded into the Sense engine.
The idea behind Butler SOS ( SOS = SenseOps Stats) is very simple:
Get the healthcheck metrics for all servers in a Sense cluster. Then send the information to MQTT for immediate, real-time use cases.
It is directly aimed at bringing better features to the monitoring step of SenseOps – please visit SenseOps.rocks for more info on SenseOps.
Butler SOS is nice and sending data via MQTT make the health metrics available in for example Node-RED. Node-RED has some basic graph options, but not anywhere near those offered by Grafana. Grafana is very, very cool… A live demo is available here – do check it out – it is very nice indeed.
Creating real-time dashboards in Grafana is greatly simplified if the data is stored in some kind of time series database. Influxdb is an obvious choice. It is open source, installation is very easy, and there are good Node.js libraries that make it trivial to insert data into a Influxdb database.
Thus – Butler SOS also sends the Sense health metrics to an Influx db of choice.
Only need Influxdb and not MQTT? Or the other way around?
No problem, the Butler SOS config file include options for independently turning on/off sending of data to MQTT and Influxdb.