Black box monitoring of Butler tools - monitoring the monitor
This is part 3 in a series about how healthchecks.io can be used to solve various kinds of monitoring scenarios for Qlik Sense Enterprise on Windows (QSEoW) environments.
Previous articles:
Part 1: Oh data where art thou?
Part 2: Black box monitoring of Windows servers
Tl;dr
Most of the Butler family of DevOps tools for Qlik Sense can now be monitored using healthchecks.io or similar tool.
If one of the supported Butler tools go down, an alarm is raised and one or more alerts are sent via email, Slack, Teams, PagerDuty or other supported channels.
This means that both infrastructure (servers etc), applications/services and data sources can be monitored using the same tool and on the same dashboards.
Black box monitoring
The previous posts in this series has already explored black box testing, but it’s worth repeating what it means. The idea comes from the software engineering world:
BLACK BOX TESTING, also known as Behavioural Testing, is a software testing method in which the internal structure/design/implementation of the item being tested is not known to the tester.softwaretestingfundamentals.com/black-box-testing
In our case we modify the definition slightly and apply it to a monitoring scenario (rather than software testing).
The idea is the same though: the monitoring tool does not know – or care – what’s inside of the system being monitored. The monitored system is a “black box” which we cannot see inside.
The Butler family
The Butler family has grown to a handful of tools by now. They all focus on different aspects of Qlik Sense Enterprise on Windows, adding features that make life easier for Sense admins and developers.
| Which | What | Available | Can be monitored |
| Butler | Export apps to JSON, start tasks etc, misc utilities | Now | Yes |
| Butler SOS | Real time monitoring of operational metrics | Now | Yes |
| Butler CW | Cache warming (preload apps into memory) | Now | Yes |
| Butler Auth | Sign into Sense using for example Google account | Soon | Yes |
| Butler App Duplicator | Use templates when creating Sense apps | Now | No |
| Butler Icon Uploader | Upload batches of images to Sense | Now | Not applicable |
| Butler Spyglass | Data lineage tool | Now | No |
The Butler family of tools for Qlik Sense Enterprise on Windows, as of July 2020.
While some of the tools (such as Butler App Duplicator) are nice to have, other tools (Butler SOS, Butler Auth) can be close to mission critical.
All Butler tools are built on the same technology (Node.js) and are generally very stable. Still, it would be nice to monitor them for failures, and get notified when that happens.
Failures could actually happen for many reasons, not only the Butler tool itself failing:
- Network issues
- Server crashes
- Changed firewall rules etc
Thank’s to the passive nature of black box monitoring, all such issues will be caught if they result in the monitored system (i.e. one of the Butler tools) is no longer able to contact the monitoring tool (healthchecks.io).
Setting up monitoring of the Butler tools
It’s really quite easy to get started, especially if the cloud based/SaaS version of healthchecks.io is used. In that case the whole setup and configuration probably takes just a few minutes and their free tier will get you started at zero cost.
The general process is as follows (looking at Butler SOS in this example):
It’s a good idea to set the heart beat frequency in the Butler config file to a (slightly) lower value, compared to the value set in healthchecks.io. This way you can avoid false alerts being triggered.
The Butler SOS config file could look something like below.
Alerts when something breaks
The whole point of this exercise is to get alerts when something breaks, so let’s take a look at some examples. As healthchecks.io integrates with many different tools, you have a lot of options when it comes to the alerting part of the monitoring setup.
