Black box monitoring of Butler tools – monitoring the monitor

This is part 3 in a series about how healthchecks.io can be used to solve various kinds of monitoring scenarios for Qlik Sense Enterprise on Windows (QSEoW) environments.

Previous articles:

Part 1: Oh data where art thou?
Part 2: Black box monitoring of Windows servers

Tl;dr

Most of the Butler family of DevOps tools for Qlik Sense can now be monitored using healthchecks.io or similar tool.

If one of the supported Butler tools go down, an alarm is raised and one or more alerts are sent via email, Slack, Teams, PagerDuty or other supported channels.

This means that both infrastructure (servers etc), applications/services and data sources can be monitored using the same tool and on the same dashboards.

Health dashboard at Ptarmigan Labs, covering both servers, applications and data.
Continue reading “Black box monitoring of Butler tools – monitoring the monitor”

Oh data where art thou?

When implementing Qlik Sense solutions for enterprise clients they usually have various requirements regarding alerts for failed reloads, support SLAs etc.
Those are all interesting challenges to implement, but the most common request is probably:

“The system should alert me if data doesn’t arrive on time”

It’s a reasonable request. If some source system is delayed and doesn’t delivery data to Sense on time, they as a system or application owner should be notified.

Don’t bother me unless it’s broken

The second thing almost all clients say is:

“Oh, I only want alerts when data is delayed. No messages when data is on time.”

Again, this is very reasonable.

Let’s say the extract app in Sense sends a notification email once data has been loaded from the source system. Great – we now know that data has arrived, and when it did.
Except that we will drown in such notifications from all our dozens (or hundreds..) of extract apps.

It’s kind of hard to implement this in a good way though, at least I have never found a really good, generic solution for this request.
Sure, you can have a Sense app who’s only job is to execute every 15 minutes and check whether data has arrived, and alert if data is delayed. That app will however put load on your Sense environment and use one of the reload slots. Even if the app reloads quickly it’s still pretty bad system design, IMHO.

I have a suggestion for a better option:

Set up a monitoring tool that views the Sense app (or rather its load script) as a black box with unknown function. The only thing the monitoring tool cares about is whether that black box has checked in within some configured interval.

I stumbled upon this while looking at ways to monitor servers in general, but quickly realised it could be used also with Sense.
By the way: while I have only used the described concept on Qlik Sense, it should work equally well with QlikView.

Don’t care for reading? This video outlines the concept, otherwise keep reading below.

Continue reading “Oh data where art thou?”