Black box monitoring of Butler tools – monitoring the monitor

This is part 3 in a series about how healthchecks.io can be used to solve various kinds of monitoring scenarios for Qlik Sense Enterprise on Windows (QSEoW) environments.

Previous articles:

Part 1: Oh data where art thou?
Part 2: Black box monitoring of Windows servers

Tl;dr

Most of the Butler family of DevOps tools for Qlik Sense can now be monitored using healthchecks.io or similar tool.

If one of the supported Butler tools go down, an alarm is raised and one or more alerts are sent via email, Slack, Teams, PagerDuty or other supported channels.

This means that both infrastructure (servers etc), applications/services and data sources can be monitored using the same tool and on the same dashboards.

Health dashboard at Ptarmigan Labs, covering both servers, applications and data.
Continue reading “Black box monitoring of Butler tools – monitoring the monitor”

Butler SOS 5.4 is out: good looks and app details

Butler SOS provides real-time insights into Qlik Sense Enterprise on Windows environments.

Time for another update of Butler SOS, this time to version 5.4.

Github release : https://github.com/ptarmiganlabs/butler-sos/releases
Docker image   : https://hub.docker.com/r/ptarmiganlabs/butler-sos
Documentation  : butler-sos.ptarmiganlabs.com

This release both adds some nice new features as well as enhancing existing ones and fixing some bugs. Let’s take a look at the highlights.

  • Track in detail what apps are loaded into each Sense server.
  • Regular apps and session apps are handled separately, making app metrics easier to understand and more relevant
  • Sample dashboards are now built using the brand new, shiny and all together awesome Grafana 7. Did I mention that Grafana 7 is awesome? Awesome.
  • Ever wondered how long Butler SOS has been running or how much memory it uses? The new uptime messages have you covered.
  • You are properly impressed with the uptime messages – good. Why not store them to Influxdb, so you can also visualize Butler SOS’ own memory use? It’s just a couple of changes in the config file away.
  • Better control over what features are enabled. Don’t need Docker health checks? Now you can turn that (and other) feature off.
  • Ah, you are a serious Sense user and have separate DEV and PROD environments? Good – now Butler SOS supports multiple instances running on a single server.
  • Who will monitor the monitor? Butler SOS can now send heartbeats to customisable URLs at desired intervals. Perfect if you want to monitor Butler SOS using for example healthchecks.io. Very, very cool actually.
  • Bugs, bugs and bugs. The known ones have been fixed. Keep reporting new ones!
  • Update all dependencies to latest versions, to ensure security concerns are adressed.

Curious what it looks like in practice?
Seeing is believing:

If you don’t measure it, you can’t improve it: DevOps concepts meets Qlik Sense

Real-time monitoring of user sessions using Butler SOS

Butler SOS 4.0 is out, adding features that make it easier than ever to monitor large Qlik Sense environments. We’ll return to this topic of course, but let’s first take a few steps back.

There are many variants of that quote: “If you can’t measure it, you can’t improve it”, “Measure what matters”, “Measure what is measurable, and make measurable what is not so.” and others. The last one supposedly originates with Galileo Galilei. Smart guy.

The development of Butler SOS continues in that spirit. Qlik Sense provides an awesome platform on top of which all kinds of data analysis, visualisation and presentation solutions can be built. A key word there is platform. Sense does not offer solutions to all software development challenges, nor should it. Instead, use the tools and best practices that millions of developers around the world have refined over the years.

Qlik Sense does on the other hand offer a very comprehensive set of APIs that give developers access to its internals – and this is part of why it’s such a powerful platform. Butler SOS taps into some of these APIs, exposing their data in the form of real-time dashboards, charts and alerts. Suddenly sysadmins know can get both an overview of how all servers are doing, as well as look at detailed server metrics when so needed.
All made possible using the Sense APIs, but in general powered by various open source tools.

We’re basically back to Galileo – let’s make sure the important parts of Qlik Sense are measurable. It is then possible to improve the parts that don’t work well.

Continue reading “If you don’t measure it, you can’t improve it: DevOps concepts meets Qlik Sense”

Monitoring for Qlik Sense: Butler SOS v3 is out

The latest version fo Butler SOS is out, taking the version number to 3.0.
A lot of the code has been fine tuned to better meet the needs of enterprise grade Qlik Sense deployments.

Docker (or some compatible container platform) is now the preferred and recommended way of running Butler SOS. Butler SOS has been developed and tested on Linux and Mac OS, but should in theory run also other Docker enabled platforms.

New features

Version 3 adds a few – but useful – features:

  • Per-server config option “serverGroup”. Use this to group or categorize servers, for example as being part of a production vs development Qlik Sense cluster. This enables the creation of Grafana dashboards that use Grafana variables to automatically show metrics for all PRODUCTION servers. This greatly simplifies using Butler SOS in large Qlik Sense environments, where servers are frequently added/removed. No need to manually update the Grafana dashboards any more.
  • Config option “queryPeriod” for controlling how far back querying for Sense log entries should be done. Used together with the logdb.pollingInterval setting, it is now possible to fine tune how often the Qlik Sense log database is queried for errors and warnings.
Continue reading “Monitoring for Qlik Sense: Butler SOS v3 is out”

Let there be (blinky) light!

I was recently in Helsinki, giving a talk at a QlikDevGroup event. Great event, great crowd. The topic was SenseOps and Butler SOS, and I showcased the lamp above as an example of a funky, but still relevant way to monitor user activity in a Qlik Sense Enterprise environment.

A person in the audience asked how the map works. I claimed it was super simple, costing less than USD 10 to build (assuming you already have a suitable enclosure) and uses just four wires hooked up between some pre-made modules. Time to prove it.

The four wires part might have been a slight exaggeration… but it depends on which wires you count – right?

Continue reading “Let there be (blinky) light!”