Posted on

Butler SOS v2.0 – Real-time error and warning dashboards for Qlik Sense

This one is long overdue, but finally here: Butler SOS v2.0

The new version is an almost complete re-write of v1.0. Changes are plentiful and include

  • All warnings and errors stored by Sense in its log database are now pulled into Butler SOS, from where it can be graphed and acted upon.
    This is a big deal, as it was previously not possible to get notifications or alerts when errors or warnings started to pile up in the logs.
  • Operational health metrics are still pulled from Qlik Sense, but this is now done directly from the QIX engine rather than via a hard-to-secure virtual proxy.
  • Using certificates for authentication with Sense removes potential security issues with v1.0.
  • Config file is now YAML instead of JSON. More human readable and with inline comments.
  • Config file now allows for more fine-grained control of Butler SOS.
  • Several bugs fixed, especially around sending metrics to MQTT.

The readme file on GitHub has all the details, here are some screen shots to get you started though:

 

The new version is available on GitHub.

 

PS. This is blog post # 100. Yaaay!!!

Posted on

Monitoring and auto-starting Node.js services on Windows Server

When relying on various Node.js services (e.g. Butler SOS, Butler, App Duplicator etc), you quickly run into the challenge to ensure all services are always up and running.
A failing service might be fine, as long as it is quickly restarted in a predictable way.

A concrete example could be Qlik Sense or QlikView apps that send status messages to Slack during the execution of their reload scripts. Those messages will fail if Butler is for some reason not running.

This leads us to the conclusion that the services must automatically be

a) started when a server is rebooted, and
b) restarted if they for some reason terminate/die.

Enter process monitors.

At their core, process monitors ensure that the desired processes are always running, i.e. bullet b) above. Some process monitors also offer additional features such as zero-downtime restart of services, memory and performance profiling of the monitored services, being able to monitor different kinds of processes (not only Node.js ditto).

Adding to the pain is the fact that Sense and QlikView runs on Windows servers, meaning that all those great tools available on Linux cannot be used.

Looking at Node.js specifially, I have found two process monitors to work well on Windows: Forever and PM2.

Continue reading “Monitoring and auto-starting Node.js services on Windows Server”

Posted on

Qlik Sense app duplicator 2.0 released

On the surface everything looks the same.

Under the hood everything has changed.

The 2.0 release of the Qlik Sense app duplicator is a major update. Most code has been refactored.

The most notable changes are:

  • Switched to using Enigma.js for talking to the Sense engine, instead of the previously used Qsocks library. Enigma is open source just like Qsocks, but the former is supported and actively developed by Qlik themselves – that is really important.
  • Much improved logging.
  • A nasty bug has been fixed.
    The app duplicator would fail semi-silently when multiple app duplication requests were sent to it simultaneously. No more of that – the duplicator now handles also high-load scenarios gracefully, and have been battle-tested in hands-on sessions when many developers created their own apps from templates at the same time. Nice.The 2.0 version is available on GitHub, as always.
Posted on

Butler SOS goes main stage at Qonnections 2017

Edit:

Qlik has made available a video from Anthony Deighton’s (Qlik’s CTO) keynote session.

Very much worth looking through the entire video – it was very interesting indeed. 
His highlight of customer and partner project’s is found 12 min 36 sec into the video.

 

 

A few weeks back something cool happened.

During the big, main stage keynote session at Qlik‘s annual Qonnections conference, my Butler SOS project was highlighted as an example of a great application built on top of the Sense APIs.

I have to admit that it is pretty neat to get that kind of recognition from industry peers… a bit of ego boost :).

And it makes it easier and even more fun to work out coming, similar concepts and services. Lots more to be done around DevOps and Qlik Sense.

Speaking of which:
I started playing around with Qlik’s new Enigma.js library a week or so ago, to see if it could be used in coming projects.

Very nice indeed. Nothing wrong with good old Qsocks, but Enigma is a step in the right direction. Just like qsocks, Enigma is open source – but also fully supported by Qlik. That is a huge advantage!