When relying on various Node.js services (e.g. Butler SOS, Butler, App Duplicator etc), you quickly run into the challenge to ensure all services are always up and running.
A failing service might be fine, as long as it is quickly restarted in a predictable way.
A concrete example could be Qlik Sense or QlikView apps that send status messages to Slack during the execution of their reload scripts. Those messages will fail if Butler is for some reason not running.
This leads us to the conclusion that the services must automatically be
a) started when a server is rebooted, and
b) restarted if they for some reason terminate/die.
Enter process monitors.
At their core, process monitors ensure that the desired processes are always running, i.e. bullet b) above. Some process monitors also offer additional features such as zero-downtime restart of services, memory and performance profiling of the monitored services, being able to monitor different kinds of processes (not only Node.js ditto).
Adding to the pain is the fact that Sense and QlikView runs on Windows servers, meaning that all those great tools available on Linux cannot be used.
Looking at Node.js specifially, I have found two process monitors to work well on Windows: Forever and PM2.
Continue reading “Monitoring and auto-starting Node.js services on Windows Server”
On the surface everything looks the same.
Under the hood everything has changed.
The 2.0 release of the Qlik Sense app duplicator is a major update. Most code has been refactored.
The most notable changes are:
- Switched to using Enigma.js for talking to the Sense engine, instead of the previously used Qsocks library. Enigma is open source just like Qsocks, but the former is supported and actively developed by Qlik themselves – that is really important.
- Much improved logging.
- A nasty bug has been fixed.
The app duplicator would fail semi-silently when multiple app duplication requests were sent to it simultaneously. No more of that – the duplicator now handles also high-load scenarios gracefully, and have been battle-tested in hands-on sessions when many developers created their own apps from templates at the same time. Nice.The 2.0 version is available on GitHub, as always.
For some time I have been thinking about how to improve the Sense development process as a whole. There is a lot of gathered experience and best practices from the wider software development community, but how can we apply this to Qlik Sense development?
Some initiatives are starting to appear. Qlik themselves are open sourcing microservices such as the Governed Metrics Service , as well as open sourcing key libraries that make it easier to use Sense as a general purpose analytics platform. I have a couple of contributions in this space too, with the Butler and App duplicator microservices.
I think more can be done though.
Looking at the concepts promoted in DevOps, it struck me that Sense development follows about the same phases as those in DevOps. Combining Sense and DevOps of course gives us….
The more I looked at it, the more I felt “wow – SenseOps really rocks!”
Thus, please meet a new site: senseops.rocks
Head over for a look, and feel free to feedback in the discussion forums there.
For a long time I wanted to learn more about client side development.
Javascipt, CSS, reactive frameworks etc – all those concepts that make today’s web sites tick. I really think backend developers can benefit from having at least a basic understanding of how the front-end works – this will make the finished app better as a whole.
Couple this with a desire to learn Vue.js specifically, and we get a user interface for the Qlik Sense app duplicator service.
Continue reading “Qlik Sense app duplicator 1.2: User interface included”
A quick note before the weekend:
Version 1.1.0 of the app duplicator for Qlik Sense was just release on GitHub, feel free to check it out.
Main features of the new version are
- https is now mandatory
- Logging all requests to files on disk
- Improved stability and better handling of malformed requests