For some time I have been thinking about how to improve the Sense development process as a whole. There is a lot of gathered experience and best practices from the wider software development community, but how can we apply this to Qlik Sense development?
I think more can be done though.
Looking at the concepts promoted in DevOps, it struck me that Sense development follows about the same phases as those in DevOps. Combining Sense and DevOps of course gives us….
The more I looked at it, the more I felt “wow – SenseOps really rocks!”
In a large Qlik Sense installations you are likely faced with a high number of reloads every hour.
But some apps are more important than others – those apps should reload immediately when triggered.
The alternative is that those important apps end up being queued after ongoing long running tasks – not good.
In the current Sense version (3.1.5) it is not possible to set priorities for app reload tasks. But we can achieve the same thing using security rules – let’s take a look at how.
I value this quite high. Being part of the 2016 Luminary program too, I know how much value it brought both in terms of great networking opportunities with fellow Luminaries (and others), as well as more and closer contacts with Qlik themselves.
The one thing that makes Sense stand out (IMHO) is the fact that it is a very solid platform, on top of which you can build all kinds of interesting apps.
Qlik’s own user-facing standard Sense client is great, but there are for sure times when you want something else. The APIs very nicely enable that kind of development.
On the other hand, with so much great work happening in the open source field (around back-end technologies, visualisations etc), it is also extremely promising to see Qlik open sourcing some of their tech.
2017 has indeed started in an interesting way for Sense, can’t wait what’s in store for the rest of the year.
For a long time I wanted to learn more about client side development.
Javascipt, CSS, reactive frameworks etc – all those concepts that make today’s web sites tick. I really think backend developers can benefit from having at least a basic understanding of how the front-end works – this will make the finished app better as a whole.
I worked on a small node.js server during the past couple of weeks. The idea came from the fact that
I over and over again find myself creating Sense apps that are almost identical, and
people starting out as Sense developers spend too much time learning the basics. I wanted to bootstrap their learning process by providing well written skeleton apps for them.
This could of course be achieved by just duplicating apps using the QMC, but it would be way better if there was a nice little web app that listed the available Sense app templates, and allowed anyone (with permissions to create apps) to create new apps based on them. Or maybe this feature could even be integrated into one of the different Sense hubs that are now available…
One of the recent versions (3.0? 3.1? 3.1.1?) of Sense added much improved syntax color coding of the load scripts, as well as autocompletion of function names, field names, variables etc.
What might seem like a small change is at least to me super, super useful. I can suddenly keep my hands on the keyboard, and don’t have to switch to another browser tab to check in the app data model to check the exact name of some field – I just type a couple of characters and a list of suggestions pop up. Productivity is up and frustration down – that is a very good combo.
Real-time monitoring of failed reload tasks, with notifications sent as emails, to Slack channels and as MQTT messages
Starting Sense reload tasks from the load script or from external systems, using a REST API
Posting messages to Slack from the reload script. Great for sysadmin notifications as well as notifying end users that new data is available in an app
Sending MQTT pub-sub messages from the load script
Get full metadata, including load script, for any app
Getting free disk info for the Sense server
All in all, Butler 2.0 includes a set of features that offers greatly enhanced features for anyone involved in the operation of a Sense Enterprise environment.
Butler is available via Qlik Branch, with all source code hosted on GitHub. Documentation is available on github.io. That documentation is the best starting point for learning more about what Butler does and how to use it.
Some sample screen shots shows what is possible to achieve:
I recently investigated a strange behaviour of the Qlik Sense REST connector, where it would become slower and slower executing REST queries, the longer the server had been running. RAM or CPU consumption was not unusually high, but something was clearly eating away some kind of resources from the system. Turned out it was disk access that was the bottleneck.