Running things on Apple hardware here, all those Windows-only software packages are a somewhat of a pain… As good as they are, both Qlik Sense, some of the vendor specific embedded software IDEs and other need-to-have software are Windows-only.
VirtualBox to the rescue. It is free and works really well (I do hear good things about Parallels too, though). I have used it for years without any real issues, until recently when I simply ran out of disk space on the host iMac where it lives, as well as in the virtual machine itself. The client OS (Windows 10) simply did not have enough space to download and install service packs.
Time to upgrade.
Such a simple thing – but oh how useful.
One of the recent versions (3.0? 3.1? 3.1.1?) of Sense added much improved syntax color coding of the load scripts, as well as autocompletion of function names, field names, variables etc.
What might seem like a small change is at least to me super, super useful. I can suddenly keep my hands on the keyboard, and don’t have to switch to another browser tab to check in the app data model to check the exact name of some field – I just type a couple of characters and a list of suggestions pop up. Productivity is up and frustration down – that is a very good combo.
Butler 2.0 is out!
A summer project is over, and Butler has been released in a version that is a total rewrite of Butler version 1.x. Butler 2.0 includes features such as
- Real-time monitoring of what users are active on a Sense Enterprise system
- Real-time monitoring of failed reload tasks, with notifications sent as emails, to Slack channels and as MQTT messages
- Starting Sense reload tasks from the load script or from external systems, using a REST API
- Posting messages to Slack from the reload script. Great for sysadmin notifications as well as notifying end users that new data is available in an app
- Sending MQTT pub-sub messages from the load script
- Get full metadata, including load script, for any app
- Getting free disk info for the Sense server
- …and more
All in all, Butler 2.0 includes a set of features that offers greatly enhanced features for anyone involved in the operation of a Sense Enterprise environment.
Butler is available via Qlik Branch, with all source code hosted on GitHub.
Documentation is available on github.io. That documentation is the best starting point for learning more about what Butler does and how to use it.
Some sample screen shots shows what is possible to achieve:
Long overdue, but during last few months I have finally started to play around with the various APIs available on the Qlik Sense platform.
It has been said before, but it is worth repeating: those APIs are really, really powerful. They are one of the main reasons why I think Qlik have some good years ahead of them.
Anyway – a recent experiment involved serialising apps to JSON text files, so they can be stored in Git in an effective way. Works really well and is surprisingly simple to achieve thanks to the groundwork done by members of the Qlik developer community.
So far so good, but the JSON blob describing a Sense app is extremely large and hard to navigate. Continue reading
I recently investigated a strange behaviour of the Qlik Sense REST connector, where it would become slower and slower executing REST queries, the longer the server had been running. RAM or CPU consumption was not unusually high, but something was clearly eating away some kind of resources from the system. Turned out it was disk access that was the bottleneck.
The goal of the previously described Slack proxy project was to allow posting of messages from Qlik Sense load scripts to Slack (great instant messaging platform!). The project has however developed into something larger and slightly more ambitious, integrating other connections than Slack. The original project name “Slack proxy” has become less and less relevant, and the project has thus been renamed to “Butler”.
As of right now (May 2016), Butler’s feature set include
- Send messages from Sense load script to Slack
- Send/publish MQTT messages from Sense load script (i.e. outbound MQTT).
- Sense reload failures as emails, and to Slack.
- Sense audit events (session start/stop and connection open/close) to Slack and to MQTT messages.
- Create new directories on the Sense server’s (where Butler is running) disks.
- Get disk space info, for disks on the Sense server where Butler is running.
Butler version 1.1 was just released, please refer to the Butler GitHub repository for further information. Feel free to fork it and contribute if you feel some feature is missing – the node.js app should be pretty easy to understand and extend upon.
Over the air convenience
A real pain point when developing sensor nodes that are scattered around a building (or a country or the world!) is the updating part. How do you get new firmware onto the devices?
The cell phone manufacturers were early to investigate the options, but from my days in that industry I know first-hand how challenging it was to send new firmware to tens or hundreds of thousands of devices. Lot’s have happened since, today a system update for tens of millions of Android or IOS devices is no big deal, you just make sure your phone is plugged into power before starting the upgrade. Leave it overnight, and next morning you have the latest (and hopefully greatest) version installed.
Looking specifically at the ESP8266, it does have drawbacks in terms of power consumption (it’s hard to get the power consumption down for a device that needs 3-4 seconds to wake up from sleep, connecting to a wifi network and send its sensor readings – but improvements are possible). On the other hand, its wifi connectivity means Over The Air (OTA) updating is possible.
Having a bunch of ESP8266’s lying around, I looked into the options for doing OTA on ESP8266.