Butler for Qlik Sense version 2.0

Leave a reply

Butler 2.0 is out!

Champagne

A summer project is over, and Butler  has been released in a version that is a total rewrite of Butler version 1.x. Butler 2.0 includes features such as

  • Real-time monitoring of what users are active on a Sense Enterprise system
  • Real-time monitoring of failed reload tasks, with notifications sent as emails, to Slack channels and as MQTT messages
  • Starting Sense reload tasks from the load script or from external systems, using a REST API
  • Posting messages to Slack from the reload script. Great for sysadmin notifications as well as notifying end users that new data is available in an app
  • Sending MQTT pub-sub messages from the load script
  • Get full metadata, including load script, for any app
  • Getting free disk info for the Sense server
  • …and more

All in all, Butler 2.0 includes a set of features that offers greatly enhanced features for anyone involved in the operation of a Sense Enterprise environment.

Butler is available via Qlik Branch, with all source code hosted on GitHub.
Documentation is available on github.io. That documentation is the best starting point for learning more about what Butler does and how to use it.

Some sample screen shots shows what is possible to achieve:

Continue reading →

Qlik Sense data model of the year

6 Replies

Qlik-Sense-Logo1Long overdue, but during last few months I have finally started to play around with the various APIs available on the Qlik Sense platform.

It has been said before, but it is worth repeating: those APIs are really, really powerful. They are one of the main reasons why I think Qlik have some good years ahead of them.

Anyway – a recent experiment involved serialising apps to JSON text files, so they can be stored in Git in an effective way. Works really well and is surprisingly simple to achieve thanks to the groundwork done by members of the Qlik developer community.
So far so good, but the JSON blob describing a Sense app is extremely large and hard to navigate. Continue reading →

Qlik Sense REST connector log file overload

Leave a reply

Qlik-Sense-Logo1I recently investigated a strange behaviour of the Qlik Sense REST connector, where it would become slower and slower executing REST queries, the longer the server had been running. RAM or CPU consumption was not unusually high, but something was clearly eating away some kind of resources from the system. Turned out it was disk access that was the bottleneck.

Continue reading →

Butler for Qlik Sense – connecting Sense to the world

Leave a reply

QlikLogoThe goal of the previously described Slack proxy project was to allow posting of messages from Qlik Sense load scripts to Slack (great instant messaging platform!). The project has however developed into something larger and slightly more ambitious, integrating other connections than Slack. The original project name “Slack proxy” has become less and less relevant, and the project has thus been renamed to “Butler”.

As of right now (May 2016), Butler’s feature set include

  • Send messages from Sense load script to Slack
  • Send/publish MQTT messages from Sense load script (i.e. outbound MQTT).
  • Sense reload failures as emails, and to Slack.
  • Sense audit events (session start/stop and connection open/close) to Slack and to MQTT messages.
  • Create new directories on the Sense server’s (where Butler is running) disks.
  • Get disk space info, for disks on the Sense server where Butler is running.

Butler version 1.1 was just released, please refer to the Butler GitHub repository for further information. Feel free to fork it and contribute if you feel some feature is missing –  the node.js app should be pretty easy to understand and extend upon.

ESP8266 Over The Air updating – what are the options?

Leave a reply

1950's TV link dish - a bit overkill for ESP8266, maybe..Over the air convenience

A real pain point when developing sensor nodes that are scattered around a building (or a country or the world!) is the updating part. How do you get new firmware onto the devices?

The cell phone manufacturers were early to investigate the options, but from my days in that industry I know first-hand how challenging it was to send new firmware to tens or hundreds of thousands of devices. Lot’s have happened since, today a system update for tens of millions of Android or IOS devices is no big deal, you just make sure your phone is plugged into power before starting the upgrade. Leave it overnight, and next morning you have the latest (and hopefully greatest) version installed.

Looking specifically at the ESP8266, it does have drawbacks in terms of power consumption (it’s hard to get the power consumption down for a device that needs 3-4 seconds to wake up from sleep, connecting to a wifi network and send its sensor readings – but improvements are possible). On the other hand, its wifi connectivity means Over The Air (OTA) updating is possible.

Having a bunch of ESP8266’s lying around, I looked into the options for doing OTA on ESP8266.

Continue reading →

Microscope software for OSX

Leave a reply

andonstar USB microscopeGot myself an Andonstar 2 megapixel USB microscope about a year ago, mainly just for testing it out, and having it readily available when some repair project demanded it. The microscope is actually pretty nice given its ca USD 60 price point, with decent picture quality, a sturdy stand (even though it’s somewhat of a pain adjusting it) and a built-in adjustable light source.

To be honest it hasn’t been used much (it did come in handy during the repair of the BK Precision LCR meter though).. but this is in part due to the lack of good OS X software for use with generic USB microscopes. Over the past year I have tried some video software, and still have a few on the try-these-out-list.

Continue reading →

URL encoding in Qlik Sense and QlikView

Leave a reply

URLWith so much cool data available online, tools like QlikView and Sense becomes real Swiss Army knives of data integration.

Pull in some data from company internal databases, some data from previously created QVDs, and more and more commonly also from various online sources, both public and private. Depending on how you call those online APIs you might get away with just sending in query parameters as they are, but in other cases – and this is especially true if you need to send more complex text strings to the API – you need to URL encode the query parameters.

Over the years I have run into this numerous times, but the other weekend I realised it’s actually very easy. Just create a mapping table using an online source for the utf8-to-URL encoded mapping, then use MapSubstring to convert each character in the URL parameter to its hex counterpart.

Code for this found over at GitHub, it’s just 10 or so lines of code.

Occam’s razor holds true again – the easy solutions prevail and are usually preferred!