Ctrl-Q NR: Qlik Sense nodes for Node-RED

A few weeks ago we had a teaser blog post here, showing off a couple of Youtube videos (here and here) where Node-RED was used to do app reloads (and more) in Qlik Sense.

The tool enabling those videos is called "Ctrl-Q NR".

It's essentially a Node-RED module that contains a set of nodes that can be used to interact with both Qlik Sense Cloud and client-managed Qlik Sense from within Node-RED.
A low-code tool for both Qlik Sense products, if you like.

Node-RED? Low-code?
What's the big deal here?

Good questions.

Node-RED is an open source, very mature and powerful low-code platform that lets you integrate tools and datasources using a drag-and-drop concept.

I.e. what's usually called "low-code" - you don't need to write code to get things done.
On the other hand, if you want or need to enhance the drag-and-drop logic with your own custom code, that is very possible and easy to do too.

Before we kick off, a small note about feedback on Ctrl-Q NR.
Writing comments on this site works great, but there's also the GitHub repository which is where bug reports, feature suggestions etc are collected and discussed.

Let's start small: version 0.1.0

First: what's launching today is version 0.1.0 of Ctrl-Q NR.

It is the first public version and as such it is very much early days.
There will be bugs and the number of nodes (and thus Sense features supported) is limited.

But still - what is available today can still be very useful if you want to prototype some idea, create a proof-of-concept of something discussed in that meeting last week, or maybe create a temporary integration between client-managed Sense and Sense cloud during a migration from the former to the latter.

Need a node handling webhooks in Sense Cloud? Easy, can be developed in a few hours. A bookmark node for client-managed Sense? Done.

Who is Ctrl-Q NR for?

You probably don't want to run your company's most mission critical tool in Node-RED.
But you probably wouldn't do that in Qlik Cloud Automation either, right?
Or in Sense in general, most likely.
At least that's how most companies tend to reason.

On the other hand, Node-RED has evolved a lot over the years and proven to be incredibly stable. I personally manage Node-RED instances that have been running 24/7 for years without any problems, gathering various IoT data and storing that data in time-series databases. Not a single failure (that I am aware of) across tens of millions of events.

Getting started

It's pretty easy to get started, it's mostly about getting Node-RED running.

The Node-RED site has excellent instructions for setting up Node-RED on all major platforms, including Docker.
The short version is that you first install Node.js, then follow instructions on the Node-RED site for installing Node-RED.

The Node-RED forum is also very helpful if things do not work as expected.

Installing Ctrl-Q NR

Once Node-RED is running you install Ctrl-Q NR just like any other of the 4500+ modules that are available:

1. Click on the menu icon in the top-right corner
2. Click on "Manage palette"
3. Click on the "Install" tab
4. Search for "ctrl-q-nr"
5. Click "Install"

Wait a bit and you should see a set of new nodes in your palette to the left:

Set up connections to Sense

Node-RED uses a concept of "configuration nodes" to deal with cases where you connect repeatedly to the same system.

Ctrl-Q NR uses such nodes to represent client-managed Sense servers as well as Qlik Sense Cloud tenants. It is possible to add many different sets of settings, useful if you have multiple tenants or servers.

Cloud

1. Drop any Ctrl-Q NR node related to Qlik Sense Cloud on the canvas.
   Double click it to open its settings.

2. Click the edit button marked in the image above. This lets you add a new (or edit existing) configuration node.

3. You have two options for authenticating with Qlik Sense Cloud: "OAuth2 m2m" or "API key". Both are well documented on qlik.dev.
   Some mean that API keys are easier to set up and get started with, but OAuth2 might provide better/more granular security.
   Use the method that makes sense for you.

Client-managed

1. Drop any Ctrl-Q NR node related to client-managed Sense on the canvas. Double click it to open its settings.

2. Click the edit button marked in the image above. This lets you add a new (or edit existing) configuration node.

3. The various settings are probably familiar if you have worked with client-managed Sense before. The certificate authority file (root.pem) may or may not be needed, depending on how your computer and Sense server is set up.
   
   The certificates are exported from the QMC, as per Qlik's instructions.
   
   Fill in the fields with settings for your Sense environment, then click "Add", then click "Done" in the node settings dialog. You're now back to the main canvas.

Flow 1: Get app metadata

Let's start easy, getting app metadata from both a client-managed and Qlik Sense Cloud:

The input node is empty, as is the app nodes. The "Read" operation is chosen for both app nodes.
According to the app nodes' documentation, not specifying any app IDs will retrieve info for all apps.

You can build the flow above yourself or copy-paste from below (which uses a dummy tenant/server).

[{"id":"9fcf3c7d452da989","type":"inject","z":"1aac7bbf867938f8","name":"Get all apps","props":[],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","x":110,"y":80,"wires":[["a48ad2da6e3594ab"]]},{"id":"acbd86e3470ba80f","type":"debug","z":"1aac7bbf867938f8","name":"All apps","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","targetType":"msg","statusVal":"","statusType":"auto","x":620,"y":80,"wires":[]},{"id":"a48ad2da6e3594ab","type":"qseow-app","z":"1aac7bbf867938f8","name":"","server":"804a3e6564522ecf","op":"r","appId":"","appSource1":"predefined","appSource2":"msg-in","x":370,"y":80,"wires":[["acbd86e3470ba80f"]]},{"id":"7647c39fcd87c097","type":"comment","z":"1aac7bbf867938f8","name":"client-managed","info":"","x":100,"y":40,"wires":[]},{"id":"d3993fc64c2270c0","type":"comment","z":"1aac7bbf867938f8","name":"cloud","info":"","x":70,"y":200,"wires":[]},{"id":"a82ededd52877478","type":"qscloud-app","z":"1aac7bbf867938f8","name":"","tenant":"d616be8cb521e242","op":"r","appId":"","appSource1":"predefined","appSource2":"msg-in","x":370,"y":240,"wires":[["7d5c96a560e63226"]]},{"id":"c98801056620cce4","type":"inject","z":"1aac7bbf867938f8","name":"Get all apps","props":[],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","x":110,"y":240,"wires":[["a82ededd52877478"]]},{"id":"7d5c96a560e63226","type":"debug","z":"1aac7bbf867938f8","name":"All apps","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","targetType":"msg","statusVal":"","statusType":"auto","x":620,"y":240,"wires":[]},{"id":"804a3e6564522ecf","type":"qseow-sense-server","name":"Dummy Qlik Sense server","protocol":"https","host":"my.qlikserver.com","port":"4242","authType":"cert","certFile":"c:\\secret\\client.pem","keyFile":"c:\\secret\\client_key.pem","certCaFile":"","jwt":""},{"id":"d616be8cb521e242","type":"qscloud-tenant","name":"Dummy Qlik Sense Clound tenant","tenant":"abcdefgh01234567890","region":"eu","authType":"apikey","clientId":"","clientSecret":"","apiKey":"abcdefgh01234567890"}]

Each of the two sub-flows give will output an array with app objects:

Flow 2: Doing app reloads

Now let's reload some apps.

Similar to retrieving app metadata, reloads can be done in both client-managed and cloud versions of Qlik Sense.

For client-managed Sense, reload tasks can be started by specifying their task IDs. In addition to reload tasks and external program tasks are supported. The app task node will figure out itself which kind of task each task ID represents.

For cloud Sense, app IDs are used to specify which app to reload in cloud (as reload tasks don't exist in cloud in the same way as in client-managed).

Running the flows gives us the following. Note how Ctrl-Q NR figured out what kind of tasks each client-managed task ID represented.

Here is the flow for the nodes above:

[{"id":"fa7b2b1bd4fe2cd3","type":"inject","z":"57f203f9bfd86db2","name":"Start tasks","props":[],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","x":120,"y":100,"wires":[["a4c0bfb5d791f3d5"]]},{"id":"5ad3a8ae33fd98bc","type":"debug","z":"57f203f9bfd86db2","name":"Reload result","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","targetType":"msg","statusVal":"","statusType":"auto","x":530,"y":100,"wires":[]},{"id":"a238ee58bf4c2c66","type":"comment","z":"57f203f9bfd86db2","name":"client-managed Qlik Sense","info":"","x":150,"y":60,"wires":[]},{"id":"5bd08864f9f9f5bb","type":"comment","z":"57f203f9bfd86db2","name":"Qlik Sense Cloud","info":"","x":120,"y":220,"wires":[]},{"id":"493b66fe8a9f5a5b","type":"qscloud-app","z":"57f203f9bfd86db2","name":"","tenant":"87ac58bf1227411b","op":"reload","appId":"# Existing app\r\n25dd1bf9-bc47-47c7-8ed1-16e198b26096","appSource1":"predefined","appSource2":"msg-in","x":330,"y":260,"wires":[["f4cfcb7fa8a294d0"]]},{"id":"bd83546cebb3f765","type":"inject","z":"57f203f9bfd86db2","name":"Reload app","props":[],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","x":130,"y":260,"wires":[["493b66fe8a9f5a5b"]]},{"id":"a4c0bfb5d791f3d5","type":"qseow-task","z":"57f203f9bfd86db2","name":"","server":"5e42d0533190acfb","op":"start","taskId":"# Reload tasks\r\neb018d95-8c0c-4613-9081-82b3822ecca2\r\n244ca6a8-8c20-488d-aed2-504048dd4516\r\n\r\n# External program tasks\r\nd6bfc66a-393b-4eea-8ecd-1b1629c91683\r\n\r\n# Non-existing tasks\r\nd6bfc66a-393b-4eea-8ecd-1b1629c91682\r\n","taskSource1":"predefined","x":330,"y":100,"wires":[["5ad3a8ae33fd98bc"]]},{"id":"f4cfcb7fa8a294d0","type":"debug","z":"57f203f9bfd86db2","name":"Reload result","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","targetType":"msg","statusVal":"","statusType":"auto","x":530,"y":260,"wires":[]},{"id":"87ac58bf1227411b","type":"qscloud-tenant","name":"Dummy Qlik Sense Clound tenant","tenant":"abcdefgh01234567890","region":"eu","authType":"apikey","clientId":"","clientSecret":"","apiKey":"abcdefgh01234567890"},{"id":"5e42d0533190acfb","type":"qseow-sense-server","name":"Dummy Qlik Sense server","protocol":"https","host":"my.qlikserver.com","port":"4242","authType":"cert","certFile":"c:\\secret\\client.pem","keyFile":"c:\\secret\\client_key.pem","certCaFile":"","jwt":""}]

Flow 3: Manage tags

Let's wrap up with some basic work on tags for client-managed Sense.

First let's look at what tags are available on the server:

Now let's create some new tags ("startTask1", "startTask10", "startTask11"). One of the tags to be created, "startTask1", already exists on the server.

Note how there were 22 tags before the create operation and 24 after.

Finally, let's delete tags "startTask10", "startTask11" and "startTask12". The last tag does not exist on the server.

Here is the flow for all the nods above:

[{"id":"f09ab7a81780cee6","type":"inject","z":"f6fa8bcbe1c6fb64","name":"Get info about all tags","props":[],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","x":160,"y":120,"wires":[["41481c6e255590f6"]]},{"id":"33562d468dce1c06","type":"debug","z":"f6fa8bcbe1c6fb64","name":"All tags","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","targetType":"msg","statusVal":"","statusType":"auto","x":540,"y":120,"wires":[]},{"id":"41481c6e255590f6","type":"qseow-tag","z":"f6fa8bcbe1c6fb64","name":"","server":"5e42d0533190acfb","op":"r","tagName":"","tagSource":"predefined","x":370,"y":120,"wires":[["33562d468dce1c06"]]},{"id":"48efe130426faa30","type":"comment","z":"f6fa8bcbe1c6fb64","name":"Read tags (client-managed)","info":"","x":140,"y":80,"wires":[]},{"id":"c2ac8de8bd3917df","type":"inject","z":"f6fa8bcbe1c6fb64","name":"Create tags set in tag node","props":[],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","x":170,"y":260,"wires":[["30b49e8d1caee74b"]]},{"id":"a903ecf6982adebc","type":"inject","z":"f6fa8bcbe1c6fb64","name":"Create tags included in this message","props":[{"p":"payload.tagName","v":"[\"startTask10\",\"startTask1\",\"startTask11\"]","vt":"json"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","x":200,"y":340,"wires":[["103c8615a1d18f9f"]]},{"id":"103c8615a1d18f9f","type":"qseow-tag","z":"f6fa8bcbe1c6fb64","name":"","server":"5e42d0533190acfb","op":"c","tagName":"","tagSource":"msg-in","x":510,"y":340,"wires":[["4b8e5977dc8ee3b4"]]},{"id":"30b49e8d1caee74b","type":"qseow-tag","z":"f6fa8bcbe1c6fb64","name":"","server":"5e42d0533190acfb","op":"c","tagName":"# Tags\nstartTask10\n\n# Some more tags\nstartTask1\nstartTask11\n\n","tagSource":"predefined","x":510,"y":260,"wires":[["14e78736d20c3b59"]]},{"id":"14e78736d20c3b59","type":"debug","z":"f6fa8bcbe1c6fb64","name":"Result","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","targetType":"msg","statusVal":"","statusType":"auto","x":670,"y":260,"wires":[]},{"id":"4b8e5977dc8ee3b4","type":"debug","z":"f6fa8bcbe1c6fb64","name":"Result","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","targetType":"msg","statusVal":"","statusType":"auto","x":670,"y":340,"wires":[]},{"id":"ad46e3cc6d8a68a8","type":"comment","z":"f6fa8bcbe1c6fb64","name":"Create tags (client-managed)","info":"","x":140,"y":220,"wires":[]},{"id":"874e528cbfeed8de","type":"inject","z":"f6fa8bcbe1c6fb64","name":"Create tags included in this message (empty)","props":[],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","x":230,"y":380,"wires":[["103c8615a1d18f9f"]]},{"id":"04aab8333ae2ba6b","type":"inject","z":"f6fa8bcbe1c6fb64","name":"Delete tags set in tag node","props":[],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","x":170,"y":520,"wires":[["69a3f3233ec7462b"]]},{"id":"69a3f3233ec7462b","type":"qseow-tag","z":"f6fa8bcbe1c6fb64","name":"","server":"5e42d0533190acfb","op":"d","tagName":"# Tags\nstartTask10\n\n# Some more tags\nstartTask11\nstartTask12\n","tagSource":"predefined","x":450,"y":520,"wires":[["e0c20553443708ca"]]},{"id":"e0c20553443708ca","type":"debug","z":"f6fa8bcbe1c6fb64","name":"Result","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","targetType":"msg","statusVal":"","statusType":"auto","x":670,"y":520,"wires":[]},{"id":"c92e04a9e6cee6f1","type":"comment","z":"f6fa8bcbe1c6fb64","name":"Delete tags","info":"","x":90,"y":480,"wires":[]},{"id":"5e42d0533190acfb","type":"qseow-sense-server","name":"Dummy Qlik Sense server","protocol":"https","host":"my.qlikserver.com","port":"4242","authType":"cert","certFile":"c:\\secret\\client.pem","keyFile":"c:\\secret\\client_key.pem","certCaFile":"","jwt":""}]

What's next?

A few blog posts showing what can be done using the currently available nodes will following during coming days.

Then probably a few more nodes, exactly which ones is TBD.
Feel free to pitch your suggestions and ideas, or fork the Github repo and contribute with your own nodes!