Monitoring Qlik Sense Cloud app reloads using Node-RED

Let's say you want to kick off various tasks when a Sense Cloud app finish reloading. Maybe start a few client-managed Sense reload tasks, send an email or tell some downstream system that new data is available.

Or send email/Slack/Teams/... messages when an app reload fails.

Many of these things can be done using the excellent Qlik Sense Cloud Automation product that is an integral part of Qlik Sense Cloud.

Other things - especially interaction with local or on-premise systems - are hard to do from within Qlik Sense Cloud Automation workflows.

Ctrl-Q NR on the other hand runs outside of Sense Cloud.
If there is network connectivity from the Node-RED server (where Ctrl-Q NR is installed) to a system or tool, Ctrl-Q NR can probably access and interact with it.

The reload-status node

The qscloud-reload-status node is a rather complex node that does many different things.

At its core it has logic that mirrors the reload state of each app in the Sense Cloud tenant. This concept is also known as a "state machine".

The reload states tracked by a qscloud-reload-status node can be updated in two ways:

1. Send a value of updateReloadStates in the msg.payload.operation property to the reload-status node.
2. Set a value for and then start the reload-update timer that is build into the node. Whenever the timer reaches zero the reload-status node will update itself with the latest app reload status from Sense Cloud.
   I.e. a classic polling concept.

What else can the qscloud-reload-status do?

Solution

Here is a flow that can serve as inspiration for app reload monitoring in Sense Cloud:

Some hints to get you started:

• Start by clicking "Update reload states". This will update the state machine with an initial reload state for each app in the tenant.
• The first (upper) output from qscloud-reload-status receives one message for each app reload state - no matter if the app's state has changed or not.
• The second (lower) output from qscloud-reload-status receives status messages from the node itself, for example about the state machine being updated, the reload timer being started or stopped etc.
• The lower right nodes ("Status switch for all messages" and its debug nodes) will get messages for all app states, no matter if the state has changed or not.
  The yellow switch node will fan out the messages to different debug nodes.
• The upper right debug nodes will only get messages when an app's reload state has changed. This is detected by checking the msg.payload.message property, which gets a value of reload status changed when an app's reload state changes.

The flow's JSON can be imported into Node-RED:

[{"id":"cf1637d86e7b24d9","type":"qscloud-reload-status","z":"a8d0ac72dabc9c74","name":"","tenant":"a329efb577d33158","x":530,"y":380,"wires":[["34bd4822f3e25f18","29da37b9359946d8","61a1d0801b4613c7"],["01c6e3ea91f74a85"]]},{"id":"34bd4822f3e25f18","type":"debug","z":"a8d0ac72dabc9c74","name":"Result","active":false,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","targetType":"msg","statusVal":"","statusType":"auto","x":810,"y":380,"wires":[]},{"id":"9d9d8c1ed4bb6f6f","type":"inject","z":"a8d0ac72dabc9c74","name":"Dump state machine contents to output 2","props":[{"p":"payload.operation","v":"getFullState","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","x":230,"y":420,"wires":[["cf1637d86e7b24d9"]]},{"id":"01c6e3ea91f74a85","type":"debug","z":"a8d0ac72dabc9c74","name":"Result","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","targetType":"msg","statusVal":"","statusType":"auto","x":810,"y":420,"wires":[]},{"id":"0f4d37345c041bb5","type":"inject","z":"a8d0ac72dabc9c74","name":"Start timer","props":[{"p":"payload.operation","v":"startTimer","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","x":130,"y":100,"wires":[["cf1637d86e7b24d9"]]},{"id":"9d36237d62a49243","type":"inject","z":"a8d0ac72dabc9c74","name":"Stop timer","props":[{"p":"payload.operation","v":"stopTimer","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","x":130,"y":140,"wires":[["cf1637d86e7b24d9"]]},{"id":"2e6249d00b49dc61","type":"inject","z":"a8d0ac72dabc9c74","name":"Update reload states","props":[{"p":"payload.operation","v":"updateReloadStates","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","x":160,"y":340,"wires":[["cf1637d86e7b24d9"]]},{"id":"1618e2980536a359","type":"inject","z":"a8d0ac72dabc9c74","name":"Set update interval to 10 sec","props":[{"p":"payload.operation","v":"setUpdateInterval","vt":"str"},{"p":"payload.updateInterval","v":"10000","vt":"num"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","x":190,"y":220,"wires":[["cf1637d86e7b24d9"]]},{"id":"bff05acceb0c116d","type":"inject","z":"a8d0ac72dabc9c74","name":"Set update interval to 1 min","props":[{"p":"payload.operation","v":"setUpdateInterval","vt":"str"},{"p":"payload.updateInterval","v":"60000","vt":"num"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","x":180,"y":260,"wires":[["cf1637d86e7b24d9"]]},{"id":"e3303fb757de74fd","type":"inject","z":"a8d0ac72dabc9c74","name":"Set update interval to 10 min","props":[{"p":"payload.operation","v":"setUpdateInterval","vt":"str"},{"p":"payload.updateInterval","v":"600000","vt":"num"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","x":190,"y":300,"wires":[["cf1637d86e7b24d9"]]},{"id":"4420c25ee27eaa9d","type":"inject","z":"a8d0ac72dabc9c74","name":"Get update interval","props":[{"p":"payload.operation","v":"getUpdateInterval","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","x":160,"y":180,"wires":[["cf1637d86e7b24d9"]]},{"id":"a8e06a693c9440e8","type":"switch","z":"a8d0ac72dabc9c74","name":"Status switch","property":"payload.reloadStatus","propertyType":"msg","rules":[{"t":"eq","v":"QUEUED","vt":"str"},{"t":"eq","v":"RELOADING","vt":"str"},{"t":"eq","v":"CANCELING","vt":"str"},{"t":"eq","v":"SUCCEEDED","vt":"str"},{"t":"eq","v":"FAILED","vt":"str"},{"t":"eq","v":"CANCELED","vt":"str"},{"t":"eq","v":"EXCEEDED_LIMIT","vt":"str"}],"checkall":"true","repair":false,"outputs":7,"x":590,"y":140,"wires":[["0ec46d8cd5de6534"],["31b15722ebe4754b"],["0b6f7b146542cf01"],["a64275992edcfcf0"],["8a91099a67b0b09f"],["1c3dbb28a518549a"],["b8bf7a84912b0b3f"]]},{"id":"0ec46d8cd5de6534","type":"debug","z":"a8d0ac72dabc9c74","name":"QUEUED","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","targetType":"msg","statusVal":"","statusType":"auto","x":820,"y":80,"wires":[]},{"id":"31b15722ebe4754b","type":"debug","z":"a8d0ac72dabc9c74","name":"RELOADING","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","targetType":"msg","statusVal":"","statusType":"auto","x":830,"y":120,"wires":[]},{"id":"0b6f7b146542cf01","type":"debug","z":"a8d0ac72dabc9c74","name":"CANCELING","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","targetType":"msg","statusVal":"","statusType":"auto","x":830,"y":160,"wires":[]},{"id":"a64275992edcfcf0","type":"debug","z":"a8d0ac72dabc9c74","name":"SUCCEEDED","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","targetType":"msg","statusVal":"","statusType":"auto","x":840,"y":200,"wires":[]},{"id":"8a91099a67b0b09f","type":"debug","z":"a8d0ac72dabc9c74","name":"FAILED","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","targetType":"msg","statusVal":"","statusType":"auto","x":820,"y":240,"wires":[]},{"id":"1c3dbb28a518549a","type":"debug","z":"a8d0ac72dabc9c74","name":"CANCELED","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","targetType":"msg","statusVal":"","statusType":"auto","x":830,"y":280,"wires":[]},{"id":"b8bf7a84912b0b3f","type":"debug","z":"a8d0ac72dabc9c74","name":"EXCEEDED_LIMIT","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","targetType":"msg","statusVal":"","statusType":"auto","x":850,"y":320,"wires":[]},{"id":"29da37b9359946d8","type":"switch","z":"a8d0ac72dabc9c74","name":"\"reload status changed\"","property":"payload.message","propertyType":"msg","rules":[{"t":"eq","v":"reload status changed","vt":"str"}],"checkall":"true","repair":false,"outputs":1,"x":560,"y":300,"wires":[["a8e06a693c9440e8"]]},{"id":"61a1d0801b4613c7","type":"switch","z":"a8d0ac72dabc9c74","name":"Status switch for all messages","property":"payload.reloadStatus","propertyType":"msg","rules":[{"t":"eq","v":"QUEUED","vt":"str"},{"t":"eq","v":"RELOADING","vt":"str"},{"t":"eq","v":"CANCELING","vt":"str"},{"t":"eq","v":"SUCCEEDED","vt":"str"},{"t":"eq","v":"FAILED","vt":"str"},{"t":"eq","v":"CANCELED","vt":"str"},{"t":"eq","v":"EXCEEDED_LIMIT","vt":"str"}],"checkall":"true","repair":false,"outputs":7,"x":560,"y":560,"wires":[["ba1e655e89b7b0c9"],["8827c240e3a7a045"],[],["3fbc0da08a807a4c"],["4ba7a7d597e343d1"],[],[]]},{"id":"ba1e655e89b7b0c9","type":"debug","z":"a8d0ac72dabc9c74","name":"QUEUED","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","targetType":"msg","statusVal":"","statusType":"auto","x":820,"y":500,"wires":[]},{"id":"8827c240e3a7a045","type":"debug","z":"a8d0ac72dabc9c74","name":"RELOADING","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","targetType":"msg","statusVal":"","statusType":"auto","x":830,"y":540,"wires":[]},{"id":"3fbc0da08a807a4c","type":"debug","z":"a8d0ac72dabc9c74","name":"SUCCEEDED","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","targetType":"msg","statusVal":"","statusType":"auto","x":840,"y":580,"wires":[]},{"id":"4ba7a7d597e343d1","type":"debug","z":"a8d0ac72dabc9c74","name":"FAILED","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","targetType":"msg","statusVal":"","statusType":"auto","x":820,"y":620,"wires":[]},{"id":"90d2d987d8228865","type":"comment","z":"a8d0ac72dabc9c74","name":"Monitor reloads in Qlik Sense Cloud","info":"1. Do an initial update of reload states, to get the current status of reloads.\n2. Start the timer to get periodic updates of the status of all reloads in the tenant.\n3. Start a reload that ideally runs for longer than the update interval.\n4. You will get updates on the various debug nodes when the reload changes status.\n","x":160,"y":40,"wires":[]},{"id":"f2ad9dab33b84458","type":"inject","z":"a8d0ac72dabc9c74","name":"Invalid opeation","props":[{"p":"payload.operation","v":"invalidOp","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","x":150,"y":500,"wires":[["cf1637d86e7b24d9"]]},{"id":"a329efb577d33158","type":"qscloud-tenant","name":"Dummy API key authorisation","tenant":"abcdefgh123456789","region":"eu","authType":"apikey","clientId":"","clientSecret":"","apiKey":"abcdefgh123456789abcdefgh123456789abcdefgh123456789"}]

Does it work?

Clicking "Update reload states" reads the current reload state of all apps in the tenant:

Now set the update interval to 10 seconds. This will also start the reload timer:

The app reload state of all apps will be retrieved every 10 seconds. This is probably way too frequent if you plan to use this concept continuously, but for demo purposes it works well.

Let's look at what happens when an app is reloaded from the standard Qlik Sense Cloud web interface.

The changed reload state is detected by Ctrl-Q NR.
First the app is reloading, then the state changes to SUCCEEDED.

Caveat

It should be noted that polling for app status has some drawbacks.

Let's say Ctrl-Q NR is configured to update its internal state machine every 5 minutes.
If an app reload is started 1 minute after a state machine update was done, and the app reload finished within 1 minute.

I.e. the entire app reload started and finished in between two state machine updates. You will thus miss the intermediate states such as QUEUED and RELOADING.

You will however be able to capture reload failures - simply check the FAILED debug output in the flow above.
Or connect whatever logic you want to trigger on reload failures to that output - easy!