60 second trick saves hours debugging Qlik Sense apps

During the past weeks I have developed what has turned into a fairly complex Qlik Sense solution, where reloading the main application takes on the order of 10 minutes.

I know – Qlik apps should be kept small, with intermediate data sets stored as QVDs. But sometimes that’s just not practical.

Qlik apps works the same as most apps – if you made a typo somewhere the reload (or compilation for regular computer applications) will fail. Sometimes after a few seconds, more often right towards the end, just before those 10 minutes have passed… Fix the typo and start another 10-minute reload. Sigh.

The other day I realised that I had been using an old trick without even thinking about it, to cut down the reload times from 10 minutes to a few seconds. Talking to some colleagues, it turned out they didn’t use this trick – so here it is: The best time saver ever for Qlik developers.

The idea is very simple: You want to run the entire load script to make sure it works, but without processing 100s of megabytes of data or more.

In Qlik scripting, this is achieved by using the First prefix, added before a Load statement. Two things needed:

  1. Add a variable at the beginning of the script, and make it easy to change it to a few different values (for loading 1000 rows of source datasets, or something larger).
  2. Add the variable before all load statements where large datasets are loaded into the app.

It can look something like this:

Sense’s load script editor does not quite like using variables in this way, and complains about the syntax being incorrect. Don’t worry about that, the script loads fine.

To do a quick load (to ensure the script is solid from a syntax perspective), just enable one of the commented out lines at the top. This will limit the numer of lines loaded in the following load(s). The load time will be nice and short.

Once the script works, comment out that first variable again, setting it to an empty string. All data will be loaded from sources.

Easy trick – I know. But it has literally saved me days during past months!

4 Replies to “60 second trick saves hours debugging Qlik Sense apps”

  1. Nice tip Göran, thank you for sharing.

    The thing you need to be careful of is having tables which don’t have any related values in, due to the first n from one table not appearing in the first n of another. This can be avoided by adding WHERE EXISTS to all tables bar the first. If all tables are optimised QVD loads this can still be super quick.

    At one site where they had loads which took hours I created a subsequent QVD layer, where a query was put on the first table and then all other QVD loads used WHERE EXISTS, pushing the resultant QVDs into a new folder. The front end app could then run over the main QVD folder or any one of the subsets created, just by changing the path. This was good for testing, but also for pushing out smaller apps for people that only needed to see a subset.

    All kind of obvious, but as you say, not necessarily to everyone.

    1. Very true, Steve.

      It usually comes down to the same old question:
      Is it worth spending time setting up a good dev/test workflow, in order to potentially saving time in the future…

  2. Does not using the debug process and restricting the number of lines loaded do the same thing? No extra code is required. Am I missing something?

    1. You are quite right in that you can do limited loads using the debugger.

      There are however several reasons I rarely use the debugger:

      • It’s an all-or-nothing approach. I usually only want to restrict number of loaded lines on a few fact tables, doing full loads on support and dimensional tables. The debugger does not support this scenario.
      • The incremental stepping through the script that the debugger does is… extremely annoying, IMHO. Gives me a headache. AND it takes forever for a long script (think 2000+ lines spread across 25 .qvs files).
      • For those of us used to “proper” debuggers and IDEs (think Node.js, C/C++, Python etc), the Sense debugger is extremely limited, and I usually achieve the needed debugging/variable inspection/… by means of trace messages, sending progress messages to a Teams/Slack channel etc.

Leave a Reply

Your email address will not be published.