Butler Spyglass: Data lineage and metadata tool for Qlik Sense

For years I have thought about ways to get data lineage info for all apps in a Qlik Sense Enterprise environment.

It would be super useful to know exactly what apps use a particular data source, as well as vice versa (what data sources are used by a specific app). I know there are commercial tools doing this and much more, but I wanted something easy to use, yet still effective and free.

Same thing for app load scripts: Extracting and storing them to disk in human readable format has more than once save days of work, when something has gone badly wrong in an app.
Dumping load scripts to disk was possible in my original Butler tool, but then only one app at a time. So not quite what was needed in an enterprise context.

Enter Butler Spyglass.

It’s open source, it’s tested on the latest Qlik Sense Enterprise version (Feb 2019) and it was released to the public earlier today.

Run it on your server using Node.JS or as a Docker container.
The needed Docker image is available on Docker Hub. A docker-compose file is available in the Github repo, making it really easy to get started.

Data lineage info is stored in a single (potentially quite large) CSV file, which can be loaded into a Sense app and analysed there.

The load scripts are stored as individual text files (one file per app). Combine that with a cron job (or Windows scheduled task) that zips them up daily, and you get a very useful daily snapshot of all apps’ load scripts.

Documentation and examples available on Github: https://github.com/ptarmiganlabs/butler-spyglass

The version 1.0.0 release available here: https://github.com/ptarmiganlabs/butler-spyglass/releases/tag/v1.0.0