One of the great new features in Qlik Sense is snapshots and storytelling. It allows you to create a snapshot of your visualizations, with the current selection state, and then include the snapshot in a story. Behind the scenes Qlik Sense also uses snapshot for export to PDF, PowerPoint and image. This makes it even more important for you to make your extension work well with snapshots. Sometimes this is however not easy…

The basics

To turn snapshots on, you need to include a few lines in your extension:

This will give you a context menu in the client with alternatives to create snapshot and export the visualization. That might be enough, but it might not be enough. To verify it you should:

  • create a snapshot of your extension, include it in a story and verify that it works
  • export your extension to PDF and/or PowerPoint (possibly also image)
  • and export the story to Powerpoint/PDF

You visualization is actually used in three contexts, that are somewhat different:

  1. the normal analyse mode, with live data, affected by changes in selection state
  2. the snapshot/story mode with a snapshot of data and the selection state as it was when the snapshot was created
  3. the export mode, which is also based on a snapshot, but run in a separate service on the server

To make it actually work in all, you might need to do some more work.

Make Qlik Sense wait for your extension

One problem, mainly in export, is to let Qlik Sense know when your extension actually is ready with the rendering. You might get just an empty area in your powerpoint, or just the header but no content. This might be because the image is taken before your extension is ready.

The mechanism to fix this is a javascript concept called a Promise. Your paint method should return a Promise, that resolves when the rendering is ready. Qlik Sense will then wait until the Promise is resolved and then grab the image.

This is described where briefly in Qlik Sense help under ‘Set up “finished rendering” notification, like this:

This will return a Promise, that immediately resolves. This might not be good enough, since you might want to wait for something until you resolve. In my case, I used highcarts and highcharts has a callback function that will be called when rendering is ready. The final solution is like this:

So, I create a deferred object before calling the Highcharts rendering method. I can then return the promise that belongs to the deferred, but I do not resolve it until Highcharts tells me that rendering is complete, actually I had to add an additional delay of 1 second to make sure. There is currently no documentation of qlik.Promise in Qlik Sense help, but you can look in the angularjs documentation.

Turn animations off

If it’s still not working, you should take a look at animations. Animations in analyse mode are good (if used the right way…) and might help the user discover insights in their data, but in export they should be turned off.

In Highcharts there are actually two flags you should turn off to diable animations. It might be enough to turn off just one of them, but why not both:

The isSnapshot flag which I use to determine if we are in analyse mode or in snapshot (story or export) is undocumented, so this solution might break in future Qlik Sense versions, but in September 2017 it works.

Sometimes in your extension you need to acces the QIX Engine API. You might need to make or clear selections, apply a bookmark or something else. Traditionally you would use the Capabilities APIs for this. You would load the qlik module, get hold of the app with the help of the currApp() method, and call whatever function you needed.

But there is another way. You could do it the way the built-in client does, and use enigmajs instead. Enigmajs creates a javascript wrapper around the QIX api, a wrapper that includes all methods defined in the version of QIX your system is using. Qlik actually has open-sourced enigmajs, you can read more, and download it from the Github repository.

Enigmajs features

Important features of enigmajs:

  • it includes the complete QIX API for the actual version of QIX, while the capabilities API just includes a subset
  • it only includes the QIX Engine API, while the capabilities APIs wraps some client-side functionality too
  • methods calls return a Promise, that resolves when there is a response available
  • it handles the invalidation of objects as the selection state changes so it works well from inside an extension

Unlike the Capabilities APIs, enigmajs does not come with a lot of dependencies on requirejs, angularjs and Qlik Sense client code. This makes it well suited for use in envorinments where you use other frameworks, like react or future frameworks we have not heard of yet. Of course if you’re using it inside an extension, you already have all of these dependencies, so that is not so important.

Note that Qlik flags enigmajs as experimental in the current release. I would be very surprised if it goes away any time soon, but details in the implementation might change, you need to be aware of that.

Finding enigmajs in an extension

Qlik Sense creates a server side Generic Object for each instance of your extension (for all built-in charts too, actually). Around that Generic Object will be a javascript wrapper that exposes all the methods available. In the extension you can find it at:

this.backendApi.model.enigmaModel

It will expose all the methods defined in the Generic Object QIX engine API. As you can see, there are a lot of methods. It also has an app property, that exposes QIX Engine API app object. And the app has a global property, that is a wrapper around QIX Engine Global. This gives you access to the complete QIX Engine API.

An example

I have converted my Variable extension for Qlik Sense to use enigmajs instead of the Capabilities API, as versions before 4.0 did. The key call of the extension is after the conversion like this:

function setVariableValue(ext, name, value) {
        return ext.backendApi.model.enigmaModel.app.getVariableByName(name).then(function (v) {
            return v.setStringValue(value);
        });
    }
The parameter ext is a reference to the extension itself. Since methods return a Promise, I can use the then method to run an additional command after the first command resolves. The getVariableByName reurn an enigmajs wrapper aroud a QIX Engine Variable

Conclusion

If you need access to QIX Engine API from within your extension, enigmajs can help you with that. Unlike the Capabilities API it provides wrappers for all methods. But you should be aware that it only wraps engine api, not client-side functionality. It however works nicely within the client and invalidation and refresh of objects will work as expected.

While error handling in a Qlik Sense mashup is not automatic, and the documentation is not that good, it’s really not that difficult to get it right. But there are a few things you need to know.

Why do you need it?

Your mashup might work perfectly well when you are testing it. All code is verified, all object id’s refer to objects that actually exist in the app, you have solved the problem with appid’s that differ between your development environment and production. Still there are situations where you need your error handling:

  • unauthorized users: the Qlik Sense hub makes sure to only show apps the user has access to, but for mashups there is no such mechanism, so it’s up to you to handle this situation. Qlik Sense will not allow the user to actually open the app, but he will probably be able to open the mashup, so you will need to handle the situation where he has no access to the app
  • timeout: if your users are inactive, their sessions will be closed. This means that they will not be able to continue (or restart) working in your mashup without reconnecting, most likely by doing a reload of the mashup. You need to tell the user what has happened and what they should do.

Api support

To help you with this is the setOnError method. It allows you to register a function that is called whenever an error occurs. This is both for errors that are the result of a call, like opening an app that does not exist (or the user is not authorized to access), and errors that are generated from the server, like timeouts. Obviously a good start is to display the error.

qlik.setOnError(function (error) {
        // TODO:error handling
        console.log('Qlik error', error);
        $("#errmsg").html(error.message).parent().show();
 });
Or perhaps, a angular version:
//create an array for errors
$scope.errors = [];
qlik.setOnError(function (error) {
    // TODO:error handling
    console.log('Qlik error', error);
    $scope.errors.push(error);
});
and in your template something like this:
<div ng-repeat="err in errors" class="alert">{{err.message}}</div>
Then just add a button for clearing errors ($scope.errors.length = 0 would do the trick) add some styling and you’ve got a first attempt.

Add some intelligence

But for some errors showing the error is not enough. For example if the user gets a timeout, he needs to refresh before continuing to work. If the user has no access the rest of your mashup will probably be empty (well if you show data from other sources they might be available).

To do this we need to use the error code we get with the message. As far as I know there is no official documentation of Qlik Sense error codes, but if you’ve got Qlik Sense desktop installed, you can easily find one. Just start Qlik Sense Desktop and in your browser go to:

http://localhost:4848/resources/translate/en-US/engine.js

As you might have guessed, this file contains, among other things, error code from QIX Engine. After you formatted this file, you will find something like this in it:

"ErrorCode.-128": "Internal engine error",
"ErrorCode.-1": "Unknown error",
"ErrorCode.0": "Unknown error",
"ErrorCode.1": "Some data is not correctly specified.",
"ErrorCode.2": "The resource could not be found.",
"ErrorCode.3": "Resource already exists.",
"ErrorCode.4": "Invalid path",
"ErrorCode.5": "Access is denied",
"ErrorCode.6": "The system is out of memory.",
"ErrorCode.7": "Not initialized",
"ErrorCode.8": "Invalid parameters",
"ErrorCode.9": "Some parameters are empty.",
"ErrorCode.10": "Internal error",
"ErrorCode.11": "Corrupted data",
"ErrorCode.12": "Memory inconsistency",
"ErrorCode.13": "Action was aborted unexpectedly.",
"ErrorCode.14": "Validation cannot be performed at the moment. Please try again later.",
"ErrorCode.15": "Operation aborted",
"ErrorCode.16": "Connection lost. Make sure that Qlik Sense is running properly. If your session has timed out due to inactivity, refresh to continue working.",

So there it is, error code 16 means connection lost, so if we get that we should encourage the user to refresh, perhaps display a button or something and perhaps even disable stuff that won’t work, like selections etc.
The error code from QIX engine will be in error.code in the object that is the parameter to our error funtion, so we can simply test that:

qlik.setOnError(function (error) {
     // TODO:error handling
     console.log('Qlik error', error); 
     $scope.errors.push(error);
     if(error.code === 16){
        $scope.showRefresh = true;
     }
});

Handle proxy errors

But we’re not quite through yet. When you install your mashup on Qlik Sense server, you’ll notice that not all error object have the ‘code’ field. That’s because proxy errors have another format. Instead of the code field, there is a method field. And they’re not in the engine.js file. Instead you need to look in:

http://localhost:4848/resources/translate/en-US/client.js

In it you will find the proxy errors:

"ProxyError.OnEngineWebsocketFailed": "Connection to the Qlik Sense engine failed for unspecified reasons. Refresh your browser or contact your system administrator.",
"ProxyError.OnLicenseAccessDenied": "You cannot access Qlik Sense because you have no access pass.", "ProxyError.OnLicenseAccessDeniedPendingUserSync": "Your access pass credentials are being synced. Refresh your browser or contact your system administrator.",
"ProxyError.OnNoEngineAvailable": "No available Qlik Sense engine was found. Refresh your browser or contact your system administrator.", "ProxyError.OnSessionClosed": "Your session has been closed. Refresh your browser to continue working.",
"ProxyError.OnNoDataPrepServiceAvailable": "Data Profiling service is not available.",
"ProxyError.OnDataPrepServiceWebsocketFailed": "Data Profiling service connection failed. Refresh your browser.",
"ProxyError.OnSessionTimedOut": "Your session has timed out. Log back in to Qlik Sense to continue.",
So there we are, we need to handle also OnSessionClosed and OnSessionTimedOut. This gives us something like this:
qlik.setOnError(function (error) {
     //error handling
     console.log('Qlik error', error);
     $scope.errors.push(error); 
     if(error.code === 16 || ["OnSessionTimedOut","OnSessionClosed"].indexOf(error.method)>-1){ 
        $scope.showRefresh = true;
     }
});
And that’s it, we now have error handling thats shows error messages to users and sets a flag when the user should refresh. We can then use the flag to for example display a refresh button and/or disable interactivity like selections.

One of the most popular new features in Qlik Sense is snapshots and storytelling. When you discover something interesting in your Qlik Sense app you can take a snapshot of a visualization, including its selections and data, and then build a story where you use it. And you can recreate the selections from the snapshot and drill deeper into your data or look at it from another angle.

In many cases this works automatically for your extension too, but sometimes it doesn’t. In recent versions of Qlik Sense new features, like printing and export to PDF are also built on the snapshot feature, which makes it even more important for it to work. So here is a description of how it works and what you need to do to make your extension work in snapshots.

How does snapshots work?

You create a snapshot by clicking the camera icon in the visualization context menu.  Qlik Sense will then make a copy of the data used for the rendering, known as the layout. This structure is then included in a new bookmark, that is created in Qix engine. Note that it’s not the HTML that is saved, nor is it a bitmap of the actual visualization on the screen. It’s the data.

So when the user wants to look at the snapshot, the data is loaded from the bookmark, Qlik Sense then checks what extension was used to visualize it, loads the extension, creates an object using the visualization, and calls the paint method of the visualization. But the layout parameter will not contain live data from the current selection state, it will contain data from the snapshot.

What this means for your extension

For snapshots to work your extension should use only the layout when rendering.  You should not try to use the capabilities APIs to access data dynamically. Most likely it will not work, and if you manage to get it working, the data you get will be live and not reflect the state when the snapshot was taken. So make sure you add the data you need to the properties structure instead.

Also you need to keep the layout clean. It is good practice to treat it as read-only, adding nice-to-have references make make it impossible to serialize the layout, and creating the snapshot might fail.

And don’t forget to test creating snapshots from your extensions. Also verify that printing and exporting to PDF works, very likely your users expect them to.

I have just released a new version (3.1) of the Qlik Sense variable extension at GitHub  and Qlik Branch. News in this relaese is that you can have dynamic values for your dropdown list and buttons. It is based on a contribution, but I refactored it somewhat to make it a bit more general and make buttons dynamic too.

It should be totally backward compatible with the previous release and upgrading should not cause you any problems.

Qlik Sense has several APIs and a .Net SDK to help you work with the QIX Engine. They help you working with Qlik Sense without having to bother about the details. But if you need to work on a lower level, there are some  problems you need to solve.

1. Connect over Web Socket

Qlik Sense QIX engine talks Web Socket, and only web socket. While you might need to make a https call to trigger authentication, the actual communication with the engine is web socket only.

The format of the URL can be seen in the Engine API Explorer, which you will find in Qlik Sense devhub:

ws://localhost:4848/app/c%3A%5Cusers%5Cerikw%5Cdocuments%5Cqlik%5Csense%5Capps%5Cexecutive%20dashboard.qvf?

Note that the app id is included in the URL, but what you get is an open web socket, not an open document. Actually, in Qlik Sense desktop it will work without the app id in the URL. That’s because the app id in the URL is not for QIX Engine. It’s for the proxy you will go through to reach Engine, and used for load balancing, the proxy uses it to find which engine to use for this app. This also means that you should always open a new web socket when you open a new app, since you need to give the proxy a chance to select the correct engine.

Conclusion:

  • you need to use web sockets to talk to QIX Engine
  • the web socket URL should contain the app id
  • you should always open a new socket for each app

2. Match request and response

With traditional HTTP communication you make a request, wait, and get a response. Web sockets work differently. You send a message to QIX Engine, but don’t wait for the answer. You just keep on sending messages, and eventually QIX Engine will reply. You do not know the order of the responses and actually the protocol does not tell you what respone belongs to which request.  Typically it looks something like this:

This is the initial communication when Qlik Sense standard client opens an app. If you think it is hard to read (and it is) you can open the browser developer tools and look in the network tab, the image above is from Chrome.

The darker rows are requests sent from the client to QIX Engine, while the white rows are responses. As you can se the client makes a series of requests (seven) and after that comes the respones. It is not obvious which responses belong to which call, and the respones can not be understood without knowing the request.

So how do you know which respons belongs to which request? Qlik Sense uses a standard called JSON-RPC for this. This means that in each request there is a field called id that contains a unique identifier for this request (in this case just a number 1, 2, 3 etc). The reply for the call contains the same id. So if you are going to build your own protocol handler you need to save the ids and use them to connect request and reply.

But note the very first line above, it has no id. Thats because there is no request for it, it is what JSON-RPC calls a notification.

Conclusion:

  • web sockets are not like traditional HTTP in that you do not wait for a response for your request
  • instead requests and responses are connected with an id included in the packages
  • you might get messages without id, in that case they are notifications

3. Handle the handles

A key concept in the QIX engine protocol is the handle. For every object you open or create you get a number, a handle. Do not confuse that with the request id (also a number) or the object id (a string, in most cases a short or long GUID). You need to save the handle when you open or create an object and use it for subsequent calls.

Initially you have only one handle, the global handle, which is -1. When you open your app you get the app handle, in most cases 1.

An example from the list above:

Conclusion:

  • QIX engine uses something called handle to keep track of objects
  • you always need a handle in your calls to QIX Engine
  • the handle will be in the response from calls that create or open an object

4. Validate when changed

Some responses from the QIX engine contains another field, ‘change’. This field is an array of numbers. The numbers are actually handles and refers to objects that have been invalidated by the call. If the user for example makes a selection, she will pretty quickly get a response. This response means that QIX engine has received the selection.

It has also determined which objects of the ones you have opened that are affected (invalidated is often the term used) by this call. This means that the data you have for these objects is no longer valid.  It is your responsibility to fetch the new data, QIX engine will not automatically recalculate it for you, only tell you that the previous version is no longer valid.

For some calls there will be no change array, while for some selections(or clear selections) a lot of open objects will be affected, so the array will be long. The apis automatically refresh the data for you, but if you are building your own stuff you need to take care of it yourself. You do that with the getLayout call for the object.

If you do not refresh your data, QIX will not tell you again that it is invalid. You only get notified once, until, of course, you get new data and that is invalidated.

Conclusion:

  • QIX engine keeps track of which open objects are affected by a call
  • the field change contains an array of these objects
  • QIX will not sendd the new data until you asks for it
  • it is your responsibility to keep track of the state of the data you have fetched from the QIX engine

A Qlik Sense Generic Object can also be used to list objects in the Qlik Sense app to which it belongs. You can use it to list things like:

  • fields
  • dimensions
  • measures
  • bookmarks
  • snapshots
  • media (that is images controlled by Qlik Sense)
  • sheets
  • master objects
  • variables
  • stories

The javascript APIs provide the app.getList method, which is a wrapper that helps you get theses lists. It will create a session object for you with the list you are requesting, get that actual data in the list and call your callback function once the data is available.

List are live – your callbacks might be called several times

Just like with other Generic Objects the data might be invalidated. If you are using the javascript APIs it will then be automatically revalidated and your callback function will be called again, so you should be prepared that your callback might be called multiple times. But unlike the Hypercube and List Object the lists are not affected by the users selections. Instead they will be invalidated when the underlying data changes:

  • if the user adds a bookmark, you will get a new bookmarklist
  • if the user adds a measure, you will get a new measurelist
  • if the user adds a dimension, you will get a new dimensionlist
  • etc….

Configure the data you get in your list

The data you get in your lists is configurable. If you use the API method, you will get a default set of fields for each item in the list, but you can set the list up to suite your needs.

As an example, if you use the api method getList(‘measure’, callback), you will get a list where records look like this:

The first and second part, qInfo and qMeta, will always be there. They contain the id to use when refering to this measure and title and description use ful in a list of measures. The third part however, qData, is configurable. The default configuration just gives you tags and title one more time, but if you instead want more data about the measure, you can use your own measure list definition like this:

Under qData you provide your own mapping of the properties you want. In this case I have just specified that I want everything under qMeasure included. You can also specify individual fields.

And get output that includes more data for the measures:

And the same goes for the other lists, you can supply your own definitions and get more data included.

SWhile the HyperCube is the main object for calculating data the ListboxObject is a way to list available values for a field. Its the object behind the listboxes in the Filter Panel. It is also the object used for the listboxes that are created when you click on a field in the selection toolbar.

An example setup from the horizontal listbox extension example, included with the Qlik Sense installation:

This contains defaults for most of the parameters, except the actual field, which the user selects in the property panel, which looks like this:

As you can see the Listbox has either a predefined measure from the library, in which case qListObjectDef.qLibraryId will contain the id of the measure (some UID string) or it has a field name, which will be in the array qListObjectDef.qDef.qFieldDefs at the first position. You don’t really have to bother about which one the user selects to use, the ListboxObject will work the same anyhow.

Frequencies and sort order

There are also some other properties in the ListboxObject that are well known to QlikView users. The frequence mode gives you some additional info on the frequency of the respective values in the ListboxObject. Look to horizontal listbox extension for an example. There is also the qSortCriterias array, which allows you to specify the sort order. The default is to have the selected values first, but you might want to for example keep months order from January to December irrespective of which month is selected.

Mashup editor – even more properties

Like with the Hypercube, you can also build a ListboxObject in the mashup editor. The tool to do this looks like this:

As you can see, you also get the option to include expressions in your listbox, usable if you want to make some calculations for each value in the list. The mashup editor will create a ListboxObject for you like this:

The expressions are added to the ListboxObject in the qExpressions array. In this example I have used predefined measures, but you can also use measures defined in the actual listbox object. The mashup editor includes a expression editor (actually the same used in the Qlik Sense client) to help you with this.

Somtimes your extension does not work as expected. Here are some things you should do

1. Use your favorite browser

The browser that is built in in desktop might cache your extension files, so it is not really suited for extension development. Also in a browser you have better tools for troubleshooting and debugging. I use Google Chrome, since I think the developer tools are pretty good, but other browsers also have developer tools.

2. Look in the browser console

The console shows you if something goes wrong in your javascript code, some files are not found etc, so this is in most cases the first place to look for errors.

3. Turn off caching

When you are doing extension development the cache is your enemy: caching might mean that you are actually running an older version of your extension than you think. In Chrome you can turn it off in developer tools, in the network tab:

devtool.PNG

 

Note that the cache will only be disabled if you have developer tools open, so you should keep them open when developing.

4. Use the developer tools to look at your source code

Open your source code in the browsers developer tools:

devtool.PNG

Verify that this is the correct version.

5. Use the debugger to step through your code

The developer tools includes a very good debugger. Use it! Good places to set a breakpoint would be:

  • paint method
  • any methods triggered by an event (click etc)
  • callback methods

Note that you can also see the contents of structures like the layout and evaluate expressions used. And, of course step through your code.

6. Verify that you have only one copy of your extension under Extensions folder

Filenames of your qext file needs to be unique. If you are developing on Qlik Sense desktop it is your own responsibility to make sure that this is the case. If you make backup copies of your extension, do not save them under MyDocuments/Qlik/Sense/Extensions.

This post was originally published in Qlik Community: https://community.qlik.com/docs/DOC-8985

Probably the most important object you can have in your Generic Object is the HyperCube. It is what is used in most charts and is the way to tap into engine associative logic and calculation engine. Hypercubes reflect the users selections and calculations will be made on the current selections.

Hypercube properties

The hypercube properties are in a structure called qHyperCubeDef.  A common setup is the one from the peoplechart extension example we have seen before:

This will create a hypercube definition with an empty array of dimensions and an empty array of measures. It will also tell engine to include 2 columns (probably 1 dimension and 1 measure) and 50 rows of data with the initial fetch of data, that is in the reply to the getLayout call. This works because the user can add dimensions and measures using the property panel, which is defined like this:

In a mashup you would in most cases need to list the dimensions and measures you want, something like this (taken from my angular-based mashup example available on GitHub:

This will create a hypercube with eight dimensions and no measures, tell engine that we want all eight columns and 400 rows in the initial fetch, set some other properties and define a call back function called setCases (not included above) that will be called every time the hypercube data has changed.

Defining dimensions

While it is possible to have a hypercube without dimensions, you probably could use a simple expression instead in most cases. So you normally would have a dimension array. This array should contain one or more NxDimenion object(found here, you need to scroll down a bit). There are two alternatives:

  • use a predefined dimension from the list of dimensions. If you do that you should set the qLibraryId to the id of the dimension you want to use.
  • use a dimension that is defined only in this dimension array. In that case you should set qDef to an object that in turn contains the dimension definition. Note that qFieldDefs is actually an array, and so can contain multiple definitions. You can actually define a cyclic group here, though I have never seen it used and don’t think it is supported in the client.

Whether you use a predefined dimension or a locally defined dimension there are some additional properties that might be useful. The most complicated is qOtherTotalSpec, which you can use to limit your hypercube to the top 10 values, or bottom 10 etc.

Adding measures

The measures array works much like the dimensions array.  It also has two alternatives: referring to a predefined mease with the qLibraryId field or defining the expression in a qDef structure. Note that the measure qDef structure is not the same as the dimension qDef structure: instead of the qFieldDefs array it holds just a string field (also named qDef) with the actual expression.

Make sure you get some rows

To make sure you actually get some data in your hypercube you need to set the qInitialDataFetch property. This is actually an array, but I have never seen an example with more than one entry in the array. You should set qWidth to the number of columns (that is dimensions plus measures) in your cube and qHeight to the number of rows you need for the initial rendering of your cube.

If you do not set this property, the default is 0 rows and 0 columns, so you will get no data. A very common problem in extension development is that you add dimensions or measures and forget to update the qWidth parameter, so you do not get any values for your measures. Also rememeber that what you specify in your extension is the initial properties, so when you change your code, the objects already created will not change. A way to avoid this problem is to set qWidth to something high (10, for example) since a value that is higher than the actual number of columns will work and get you all columns.

Create a hypercube in the mashup editor

A feature not well known in Qlik Sense mashup editor is that it can help you create a hypercube. In the left column of the editor there is a button labeled ‘Add hypercube’. Select the app you want to work with and click it, and you will get a popup, something like this:

hypercubebuilder

When you click the ‘Add’ button, it will create a createCube call for you, with most of the properties filled in:

Even if you don not use the mashup editor for creating your mashup, this feature can be useful.