Sometimes when you are working with the Qlik Sense APIs you need access to information that is not connected to the Qlik Sense document you are working with, but rather to the QIX Engine running.
You find the apis for this in the global object in the Capabilities APIs, or in the Global class in the Engine API. While the Capabilities APIs only include very few methods, the full list of calls in the Engine API is much longer.
Finding the Global object
While there is a method in the Capabilities API to get the global object, in most cases you should not use it. In fact if you are building an extension you never should. Instead you should use the global property on the app.
That is because the getGlobal() method will open a new web socket to the engine, while app.global property will use the web socket used for the app. There is not reason to open another web socket to the server when you already have one, and in an extension there always is one.
The only situation getGlobal() should be used is when you do not have an open web socket, like in a mashup before the app is opened. And in that case you might be better off with a REST call.
As I wrote in Part 1 Chrome Developer tools is perhaps the most important tool for a Qlik Sense extension developer. Some important points in part 1:
you should turn caching off in the network tab, and keep developer tools open to make sure files are not cached
Qlik Sense client catches program errors in your extension, so if you need to see them temporarily check the box ‘Pause on caught exceptions’. You can’t keep this box checked all the time though.
In this post we will continue with some more useful stuff.
Checking web socket traffic
As you probably know communication with the Qlik Engine is over web sockets. You find the web socket in the Network tab, just click on WS and you will find it. Click on the websocket and then on messages, and you will see the communcation, something like this:
Lots of information, lots of transactions. Some things you should know:
messages from client to server has a dark background, messages from server to client a white background
all messages are in JSON format
if you click on a message it will be displayed in a format that’s easier to read in the area below
you can filter messages by typing in text in the field where it says ‘Enter regex..’
Connect request and reply
What you see in the web socket log is simply a list of all messages sent and received, in chronological order. Since commands are processed in paralell, the reply for a specific request might not at all be anywhere near the request. Request and reply are connected with the ‘id’ attribute (that’s a feature in the json rpc protocol, which Qlik Sense is built on). This means we can filter on the id attribute to find the reply for a specific request. If you try typing in “id”:1, in the filter box you will get request #1 and it’s reply:
Handling handles
So now we know how to find a request and it’s reply, but to really understand what’s happening we also need to understand handles.
Handles is a Qlik Sense specific concept, not part of Json RPC but an addition to it, made by Qlik. The QIX protocol is object-oriented, meaning that all commands are sent to an object. When you connect to Qlik Sense engine, the only handle available is the global handle, -1.
The first thing you do in most cases is call OpenDoc to open your Qlik Sense app (or Doc, as its called in the QIX documentation). If that succeeds you will get a handle (called qHandle) back in the reply. You can them send Doc commands to this handle (almost always handle 1). If you type in the filter “handle”:1, you will see all app commands used:
As you can see there is a lot of them. Most of them fetches or creates objects in the app, with GetObject or CreateSessionObject. You do not see the replies, only the requests, since the replies do not contain the handle. But if we instead filter by the id of one of the GetObject calls, (in my case id 13) we can see what happens:
As you can see in the reply there is a new qHandle in the reply, #5. So now we can filter on “handle”:5, and see what commands were sent to this object. If you want to know what handle is used for your extension, there is as far as I know no way to get that information from standard Qlik, but you can use my Chrome Extension Add Sense.
Find commands that invalidates object
Handles are used not only to refer to objects when you send commands, but also for invalidation. When you change selections (make a new selection, clear selection, apply bookmark etc) QIX engine tells you what objects are no longer invalid. It does this with the ‘change’ array (another addition to JSON rpc) which contains handles for the invalidated objects. So if I for example want to find commands that invalidated opbject 13, I can filter on “change”:\[.*13.*\] (note: a regex, since the handle can be at any position in the array):
We only get the replies for commands that invalidated the object, to get the actual command you need to use the id field to find the actual command and handle.
Perhaps the most important tool for a Qlik Sense extension developer is Chrome developer tools. You can use any text editor to write your code, or even Qlik Sense dev-hub for simple extension, but you really need to use Developer Tools. Debug your code, check loading of javascript, HTML and CSS, inspect HTML structure and CSS rules – there is a lot you need to use them for. So good knowledge of them is essential.
Always run Qlik Sense in a browser
Qlik Sense Desktop is a good tool for your extension development in most scenarios. But you should never use the browser bundled with it for development. Instead use Chrome, and use http://localhost:4848/hub to access the hub. This means you will have the full developer tools available, and also that you will have an up-to-date Chrome installation.
While the browser bundled with Qlik Sense Desktop is built on the same codebase, it is not updated, but will contain an old version. It looks like the Chrome version bundled with Qlik Sense Desktop September 2019 is Chrome/47.0.2526.16 while my browser Chrome version is Chrome/78.0.3904.70, so Qlik Sense Desktop is way behind. BTW Qlik Sense Desktop also reports my Windows version as ‘Windows NT 6.1’ while Chrome proper reports the real value ‘Windows NT 10.0’.
So now that you are happily in Chrome you need to open the Developer Tools. There are different ways to do that:
press Shift+Ctrl i (for inspect, this is the method I use)
from Chrome menu in the top right corner, under ‘Other tools’
When doing extension development, always keep the console open. We’ll come to why later.
Disable caching
When doing extension development, you want Qlik Sense to allways load the latest version of your code. The browser normally tries to cache loaded files, and not request the file from the server again, it it already has it in memory. The first thing you should do is to disable this caching.It’s easy to do:
open developer console
switch to network tab
click the ‘Disable cache’ checkbox
The disable cache checkbox only affects the browser when the developer console is open. When the developer console is not open, the browser will still cache javascript files. This is why you should always open the browser console when your are working with extension development.
Check installed extensions (and mashups)
There is a lot of information in the network tab. You can se all network requests made by the Qlik Sense client, and since the client is load in chunks there will be a lot of them. There are also requests for images, CSS stylesheets and other requests. To help you filter the requests there is a bar:
Click on XHR and you will get a list of other calls. One of them is of special interest when you are doing extension development, the one called schema (url is really /qrs/extension/schema). Click on that one, and then on ‘Preview’ and you will get a list of all extensions and mashups installed on the system, including all data in the qext file:
This is a good place to check that your extension is installed, and what version (just remember to keep the version number in the qext file up to date). It’s also a good place to start when you are troubleshooting an extension you did not write.
Debugging the extension code
The ‘Sources’ tab is what you use for debugging. If the code is minified (many extensions are) you can click the curly braches in the bottom left corner to get a somewhat more readable format.
The toolbar in the upper right corner can be used to step through your code. The stop button will make the browser pause when an error occurrs, which is very useful to find bugs. Unfortunately it does not really work in extensions. That’s because Qlik Sense catches all extension errors (and silently ignores them).
So to have the browser pause on your errors, you need to check the box ‘Pause on caught exceptions’. If you do that, the browser will pause when an exception is thrown from your code. Unfortunately it will also break on some exceptions (in libraries) that are perfectly OK, so you can’t really turn it on always. Just turn it on when you think there might be an exception thrown from your code and then trigger a repaint, for example by resizin the window.
Check files in extension
The navigator on the left side in the sources tab shows you extension files in a tree structure. But it will only show you javascript files, so CSS and HTML files are not included. If you want to see those, switch to the network tab, and filter on the extension name.
Note that the qext file is not included. That’s because the qext file is not loaded by the client, the information is instead included in the schema.
and there’s more
Of course there is more, but I couldn’t cover everything in one post. Look out for part 2, available when I find the time to write it..
Probably the most important object you can have in your Generic Object is the HyperCube. It is what is used in most charts and is the way to tap into engine associative logic and calculation engine. Hypercubes reflect the users selections and calculations will be made on the current selections.
Hypercube properties
The hypercube properties are in a structure called qHyperCubeDef. A common setup is the one from the peoplechart extension example we have seen before:
This will create a hypercube definition with an empty array of dimensions and an empty array of measures. It will also tell engine to include 2 columns (probably 1 dimension and 1 measure) and 50 rows of data with the initial fetch of data, that is in the reply to the getLayout call. This works because the user can add dimensions and measures using the property panel, which is defined like this:
In a mashup you would in most cases need to list the dimensions and measures you want, something like this (taken from my angular-based mashup example available on GitHub:
This will create a hypercube with eight dimensions and no measures, tell engine that we want all eight columns and 400 rows in the initial fetch, set some other properties and define a call back function called setCases (not included above) that will be called every time the hypercube data has changed.
Defining dimensions
While it is possible to have a hypercube without dimensions, you probably could use a simple expression instead in most cases. So you normally would have a dimension array. This array should contain one or more NxDimenion object(found here, you need to scroll down a bit). There are two alternatives:
use a predefined dimension from the list of dimensions. If you do that you should set the qLibraryId to the id of the dimension you want to use.
use a dimension that is defined only in this dimension array. In that case you should set qDef to an object that in turn contains the dimension definition. Note that qFieldDefs is actually an array, and so can contain multiple definitions. You can actually define a cyclic group here, though I have never seen it used and don’t think it is supported in the client.
Whether you use a predefined dimension or a locally defined dimension there are some additional properties that might be useful. The most complicated is qOtherTotalSpec, which you can use to limit your hypercube to the top 10 values, or bottom 10 etc.
Adding measures
The measures array works much like the dimensions array. It also has two alternatives: referring to a predefined mease with the qLibraryId field or defining the expression in a qDef structure. Note that the measure qDef structure is not the same as the dimension qDef structure: instead of the qFieldDefs array it holds just a string field (also named qDef) with the actual expression.
Make sure you get some rows
To make sure you actually get some data in your hypercube you need to set the qInitialDataFetch property. This is actually an array, but I have never seen an example with more than one entry in the array. You should set qWidth to the number of columns (that is dimensions plus measures) in your cube and qHeight to the number of rows you need for the initial rendering of your cube.
If you do not set this property, the default is 0 rows and 0 columns, so you will get no data. A very common problem in extension development is that you add dimensions or measures and forget to update the qWidth parameter, so you do not get any values for your measures. Also rememeber that what you specify in your extension is the initial properties, so when you change your code, the objects already created will not change. A way to avoid this problem is to set qWidth to something high (10, for example) since a value that is higher than the actual number of columns will work and get you all columns.
Create a hypercube in the mashup editor
A feature not well known in Qlik Sense mashup editor is that it can help you create a hypercube. In the left column of the editor there is a button labeled ‘Add hypercube’. Select the app you want to work with and click it, and you will get a popup, something like this:
When you click the ‘Add’ button, it will create a createCube call for you, with most of the properties filled in:
Even if you don not use the mashup editor for creating your mashup, this feature can be useful.
The main use of Qlik Sense mashup API is to visualize data from Qlik Sense apps and reuse visualizations in an app, allowing your users to interact with the data. But you can use it for other things to, like running a reload or interacting with Qlik Sense server.
But a Qlik Sense app can be used in two different contexts: desktop and server. While visualizations and interactions with those work the same in Server and Desktop, the server API (which is a REST API, unlike the QIX Engine API which works over the web socket. So the first thing we need to do is check our environment, are we in Server or Desktop. Luckily, there is a method for that, the isPersonalMode method.
Once we know if we are in Desktop or Server we can acct accordingly. If we are in Desktop, we can just call the doReload method. While this might work in Server too, we really shouldn’t use it. In a server environment you might very likely have different Qix Engines for interaction with the user and reloads. In fact it is very likely that the engine we are connected to does not have the right database connections for the reload, so running the reload there would fail. Instead we call the Qlik Server for the reload, and let it determine which computer should do the actual job. We can do this with the callRepository method.
If you are using the Qlik Sense standard client there is a Selection toolbar displayed at the top of the page. It has lots of useful functionality, including search support, so in many cases you would want to use this in a mashup, which is perfectly possible.
But sometimes the selection toolbar does not fit in your web page or you want something more compact. In that case you can listen to selection changes yourself and visualize the selection state.
The way to get the selection state is the getList method. An example:
The reply will contain a qSelectionObject with:
qBackCount and qForwardCount, the number of steps forward and backward that is possible
a qSelections array, with on entry for every field that is selected
the array entries will contain qField – the field name, qTotal – total number of different values in the field and qSelectedCount – the number of selected values
Using this you can make your own visualization of selection state, or perhaps hide and show visualizations based on selection states.
When you use the getObject. call of the Qlik Sense mashup API (also known as the Capabilities API or qlik.js) it works a bit like a black box: you just call it and it will inject some Qlik content into your page , you don’t have to bother about the content, it might be any type of chart, it might even be an extension. But it you want to, you can access the data of the visualization.
The key to this is the return value from getObject. The method returns, as it says in the documentation ‘a promise of an object model’. That means you can use the return value to get access to the data in the visualization, or even call some methods to modify the visualization automatically.
Only you have to understand:
how a promise works
and how to find data in the model you eventually get
Promise
The fact that getObject returns a Promise means that you will get an object with a then method. The then method will take a parameter, a function, that is called when getObject is ready, that is after a while, normally just some milliseconds, so to you it will look like immediately, but for the computer this is a long time.
What happens when you call getObject is:
the browser will call the QIX engine over the web socket for the definition of the object with the id you supplied
when the definition comes back it will call the engine to get the data
Only after these two calls will the promise resolve, and the method you supplied as a parameter to the then method will be called. That means that at this stage the object you get will be a model with valid data.
The model
A good way to find out the contents of model is to use the debugger in your browser and inspect it. Some important parts of the model:
layout: this is the data for the visualization. Expressions, hypercubes etc will be evaluated and contain data reflecting the current selections
events, most important is the Validated event, which will be triggered when there is new valid data in the model. a full list is here ( the page refers to AngularJS-based extensions, but this is general, the same events is used for all visualizations, whether they are extensions or not)
So, to sum this up, you can get the title of the visualization like this:
One of the really powerful features of Qlik Sense API’s is the possibility to dynamically modify visualizations. You can build solutions that allows the user not only to make selections and have them reflected in all charts displayed (that’s the default, you will get that automatically if you don’t turn it off) but also to dynamically change chart properties and do things like
change dimension from for example Product to Product Lin (or to Division)
change measure from Sales Amount to Quantity or Margin or Cost
add an additional dimension and make a barchart into a stacked bar chart
modify ordering
limit the chart to only show Top 10, or bottom 5
change the chart type from Barchart to Linechart or to an extension
And all of this can be done for both built-in visualizations and extensions in the same way. The key to all of this is the applyPatches method and soft patches.
Soft patches
A soft patch is a temporary change to properties for a visualization. The fact that it is temporary means:
it will not be persisted either in Desktop or Server
it will not affect other users using the same app
once the Qix engine session is closed it will be gone, so when the user reconnects, visualizations will be back to their original state
the user can make soft patches to visualizations that me does not have the right to modify
You make sure that its a soft patch by setting the second parameter in the applyPatches call to true. The first parameter is the list of patches you want to apply. Since you might want to apply several patches, it is an array, where each entry has three values:
qPath: the path to the property you want to change, with slash (/) where javascrip/json usually has a dot
qOp: the operation you want to perform, one of “add”, “replace”, “remove”
qValue: the value you want to set. This should be a JSON value inside a string, so a boolean value would be “true” or “false”, a numeric value something like “-1” and a string (this is the tricky part) “\”new string\”” (that is a string in a string). You could also set a javascript object, use javascript standard JSON.stringify method in that case.
Perhaps this is clearer if we look at an example:
What we do in this example is that we show a list of dimensions and measures to the user. When the user clicks on one of them, we move it to be the first one in the sort order, and use applyPatches to update the array and the chart will be resorted. Since it is all done with soft patches, it will not affect other users.
The Qlik Sense Mashup API (also known as Capabilities API or simply qlik.js) allows you to inject Qlik Sense charts with full interactivity into your web page with just a few lines of code. It also allows you to dynamically define hypercubes, lists or simple expressions, send them to Qlik calculation engine and get results back, so you can visualize the data yourself. If you use the APIs all this would be well known to you. But there is more.. Let’s have a look at some more advanced things you can do.
Show visualizations in a tab
Sometimes you want a user interface that is a bit more advanced than just showing visualizations when the page is loaded. For example you might want to show the visualization in a tab, something like this:
This examples uses jQuery tabs, but you could use any tabs implementation, the basic ideas are the same. What you need to do to make this work:
when the user wants to add a new visualization, call the getObject method, with the html element and the id of the visualization as parameters
when the user switches between tabs, you need to call the resize method, to make sure that the visualization displayed is updated.
A code example. Note that much of this is actually more jQuery than qlik, the only part that is really qlik is the two methods calls:
You’ll find a working example in the Github repository. Note that the key to this is only two calls:
call getObject the first time the user wants to see a new chart, not when the page loads
