Use Qlik in your own apps and sites
I have just released a new version (3.1) of the Qlik Sense variable extension at GitHub and Qlik Branch. News in this relaese is that you can have dynamic values for your dropdown list and buttons. It is based on a contribution, but I refactored it somewhat to make it a bit more general and make buttons dynamic too.
It should be totally backward compatible with the previous release and upgrading should not cause you any problems.
Qlik Sense has several APIs and a .Net SDK to help you work with the QIX Engine. They help you working with Qlik Sense without having to bother about the details. But if you need to work on a lower level, there are some problems you need to solve.
Qlik Sense QIX engine talks Web Socket, and only web socket. While you might need to make a https call to trigger authentication, the actual communication with the engine is web socket only.
The format of the URL can be seen in the Engine API Explorer, which you will find in Qlik Sense devhub:
ws://localhost:4848/app/c%3A%5Cusers%5Cerikw%5Cdocuments%5Cqlik%5Csense%5Capps%5Cexecutive%20dashboard.qvf?
Note that the app id is included in the URL, but what you get is an open web socket, not an open document. Actually, in Qlik Sense desktop it will work without the app id in the URL. That’s because the app id in the URL is not for QIX Engine. It’s for the proxy you will go through to reach Engine, and used for load balancing, the proxy uses it to find which engine to use for this app. This also means that you should always open a new web socket when you open a new app, since you need to give the proxy a chance to select the correct engine.
Conclusion:
With traditional HTTP communication you make a request, wait, and get a response. Web sockets work differently. You send a message to QIX Engine, but don’t wait for the answer. You just keep on sending messages, and eventually QIX Engine will reply. You do not know the order of the responses and actually the protocol does not tell you what respone belongs to which request. Typically it looks something like this:
This is the initial communication when Qlik Sense standard client opens an app. If you think it is hard to read (and it is) you can open the browser developer tools and look in the network tab, the image above is from Chrome.
The darker rows are requests sent from the client to QIX Engine, while the white rows are responses. As you can se the client makes a series of requests (seven) and after that comes the respones. It is not obvious which responses belong to which call, and the respones can not be understood without knowing the request.
So how do you know which respons belongs to which request? Qlik Sense uses a standard called JSON-RPC for this. This means that in each request there is a field called id that contains a unique identifier for this request (in this case just a number 1, 2, 3 etc). The reply for the call contains the same id. So if you are going to build your own protocol handler you need to save the ids and use them to connect request and reply.
But note the very first line above, it has no id. Thats because there is no request for it, it is what JSON-RPC calls a notification.
Conclusion:
A key concept in the QIX engine protocol is the handle. For every object you open or create you get a number, a handle. Do not confuse that with the request id (also a number) or the object id (a string, in most cases a short or long GUID). You need to save the handle when you open or create an object and use it for subsequent calls.
Initially you have only one handle, the global handle, which is -1. When you open your app you get the app handle, in most cases 1.
An example from the list above:
Conclusion:
Some responses from the QIX engine contains another field, ‘change’. This field is an array of numbers. The numbers are actually handles and refers to objects that have been invalidated by the call. If the user for example makes a selection, she will pretty quickly get a response. This response means that QIX engine has received the selection.
It has also determined which objects of the ones you have opened that are affected (invalidated is often the term used) by this call. This means that the data you have for these objects is no longer valid. It is your responsibility to fetch the new data, QIX engine will not automatically recalculate it for you, only tell you that the previous version is no longer valid.
For some calls there will be no change array, while for some selections(or clear selections) a lot of open objects will be affected, so the array will be long. The apis automatically refresh the data for you, but if you are building your own stuff you need to take care of it yourself. You do that with the getLayout call for the object.
If you do not refresh your data, QIX will not tell you again that it is invalid. You only get notified once, until, of course, you get new data and that is invalidated.
Conclusion:
A Qlik Sense Generic Object can also be used to list objects in the Qlik Sense app to which it belongs. You can use it to list things like:
The javascript APIs provide the app.getList method, which is a wrapper that helps you get theses lists. It will create a session object for you with the list you are requesting, get that actual data in the list and call your callback function once the data is available.
List are live – your callbacks might be called several times
Just like with other Generic Objects the data might be invalidated. If you are using the javascript APIs it will then be automatically revalidated and your callback function will be called again, so you should be prepared that your callback might be called multiple times. But unlike the Hypercube and List Object the lists are not affected by the users selections. Instead they will be invalidated when the underlying data changes:
Configure the data you get in your list
The data you get in your lists is configurable. If you use the API method, you will get a default set of fields for each item in the list, but you can set the list up to suite your needs.
As an example, if you use the api method getList(‘measure’, callback), you will get a list where records look like this:
The first and second part, qInfo and qMeta, will always be there. They contain the id to use when refering to this measure and title and description use ful in a list of measures. The third part however, qData, is configurable. The default configuration just gives you tags and title one more time, but if you instead want more data about the measure, you can use your own measure list definition like this:
Under qData you provide your own mapping of the properties you want. In this case I have just specified that I want everything under qMeasure included. You can also specify individual fields.
And get output that includes more data for the measures:
And the same goes for the other lists, you can supply your own definitions and get more data included.
SWhile the HyperCube is the main object for calculating data the ListboxObject is a way to list available values for a field. Its the object behind the listboxes in the Filter Panel. It is also the object used for the listboxes that are created when you click on a field in the selection toolbar.
An example setup from the horizontal listbox extension example, included with the Qlik Sense installation:
This contains defaults for most of the parameters, except the actual field, which the user selects in the property panel, which looks like this:
As you can see the Listbox has either a predefined measure from the library, in which case qListObjectDef.qLibraryId will contain the id of the measure (some UID string) or it has a field name, which will be in the array qListObjectDef.qDef.qFieldDefs at the first position. You don’t really have to bother about which one the user selects to use, the ListboxObject will work the same anyhow.
Frequencies and sort order
There are also some other properties in the ListboxObject that are well known to QlikView users. The frequence mode gives you some additional info on the frequency of the respective values in the ListboxObject. Look to horizontal listbox extension for an example. There is also the qSortCriterias array, which allows you to specify the sort order. The default is to have the selected values first, but you might want to for example keep months order from January to December irrespective of which month is selected.
Mashup editor – even more properties
Like with the Hypercube, you can also build a ListboxObject in the mashup editor. The tool to do this looks like this:
As you can see, you also get the option to include expressions in your listbox, usable if you want to make some calculations for each value in the list. The mashup editor will create a ListboxObject for you like this:
The expressions are added to the ListboxObject in the qExpressions array. In this example I have used predefined measures, but you can also use measures defined in the actual listbox object. The mashup editor includes a expression editor (actually the same used in the Qlik Sense client) to help you with this.
Somtimes your extension does not work as expected. Here are some things you should do
The browser that is built in in desktop might cache your extension files, so it is not really suited for extension development. Also in a browser you have better tools for troubleshooting and debugging. I use Google Chrome, since I think the developer tools are pretty good, but other browsers also have developer tools.
The console shows you if something goes wrong in your javascript code, some files are not found etc, so this is in most cases the first place to look for errors.
When you are doing extension development the cache is your enemy: caching might mean that you are actually running an older version of your extension than you think. In Chrome you can turn it off in developer tools, in the network tab:
Note that the cache will only be disabled if you have developer tools open, so you should keep them open when developing.
Open your source code in the browsers developer tools:
Verify that this is the correct version.
The developer tools includes a very good debugger. Use it! Good places to set a breakpoint would be:
Note that you can also see the contents of structures like the layout and evaluate expressions used. And, of course step through your code.
Filenames of your qext file needs to be unique. If you are developing on Qlik Sense desktop it is your own responsibility to make sure that this is the case. If you make backup copies of your extension, do not save them under MyDocuments/Qlik/Sense/Extensions.
This post was originally published in Qlik Community: https://community.qlik.com/docs/DOC-8985
Probably the most important object you can have in your Generic Object is the HyperCube. It is what is used in most charts and is the way to tap into engine associative logic and calculation engine. Hypercubes reflect the users selections and calculations will be made on the current selections.
Hypercube properties
The hypercube properties are in a structure called qHyperCubeDef. A common setup is the one from the peoplechart extension example we have seen before:
This will create a hypercube definition with an empty array of dimensions and an empty array of measures. It will also tell engine to include 2 columns (probably 1 dimension and 1 measure) and 50 rows of data with the initial fetch of data, that is in the reply to the getLayout call. This works because the user can add dimensions and measures using the property panel, which is defined like this:
In a mashup you would in most cases need to list the dimensions and measures you want, something like this (taken from my angular-based mashup example available on GitHub:
This will create a hypercube with eight dimensions and no measures, tell engine that we want all eight columns and 400 rows in the initial fetch, set some other properties and define a call back function called setCases (not included above) that will be called every time the hypercube data has changed.
Defining dimensions
While it is possible to have a hypercube without dimensions, you probably could use a simple expression instead in most cases. So you normally would have a dimension array. This array should contain one or more NxDimenion object(found here, you need to scroll down a bit). There are two alternatives:
Whether you use a predefined dimension or a locally defined dimension there are some additional properties that might be useful. The most complicated is qOtherTotalSpec, which you can use to limit your hypercube to the top 10 values, or bottom 10 etc.
Adding measures
The measures array works much like the dimensions array. It also has two alternatives: referring to a predefined mease with the qLibraryId field or defining the expression in a qDef structure. Note that the measure qDef structure is not the same as the dimension qDef structure: instead of the qFieldDefs array it holds just a string field (also named qDef) with the actual expression.
Make sure you get some rows
To make sure you actually get some data in your hypercube you need to set the qInitialDataFetch property. This is actually an array, but I have never seen an example with more than one entry in the array. You should set qWidth to the number of columns (that is dimensions plus measures) in your cube and qHeight to the number of rows you need for the initial rendering of your cube.
If you do not set this property, the default is 0 rows and 0 columns, so you will get no data. A very common problem in extension development is that you add dimensions or measures and forget to update the qWidth parameter, so you do not get any values for your measures. Also rememeber that what you specify in your extension is the initial properties, so when you change your code, the objects already created will not change. A way to avoid this problem is to set qWidth to something high (10, for example) since a value that is higher than the actual number of columns will work and get you all columns.
Create a hypercube in the mashup editor
A feature not well known in Qlik Sense mashup editor is that it can help you create a hypercube. In the left column of the editor there is a button labeled ‘Add hypercube’. Select the app you want to work with and click it, and you will get a popup, something like this:
When you click the ‘Add’ button, it will create a createCube call for you, with most of the properties filled in:
Even if you don not use the mashup editor for creating your mashup, this feature can be useful.
Getting data out of a Qlik Sense app is based on a concept called Generic Object. Behind all charts, tables and listboxes in Qlik Sense lies a generic object. Also behind many of the other things you see in the Qlik Sense client lies generic objects:
The purpose of the Generic Object is to make it easier to make client-side changes without having to change the QIX engine. Since the Generic Object is very flexible, many client side changes do not require any changes to the engine. This is in contrast to how it works in QlikView, where all charts have different server-side implementations and client and server code are tightly coupled. This is also a reason why extensions are so powerful in Qlik Sense: to the QIX engine there is really no difference between built-in visualizations and extensions, they are all represented with a Generic Object.
Properties and layout
When you create a Generic Object you define its properties. This is a JSON structure, which you can define yourself. An example (from the peoplechart extension example):
There are two kind of properties:
All properties defined by Engine start with a q, so it is good programming practice to let your own properties start with some other letter. The properties defined by Engine are documented here and the corresponding out data is documented here. Note that properties defined by the Qlik Sense client(like title etc) do NOT start with a q, the QIX engine does not really know what these are used for.
But the parameters known by the QIX Engine will determine the engines calculation. In the output from engine, called layout (though it’s not really the layout of a Generic Object) the definition of those properties will be replaced by the results. A hypercube definition (qHyperCubeDef) will be replaced by a actual hypercube (qHyperCube) containing the results from the calculation.
Generic Object context
Generic objects are everywhere in Qlik Sense. If you are building an extension, you define your Generic Object structure in the initialProperties property. If you are using the mashup API, you create generic objects with the createGenericObject method or one of the wrappers around it (createCube, createList, getList). In both these cases the Qlik Sense client framework will automatically refresh the Generic object with new data when it is invalidated (by for example new selections). If you are working directly on the protocol level, you need to fetch they new data yourself.
The main use of Qlik Sense mashup API is to visualize data from Qlik Sense apps and reuse visualizations in an app, allowing your users to interact with the data. But you can use it for other things to, like running a reload or interacting with Qlik Sense server.
But a Qlik Sense app can be used in two different contexts: desktop and server. While visualizations and interactions with those work the same in Server and Desktop, the server API (which is a REST API, unlike the QIX Engine API which works over the web socket. So the first thing we need to do is check our environment, are we in Server or Desktop. Luckily, there is a method for that, the isPersonalMode method.
Once we know if we are in Desktop or Server we can acct accordingly. If we are in Desktop, we can just call the doReload method. While this might work in Server too, we really shouldn’t use it. In a server environment you might very likely have different Qix Engines for interaction with the user and reloads. In fact it is very likely that the engine we are connected to does not have the right database connections for the reload, so running the reload there would fail. Instead we call the Qlik Server for the reload, and let it determine which computer should do the actual job. We can do this with the callRepository method.
This would look something like this:
You find a full working example in the Github repository.
This post is based on a session I originally held at Qonnections 2015, under the title ‘5 things you did not know you can do with the Mashup API’.
If you are using the Qlik Sense standard client there is a Selection toolbar displayed at the top of the page. It has lots of useful functionality, including search support, so in many cases you would want to use this in a mashup, which is perfectly possible.
But sometimes the selection toolbar does not fit in your web page or you want something more compact. In that case you can listen to selection changes yourself and visualize the selection state.
The way to get the selection state is the getList method. An example:
The reply will contain a qSelectionObject with:
Using this you can make your own visualization of selection state, or perhaps hide and show visualizations based on selection states.
When you use the getObject. call of the Qlik Sense mashup API (also known as the Capabilities API or qlik.js) it works a bit like a black box: you just call it and it will inject some Qlik content into your page , you don’t have to bother about the content, it might be any type of chart, it might even be an extension. But it you want to, you can access the data of the visualization.
The key to this is the return value from getObject. The method returns, as it says in the documentation ‘a promise of an object model’. That means you can use the return value to get access to the data in the visualization, or even call some methods to modify the visualization automatically.
Only you have to understand:
Promise
The fact that getObject returns a Promise means that you will get an object with a then method. The then method will take a parameter, a function, that is called when getObject is ready, that is after a while, normally just some milliseconds, so to you it will look like immediately, but for the computer this is a long time.
What happens when you call getObject is:
Only after these two calls will the promise resolve, and the method you supplied as a parameter to the then method will be called. That means that at this stage the object you get will be a model with valid data.
The model
A good way to find out the contents of model is to use the debugger in your browser and inspect it. Some important parts of the model:
So, to sum this up, you can get the title of the visualization like this:
You find a full working example in the Github repository.
This post is based on a session I originally held at Qonnections 2015, under the title ‘5 things you did not know you can do with the Mashup API’.