I’ve just started planning (well more like thinking about) the next version of Add Sense for Chrome. What would you like to see in the extension? Any bugs you hate? Now is a good time to let me know…
Guide to Qlik Sense ApplyPatches
One of the more powerful calls in the QIX Api is the ApplyPatches call. It allows you to dynamically add, change or remove basically any property of a generic object. With it, you can allow users to dynamically change visualizations: reorder data, change dimensions and measures, add limitation etc etc. But using it in your implementation can be tricky.
Parameters
The ApplyPatches call only has two parameters (plus the handle, identifying which object we are patching:
Name | Description | Mandatory | Type |
qPatches | Array of patches. | Yes | Array of NxPatch |
qSoftPatch | Patches are not saved | No | Boolean |
So it’s an arry and a boolean (true/false) flag. Let’s start with the flag. Setting the qSoftPatch flag to true does mean that the patched property is not saved, but it also has some other consequences:
- you can change objects that you otherwise cannot, like published objects, or objects in apps you cannot change
- the changes will only affect you, other users will not see them, they might very well apply a completely different patch to the same object
- the changes will only affect the current session. If you disconnect and connect again, they will be lost and you will start over with the original version of the object
So if your goal is to allow users to dynamically change visualizations you would want qSoftPatch to be true. While the default value false works OK in Qlik Sense Desktop, it’s generally not something you would want to use in a server environment.
The array of patches
The array is a bit more difficult. It’s an array so you can change several properties with one call. For example if you change dimension or measure you should consider changing the title too, and probably do that in the same call. Every entry(almost) in the array has three parameters:
Name | Description | Type |
qOp | Operation to perform. Add, remove or replace. | String |
qPath | Path to the property. | String |
qValue | The value of the property | String |
The first parameter, qOp, is an easy one. In most cases you want to change a property, use ‘replace’. Occasionaly you want to add a property, use ‘add’. Rarely you want to remove a property, in that case use ‘remove’.
The second parameter, qPath, is the path to the property, using / between the property names (not . as in javascript). Remember:
- you’re patching properties, not the layout, so it’s /qHyperCubeDef/… not qHyperCube
- for arrays you need to include the position in the array, and numbering starts with zero, so first dimension is /qHyperCubeDef/qDimensions/0/ etc
- property structures can be complicated, speciallly the hypercube..
- you can use my Chrome extension Add Sense to create the path argument. Show properties for the objects, switch to patch mode, copy the path
The second parameter, qPath, is the path to the property, using / between the property names (not . as in javascript). Remember:
- you’re patching properties, not the layout, so it’s /qHyperCubeDef/… not qHyperCube
- for arrays you need to include the position in the array, and numbering starts with zero, so first dimension is /qHyperCubeDef/qDimensions/0/ etc
- property structures can be complicated, speciallly the hypercube..
- you can use my Chrome extension Add Sense to create the path argument. Show properties for the objects, switch to patch mode
Some examples
A common case is when you want your users to be able to change the ordering of for example a table dynamically. Ordering of a hypercube is determined by the qInterColumnSortOrder property, an array with the column numbers in the order they should be used for sorting. If you want say the third column (which has number 2, since numbering starts with 0) you move the number 2 to the beginnging of the array and call applyPatches:
model.applyPatches( [{
qPath: '/qHyperCubeDef/qInterColumnSortOrder',
qOp: 'replace',
qValue: JSON.stringify(sortorder)
}], true );
You can find a more complete example in the mashup I showed at Qonnections back in 2015.
Another example is when you want to reverse the ordering of a hypercube. You do this by toggling the boolean value of qReverseSort in the dimension or measure where you want to change the order. For the first dimension this will be:
model.applyPatches( [{
qPath: '/qHyperCubeDef/qDimensions/0/qDef/qReverseSort',
qOp: 'replace',
qValue: JSON.stringify( !reversesort )
}], true );
Qlik nebula.js – first web app
A while ago I did my first test project with Qlik’s new open source library nebula.js (it’s available here). That time I tested the visualization part, corresponding to extensions in the current product (but hopefully used also for visualizations delivered by Qlik). Now it’s time to try it from the other side, building a web app with these new kind of visualizations.
Terminology first: the term mostly used in Qlik Sense is mashup. That’s because the current javascript API was originally designed for a mashup scenario, where you wanted to add Qlik Sense objects to an existing web page. But what it’s mostly used for today is building web apps, that is pages that include only Qlik Sense objects. And that is what I intend to do in this small test project.
Setting up
Nebula.js includes a small example app. You find it here. It’s just a few files, since it actually downloads the libraries (enigmajs, nebulajs) over the internet. So if we just copy those files to a local directory we have got a starting point.
The example uses a docker installation and a session app, but I want to use my Qlik Sense Desktop and the Consumer Sales app. To fix that I revised (and simplified) the connect.js file so it looks like this:
window.connect = function connect(host, id) {
return fetch('https://unpkg.com/enigma.js/schemas/3.2.json').
then(response => response.json()).
then(schema => window.enigma.create({
schema,
url: `ws://${host}/app/${id}`,
}).open().then(qix => qix.openDoc(id)));
};
And in the index.js file add host and app:
connect('localhost:4848','Consumer Sales.qvf').then((app) => {
and we have got something that’s working. Start Qlik Sense desktop, make sure your test project is served (I use serve), and open the file in the browser. You’ll se a selection toolbar, and an area that displays the simple supernova that’s included:
If you wonder how come my selection toolbar shows selections, that’s because I opened the same app with the same url(use http://localhost:4848/sense/app/Consumer%20Sales.qvf ) and made a selection. Session sharing does the rest.
Load nova-table
OK, so now we have got a page using nebulajs and connecting to an app, and the selction toolbar works. But how do we use supernovas like the one I made in my first test drive? Well first we need to make it available in our project. Just copy the files from the dist directory in nova-table and put it in your project. I have used a sub-directory called novas, since there is actually nothing super about nova-table. You can then load it in index.html:
<script src="./novas/nova-table.js"></script>
<script src="connect.js"></script>
<script src="index.js"></script>
It needs to be before index.js for this to work.
Unlike the current Qlik Sense javascript libraries, nebulajs is not bundled with a module management library like requirejs. Instead it will see if there is a module manager available and use it if there is. If not it will simply add the module as a global function. So we can access it simply as window[‘nova-table’]. In a real project we would not really want everything defined globally, but this is just a quick test.
To make a solution that can be extended with more visualization, we create a novas object, where we can put all our visualizations, and then grab the right one using the type, with the sn from the initial example as the default:
const novas = {
'nova-table': window['nova-table']()
};
const sn = {
component: {
mounted(element) {
element.textContent = 'Hello';
},
},
};
const nebbie = window.nucleus(app, {
load: (type, config) => config.Promise.resolve(novas[type.name] || sn),
});
Add a table
So now we have got the nova-table module loaded, let’s create one and add to the page. We’ll make a session object, with the definition in the javascript file, so first we change the HTML file to:
<div class="content">
<div class="toolbar"></div>
<div class="object" id="nova01"></div>
</div>
And in the javascript file we use the create method to create a nova-table object, with properties that contain a hypercube and inject it into the HTML element ‘nova01’:
nebbie.create({
type: 'nova-table',
}, {
element: document.getElementById('nova01'),
properties: {
qHyperCubeDef: {
qDimensions: [{
qDef: {
qFieldDefs: ['Customer']
}
}],
qMeasures: [{
qDef: {
qDef: 'Sum([Sales Amount])'
}
}],
qInitialDataFetch: [{
qWidth: 5,
qHeight: 1000
}]
}
}
});
And we get something like this:
A very basic page with a working app connection and displaying some data from our app using the visualization we developed earlier. The project is available here.
Qlik nebula.js – first test drive
Qlik recently has released quite a lot of open source software. One of the more interesting of those is nebulajs, described as ‘Product and framework agnostic integration APIs for Qlik’s Associative Engine’.
Nebulajs has three parts:
- supernova: A component model for Qlik Sense visualizations
- nucleus: A framework for injecting supernova’s into HTML pages
- cli: a command line interface for developing supernova’s
This somewhat resembles the current Qlik Sense model, where supernova’s have the same role as extensions, while nucleus replaces the mashup API.
My aim for this test drive is to try the supernova part by building a simple supernova, or visualization. The mashup part will have to wait for later.
Installation and setup
To use nebulajs you need nodejs version 8.0.0 or later and access to Qlik engine. Since I’ll be doing this on Windows, I will use Qlik Sense Desktop, which I have already installed. I also have the right version of nodejs, so I can install nebulajs globally and create my first project:
cd \src\nebula-test
npm install @nebula.js/cli@next -g
nebula create nova-table
When asked about picassojs template I select ‘none’ we will save picasso for another time. This will create a project for me. The project comes with build script configured to use modern javascript tools like babel and webpack.
The default for nebulajs projects is using Qlik Sense Docker, but I will be using Qlik Sense desktop, which means I have to add the Qlik Sense Desktop port number in package.json, under scripts:
"start": "nebula serve --enigma.port 4848",
Now we can start our project:
cd nova-table
npm run start
And a browser window will open with a listing of your Qlik Sense Desktop apps. Select one (I’m in the habit of using the demo app Consumer Sales, but almost anyone will do) and you will get a page with a selection bar at the top and your new supernova displayed in the middle of the screen:
Add some functionality
You find the source code of your new supernova in the src directory. Start with the file index.js, which is kind of the main program of your supernova. The component has six methods – to get something working we’ll start with two of them.
The mounted method is where you get the reference to your HTML element. Unlike in the current extension API you will get a plain HTML element, not a jQuery wrapper around it – remember supernova is platform agnostic. The default implementation just sets the content to Hello! – we’ll change that so that we keep a reference to the element (we’ll need that later) and set the content to ‘nova-table’ so that we know that our supernova is running (we’ll soon overwrite this).
mounted(element) {
this.element = element;
this.element.innerHTML = 'nova-table';
}
Next up is the render method, which plays the same role as the paint method in todays extension api. But parameters have changed: we do not get the element at this stage, that’s why we need to save it in mounted, and we get a second parameter, called context. While this feels like a good idea, there is not much inside the context yet, so we won’t use it yet. Instead we just add the layout to the element for now, so we can see what we have got:
render({
layout,
context,
}) {
this.element.innerHTML = JSON.stringify(layout);
console.log('render', this, layout, context);
}
Add a hypercube to get some data
Next step is to add a hypercube to get some data into your supernove. To do this we need to edit object-properties.js and add a hypercube definition:
qHyperCubeDef: {
qDimensions: [],
qMeasures: [],
qInitialDataFetch: [{
qWidth: 10,
qHeight: 50,
}],
},
When you have done this your supernova will have a hypercube, which you can see in the display, but it will be empty. In Qlik Sense we would use the property panel to add some dimensions and measures, but there is no property panel yet in nebulajs. We will have to edit the properties by hand. If you click the ‘Props’ button you will get the properties (including the empty hypercube definition). Add a dimension to the dimensions array, something like this:
{"qDef":{"qFieldDefs":["Invoice Number"]}}
Note that this is JSON format, so you need to enclose qDef and qFieldDefs in “. When you’ve done this, you will get a hypercube with contents, and it will look something like this:
Obviously it’s time to implement the rendering.
Basic rendering
For this test drive we will just generate some basic HTML from the hypercube data. The framework supports, even forces you to (in it’s standard setup) to use modern javascript with templates etc. That means we can generate a simple HTML table with just a few lines of code, like this:
render({
layout,
context,
}) {
const hypercube = layout.qHyperCube;
let html = '<table><thead><tr>';
html += hypercube.qDimensionInfo.map(d => `<th>${d.qFallbackTitle}</th>`).join('');
html += hypercube.qMeasureInfo.map(m => `<th>${m.qFallbackTitle}</th>`).join('');
html += '</tr></thead><tbody>';
html += hypercube.qDataPages[0].qMatrix.map(row => `<tr>${row.map(cell => `<td>${cell.qText}</td>`).join('')}</tr>`).join('');
html += '</tbody></table>';
this.element.innerHTML = html;
}
If this looks strange to you, you need to catch up with javascript features like template literals, arrow functions, Array.map and Array.join.
Now just add dimensions and measures to your properties:
"qDimensions": [{"qDef": {"qFieldDefs": ["Customer"] }}],
"qMeasures": [{"qDef":{"qDef":"Sum([Sales Amount])"}}]
And you will get a simple HTML table:
Working, but not something you would use. Let’s make it a bit better with some CSS.
Add some styling
As you might have noticed in the code, requirejs is no longer used, instead you use the import statement. That works for javascript files, but you can actually use it for css files. So we can simply add this line to our index.js:
import './style.css';
We also need a class for our root element, to make sure our styling does not affect other visualizations or the framwork. I’ve called this class ‘nova-table’ (as the extension) and added a root div with this class to the rendering code. And finally I have made sure that numeric fields get the css class ‘numeric’. These are the affected rows:
let html = '<div class="nova-table"><table><thead><tr>';
....
html += hypercube.qDataPages[0].qMatrix.map(row => `<tr>${row.map(cell => `<td${cell.qNum === 'NaN' ? '' : ' class="numeric"'}>${cell.qText}</td>`).join('')}</tr>`).join('');
html += '</tbody></table></div>';
And add a css file (style.css) where I set up some basic styling:
.nova-table {
font-family: "Source Sans Pro", "Segoe UI", "Helvetica Neue", -apple-system, Arial, sans-serif;
height: 100%;
overflow: auto;
}
.nova-table table td,
.nova-table table th {
text-align: left;
}
.nova-table table td.numeric {
text-align: right;
}
This gives us something like this:
Conclusion
Qlik nebulajs is a big step forward with it’s support for a modern javascript development environment and good key concepts. If you plan to work on Qlik visualizations in the future it’s definitely worth taking a look at. You can relatively easily get something up and running even though the framework is far from ready.
You find my test project here
Building on Qlik Sense bundled extensions
From the very start when Qlik Sense was launched a recurring question has been ‘how do I modify the xxxx chart and add feature yyyy’. That has never been supported and still isn’t. But with the visualizations provided by Qlik as in the extension bundles it’s another thing. These visualizations are bundled with the installation, but Qlik has also published the source at github.com/qlik-oss, and adding features to those are possible, even encouraged.
The multi-kpi extension
One of the extensions in the visualization bundle is the multi-kpi extension, originally written by Alexander Nerush and published under the very modest name SimpleKPI. Qlik has forked the repository, made some changes, written a few pages of help info and bundled it with Qlik Sense.
The original name SimpleKPI is pretty misleading: this is a powerful KPI object with lots of options and functionality many customers want. I encourage you to take a look at it if you haven’t already.
The requirement
One of these options is the possiblity to connect a sheet to the KPI, so that when the user clicks on the KPI, a sheet showing details behind that number opens. But sometimes you do not only want a sheet to open, but also some selections to be applied. So we want the possibility to connect not only a sheet, but also a bookmark to the KPI.
Adding this to the built-in KPI object is not possible, since you cannot modify it. Writing your own KPI object from scratch is of course possible, but would take some time (certainly if you want the features of the multi-kpi object), but adding it to the multi-kpi object is much easier.
Making the addition
To make the addition you do the following:
- make a fork of the multi-kpi repository
- run npm install to get the dependencies needed
- make your changes to the source code
- build and test like with any extension
My version of multi-kpi, with the link to bookmark feature is available here. If you feel your feature is of general interest, you might also create a pull request back to Qlik’s version, and let them decide if it’s something worth adding.
Add Sense Chrome 1.1.0: properties as patches, enable copy
Version 1.1.0 of my Add Sense Chrome Extension is now available in Chrome web store. This releas has two new features, one of which is pretty experimental.
Properties in patch mode
From the beginning the extension has allowed you to see properties as an object, that includes all properties. But sometimes you need properties in patch format, with the path to each property and the value. This is the format used in one of the most powerful API calls of the QIX Engine, the applyPatches call.
The list of patches you will see:
- includes all properties
- is sorted
- and can be filtered
Creating the paths for applyPatches calls is one use for it. Another is when you want to for example compare propertries of two objects, like the expressions used (are they the same??) or dimensions. In that case simply open properties for both objects, switch to patch mode and filter on what you are interested in.
Enable copy
Quite frequently you want to copy something from Qlik Sene and paste into a document or another program. This is hard to do. The Enable copy alternative helps you with that by simply turning on standard HTML text select. You can then select and copy text much like in a standard HTML page.
This works well in some cases, but in other it conflicts with the Qlik Sense UI and does not work at all, might even make some stuff unusable. So use it with care and do not have too high expectations.
Add Sense Chrome extension 1.0.2: IFRAME support
Version 1.0.2 of the Add Sense Chrome extension is now in the Chrome Web Store. This is a small update: no new features, the change is only that it now supports iframe’s.
This means that if you are using tools like Qlik Sense single or embedding Qlik Sense objects using iframe’s you can now use the extension to troubleshoot it much the same way as in other scenarios.
And that includes Qlik Sense CloudKit, which utses an iframe.
For those of you already using the extension the uppgraderas should be more or less automatic.
Checking performance with Add Sense Chrome extension
One of the uses for the Add Sense Chrome extension is to check performance in your Qlik Sense app or mashup. To help you with this the info boxes show you a few numbers, collected as you use the extension. When you select to show the boxes, it will start collecting statistics, and you will se something like this:
In this post I am using one of Qliks demo apps (you find it here, try it yourself!), so I don’t really expect to find any performance problems.
The Chrome extension will find all both objects defined in the app and session objects, but only ones that are shown on the page. For all such objects it will start collecting statistics:
- number of recalculations made
- latest calculation time in milliseconds
- maximum calculation time
In the top left corner you will see the handle, that’s a number that the QIX Engine assigns to objects as they are opened. If you want to see the actual messages sent for a specific object in the web socket log you need this number. Since the handles are assigned in sequence, you will also get a grip of how many objects you are using in your solution.
Now to try this lets make some selection. Open the selection tool and click on show in the extension menu (yes the extension works in the selection tool too).Make some selections and you will see something like this (excuse me, it’s in swedish):
Confirm your selections and close the selection tool. When you come back to the sheet probably only the app infobox will be visible. When Qlik Sense removes the objects from the page, the infoboxes disappear too. But the extension will continue to collect statistics, so click on show again in the extension menu, and you will se something like this:
Note that objects have only been recalculated once. That’s because the client only recalculate objects that are visible on the screen. The text object on the left should not be recalculated at all, since it was not affected by our selections.
So that’s what you need to do in a real situation:
- enable timing for objects on the screen with the show command
- make your selections, the extension wil register recalculations
- if objects have been off screen, reenable the infoboxes with the show command
- analyze the figures. Look for long calculation times or too many recalculations
Once you have started the timing, the only way to stop it is by refreshing the page, using the show command will not stop the monitoring.
Qlik Sense developer tool for Chrome now available
Some years ago I developed a Qlik Sense developer tool extension, mainly to help mashup developers with the problem of finding object id’s to use in their mashup. Over time we added more features to it, so you could now access script and variables, and use it as a tool (of several) for performance troubleshooting.
But it always had the problem that you needed to add the extension to the Qlik Sense app, something that is not always possible or practical. What if we could do this without needing to add anything to the app?
That’s the reasoning behind the Qlik Sense developer tool now available in the Chrome web store. It’s still an extension, but now a Chrome extension and not a Qlik Sense extension. That means if you install it in your browser, it will always be available.
Another advantage with this approach is that it also works with Qlik Sense mashups, using Qlik Sense Capabilities API.
Still you can find object id’s with the extension, and explore properties for objects, sheets and apps. You can also read the script (provided you have access rights) and the variables. And you can now see what extensions and charts are actually used in an app, something that is not easy with the standard client.
You find more information here.
Sharing session in Qlik Sense
One of the more interesting features of Qlik Sense is the session sharing. When you open a Qlik Sense app on your iPad that you have already open on your laptop your selections are already there. And when you make a change on the iPad, the laptop window updates. It feels a bit like magic!
Of course it isn’t magic. Qlik Sense automatically shares your sessions on different devices and rather than creating a new one when you connect from the iPad, it attaches to the one you already have on your laptop. Sessions are shared between browser tabs, browsers and devices. Sharing is made based on three factors:
- the app, of course. Sessions are app-based so the the app has to be the same for the connections to share the same sesion
- the authenticated user. sessions are not shared between users
- the URL used when connecting to the QIX engine
Controlling sharing
You might very well be using session sharing without really thinking about it. If you open the same app in more then one tab in the browser each tab has it’s own connection to the engine, and engine will connect them in just the same way as connections from different devices. Not that neither device or browser used affect the sharing, it’s just app, user and URL.
But sometimes you want to separate connections to have separate selection state. The way to do this is to make sure that the URL used to connect to QIX Engine is different. The standard URL used to connect to Engine has the format:
wss://[server][/proxy]/app/[app id]
- wss for secure websocket (ws for Qlik Sense Desktop)
- your server hostname or ip address, optionally with a port number (4848 for desktop)
- possibly a proxy if you are on a server
- and the app id
But you can add something more at the end, to make sure you separate your connections
An experiment
The Qlik Sense Client actually allows you to do this. You can add an identity to the end of the URL to separate sessions. Let’s make an experiment:
- Open an app (any app) in a browser window using the Qlik Sense client
- Open the same app in another browser window using a different device, a different browser or a different tab
- Verify that selections made in one of the windows affects the other – you have got one single session
- Now in one of the browser windows, modify the browser URL by adding ‘/identity/xxxx’ at the end and refresh
- Verify that selections made in one window does not affect the other
You have just managed to separate your connections into two different sessions! While this can be very useful to do with the standard client, for some solutions it is essential. You should really always consider this when you are building a Qlik Sense solution using the APIs.
Looking deeper
Using the browsers developer tools you can see how this is done. Open the developer tools for the browser where you added ‘/identity/xxxx’ at the end of the URL.
Click on the network tab, and filter on Web Sockets (WS). Find the connection to QIX Engine. You will see that the client has added ‘/identity/xxxx’ to the end, probably encoded to ‘%2Fidentity%2Fxxxx’.
Check the messages received on the web socket. At the begging you will find:
{“jsonrpc”:”2.0″,”method”:”OnConnected”,”params”:{“qSessionState”:”SESSION_CREATED”}}
which is a message from QIX Engine, saying that it has created a new session.
Now remove the ‘/identity/xxxx’ from the URL and refresh. Find the Web Socket again, now there is no ‘/identity/xxxx’ at the end. Check the messages again, and at the beginning you will find:
{“jsonrpc”:”2.0″,”method”:”OnConnected”,”params”:{“qSessionState”:”SESSION_ATTACHED”}}
which means that the QIX Engine found a matching session and instead of creating a new one, attached you to the existing, so you are again sharing sessions.