Chrome developer tools in Qlik Sense extension development (part 1)

Perhaps the most important tool for a Qlik Sense extension developer is Chrome developer tools. You can use any text editor to write your code, or even Qlik Sense dev-hub for simple extension, but you really need to use Developer Tools. Debug your code, check loading of javascript, HTML and CSS, inspect HTML structure and CSS rules – there is a lot you need to use them for. So good knowledge of them is essential.

Always run Qlik Sense in a browser

Qlik Sense Desktop is a good tool for your extension development in most scenarios. But you should never use the browser bundled with it for development. Instead use Chrome, and use http://localhost:4848/hub to access the hub. This means you will have the full developer tools available, and also that you will have an up-to-date Chrome installation.

While the browser bundled with Qlik Sense Desktop is built on the same codebase, it is not updated, but will contain an old version. It looks like the Chrome version bundled with Qlik Sense Desktop September 2019 is Chrome/47.0.2526.16 while my browser Chrome version is Chrome/78.0.3904.70, so Qlik Sense Desktop is way behind. BTW Qlik Sense Desktop also reports my Windows version as ‘Windows NT 6.1’ while Chrome proper reports the real value ‘Windows NT 10.0’.

So now that you are happily in Chrome you need to open the Developer Tools. There are different ways to do that:

  • press Shift+Ctrl i (for inspect, this is the method I use)
  • from Chrome menu in the top right corner, under ‘Other tools’

When doing extension development, always keep the console open. We’ll come to why later.

Disable caching

When doing extension development, you want Qlik Sense to allways load the latest version of your code. The browser normally tries to cache loaded files, and not request the file from the server again, it it already has it in memory. The first thing you should do is to disable this caching.It’s easy to do:

  • open developer console
  • switch to network tab
  • click the ‘Disable cache’ checkbox

The disable cache checkbox only affects the browser when the developer console is open. When the developer console is not open, the browser will still cache javascript files. This is why you should always open the browser console when your are working with extension development.

Check installed extensions (and mashups)

There is a lot of information in the network tab. You can se all network requests made by the Qlik Sense client, and since the client is load in chunks there will be a lot of them. There are also requests for images, CSS stylesheets and other requests. To help you filter the requests there is a bar:

Click on XHR and you will get a list of other calls. One of them is of special interest when you are doing extension development, the one called schema (url is really /qrs/extension/schema). Click on that one, and then on ‘Preview’ and you will get a list of all extensions and mashups installed on the system, including all data in the qext file:

This is a good place to check that your extension is installed, and what version (just remember to keep the version number in the qext file up to date). It’s also a good place to start when you are troubleshooting an extension you did not write.

Debugging the extension code

The ‘Sources’ tab is what you use for debugging. If the code is minified (many extensions are) you can click the curly braches in the bottom left corner to get a somewhat more readable format.

The toolbar in the upper right corner can be used to step through your code. The stop button will make the browser pause when an error occurrs, which is very useful to find bugs. Unfortunately it does not really work in extensions. That’s because Qlik Sense catches all extension errors (and silently ignores them).

So to have the browser pause on your errors, you need to check the box ‘Pause on caught exceptions’. If you do that, the browser will pause when an exception is thrown from your code. Unfortunately it will also break on some exceptions (in libraries) that are perfectly OK, so you can’t really turn it on always. Just turn it on when you think there might be an exception thrown from your code and then trigger a repaint, for example by resizin the window.

Check files in extension

The navigator on the left side in the sources tab shows you extension files in a tree structure. But it will only show you javascript files, so CSS and HTML files are not included. If you want to see those, switch to the network tab, and filter on the extension name.

Note that the qext file is not included. That’s because the qext file is not loaded by the client, the information is instead included in the schema.

and there’s more

Of course there is more, but I couldn’t cover everything in one post. Look out for part 2, available when I find the time to write it..

Qlik nebula.js – use in Qlik Sense

Previously I have written about Qliks new nebula.js framework: how to build a visualization and how you can use it in a web page. But what if you want to use it in the Qlik Sense built-in client?

Nebula.js is a new framework, built on an architecture that differs a lot from the old Qlik Sense client and has a new API. This means nebula.js-based visualization won’t work in Qlik Sense.

Add a Qlik Sense wrapper

So to run your visualization in Qlik Sense you need to wrap it in something that supports the Qlik Sense extension API. You also need to add the qext file. Luckily nebula.js can do that for you. Just type:

nebula sense

and nebulajs will wrap your visualization so it can be used as a Qlik Sense extension. You find the result in the [visualization]-ext directory, so for nova -table it will be nova-table-ext. Just copy the entire directory to your Extension directory (assuming you’re using Qlik Sense Desktop) and you will find your extension in the assets panel in sense:

and you can add your nebulaj.s visualization to your Qlik Sense app. You will also get a basic property panel, which works pretty well for the very basic nova-table.

Improving meta data

But the extension name is ‘nova-table’ and the icon used is the default extension icon and there is no description. You might be tempted to fix this in the generated qext file, but don’t do that! There is a better way. Start by getting some help by typing:

nebula sense -help

And you will get the following info:

This means we can add your meta info to a separate file and supply the filename as an argument to the ‘nebula sense’ command. So let’s create a file called meta.json and enter the following:

{
	"name": "Nebula table",
	"icon": "table",
	"description": "Nebula test table wrapped in a Qlik Sense extension"
}

Run the command:

nebula sense --meta meta.json

And copy the update -ext directory to youe Extension directory, overwriting the old version. You will see that the meta data has changed:

Working on the property panel

The wrapper also contains a default property panel, something you might want toimprove, for some visualizations you actually have to improve it to make the extension work. To do that we start with the default propertypanel. It’s in nova-table-ext/extDefinition.js, but it’s minified. To get it in a format that’s easier to work with we use the minify option to turn minification off (default is on):

nebula sense --minify false

And then copy the extDefinition.js file to the src directory. If you look at the file you can see a few things we could improve:

  • the settings section is commented out
  • features as export, exportData and snapshot are not supported

To make this work you also need to remove the define call the build script has wrapped around the definition (it will be added again when you build). So modify the file to:

const extDefinition = {
  definition: {
    type: 'items',
    component: 'accordion',
    items: {
      data: {
        uses: 'data',
      },
      settings: {
        uses: 'settings',
      },
    },
  },
  support: {
    export: true,
    exportData: true,
    snapshot: true,
    viewData: false,
  },
};

export default extDefinition;

And then run the build:

nebula sense --meta meta.json --ext src/extDefinition

Copy over the extension files again and you’ll se that you now have the ‘Appearance’ section (called ‘settings’ in the code) and export etc now works.

Guide to Qlik Sense ApplyPatches

One of the more powerful calls in the QIX Api is the ApplyPatches call. It allows you to dynamically add, change or remove basically any property of a generic object. With it, you can allow users to dynamically change visualizations: reorder data, change dimensions and measures, add limitation etc etc. But using it in your implementation can be tricky.

Parameters

The ApplyPatches call only has two parameters (plus the handle, identifying which object we are patching:

NameDescriptionMandatoryType
qPatchesArray of patches.YesArray of NxPatch
qSoftPatchPatches are not savedNoBoolean

So it’s an arry and a boolean (true/false) flag. Let’s start with the flag. Setting the qSoftPatch flag to true does mean that the patched property is not saved, but it also has some other consequences:

  • you can change objects that you otherwise cannot, like published objects, or objects in apps you cannot change
  • the changes will only affect you, other users will not see them, they might very well apply a completely different patch to the same object
  • the changes will only affect the current session. If you disconnect and connect again, they will be lost and you will start over with the original version of the object

So if your goal is to allow users to dynamically change visualizations you would want qSoftPatch to be true. While the default value false works OK in Qlik Sense Desktop, it’s generally not something you would want to use in a server environment.

The array of patches

The array is a bit more difficult. It’s an array so you can change several properties with one call. For example if you change dimension or measure you should consider changing the title too, and probably do that in the same call.  Every entry(almost) in the array has three parameters:

NameDescriptionType
qOpOperation to perform. Add, remove or replace.String
qPathPath to the property.String
qValueThe value of the propertyString

The first parameter, qOp, is an easy one. In most cases you want to change a property, use ‘replace’. Occasionaly you want to add a property, use ‘add’. Rarely you want to remove a property, in that case use ‘remove’. 

The second parameter, qPath, is the path to the property, using / between the property names (not . as in javascript). Remember:

  • you’re patching properties, not the layout, so it’s /qHyperCubeDef/… not qHyperCube
  • for arrays you need to include the position in the array, and numbering starts with zero, so first dimension is /qHyperCubeDef/qDimensions/0/ etc
  • property structures can be complicated, speciallly the hypercube..
  • you can use my Chrome extension Add Sense to create the path argument. Show properties for the objects, switch to patch mode, copy the path

The second parameter, qPath, is the path to the property, using / between the property names (not . as in javascript). Remember:

  • you’re patching properties, not the layout, so it’s /qHyperCubeDef/… not qHyperCube
  • for arrays you need to include the position in the array, and numbering starts with zero, so first dimension is /qHyperCubeDef/qDimensions/0/ etc
  • property structures can be complicated, speciallly the hypercube..
  • you can use my Chrome extension Add Sense to create the path argument. Show properties for the objects, switch to patch mode

Some examples

A common case is when you want your users to be able to change the ordering of for example a table dynamically. Ordering of a hypercube is determined by the qInterColumnSortOrder property, an array with the column numbers in the order they should be used for sorting. If you want say the third column (which has number 2, since numbering starts with 0) you move the number 2 to the beginnging of the array and call applyPatches:

model.applyPatches( [{
  qPath: '/qHyperCubeDef/qInterColumnSortOrder',
  qOp: 'replace',
  qValue: JSON.stringify(sortorder)
}], true );

You can find a more complete example in the mashup I showed at Qonnections back in 2015.

Another example is when you want to reverse the ordering of a hypercube. You do this by toggling the boolean value of qReverseSort in the dimension or measure where you want to change the order. For the first dimension this will be:

model.applyPatches( [{
  qPath: '/qHyperCubeDef/qDimensions/0/qDef/qReverseSort',
  qOp: 'replace',
  qValue: JSON.stringify( !reversesort )
}], true );

Qlik nebula.js – first test drive

Qlik recently has released quite a lot of open source software. One of the more interesting of those is nebulajs, described as ‘Product and framework agnostic integration APIs for Qlik’s Associative Engine’.

Nebulajs has three parts:

  • supernova: A component model for Qlik Sense visualizations
  • nucleus: A framework for injecting supernova’s into HTML pages
  • cli: a command line interface for developing supernova’s

This somewhat resembles the current Qlik Sense model, where supernova’s have the same role as extensions, while nucleus replaces the mashup API.

My aim for this test drive is to try the supernova part by building a simple supernova, or visualization. The mashup part will have to wait for later.

Installation and setup

To use nebulajs you need nodejs version 8.0.0 or later and access to Qlik engine. Since I’ll be doing this on Windows, I will use Qlik Sense Desktop, which I have already installed. I also have the right version of nodejs, so I can install nebulajs globally and create my first project:

cd \src\nebula-test
npm install @nebula.js/cli@next -g
nebula create nova-table

When asked about picassojs template I select ‘none’ we will save picasso for another time. This will create a project for me. The project comes with build script configured to use modern javascript tools like babel and webpack.

The default for nebulajs projects is using Qlik Sense Docker, but I will be using Qlik Sense desktop, which means I have to add the Qlik Sense Desktop port number in package.json, under scripts:

"start": "nebula serve --enigma.port 4848",

Now we can start our project:

cd nova-table
npm run start

And a browser window will open with a listing of your Qlik Sense Desktop apps. Select one (I’m in the habit of using the demo app Consumer Sales, but almost anyone will do) and you will get a page with a selection bar at the top and your new supernova displayed in the middle of the screen:

Supernova starting point

Add some functionality

You find the source code of your new supernova in the src directory. Start with the file index.js, which is kind of the main program of your supernova. The component has six methods – to get something working we’ll start with two of them.

The mounted method is where you get the reference to your HTML element. Unlike in the current extension API you will get a plain HTML element, not a jQuery wrapper around it – remember supernova is platform agnostic. The default implementation just sets the content to Hello! – we’ll change that so that we keep a reference to the element (we’ll need that later) and set the content to ‘nova-table’ so that we know that our supernova is running (we’ll soon overwrite this).

mounted(element) {
   this.element = element;
   this.element.innerHTML = 'nova-table';
}

Next up is the render method, which plays the same role as the paint method in todays extension api. But parameters have changed: we do not get the element at this stage, that’s why we need to save it in mounted, and we get a second parameter, called context. While this feels like a good idea, there is not much inside the context yet, so we won’t use it yet. Instead we just add the layout to the element for now, so we can see what we have got:

render({
    layout,
    context,
}) {
    this.element.innerHTML = JSON.stringify(layout);
    console.log('render', this, layout, context);
}

Add a hypercube to get some data

Next step is to add a hypercube to get some data into your supernove. To do this we need to edit object-properties.js and add a hypercube definition:

qHyperCubeDef: {
    qDimensions: [],
    qMeasures: [],
    qInitialDataFetch: [{
      qWidth: 10,
      qHeight: 50,
    }],
  },

When you have done this your supernova will have a hypercube, which you can see in the display, but it will be empty. In Qlik Sense we would use the property panel to add some dimensions and measures, but there is no property panel yet in nebulajs. We will have to edit the properties by hand. If you click the ‘Props’ button you will get the properties (including the empty hypercube definition). Add a dimension to the dimensions array, something like this:

{"qDef":{"qFieldDefs":["Invoice Number"]}}

Note that this is JSON format, so you need to enclose qDef and qFieldDefs in “. When you’ve done this, you will get a hypercube with contents, and it will look something like this:

Obviously it’s time to implement the rendering.

Basic rendering

For this test drive we will just generate some basic HTML from the hypercube data. The framework supports, even forces you to (in it’s standard setup) to use modern javascript with templates etc. That means we can generate a simple HTML table with just a few lines of code, like this:

render({
        layout,
        context,
      }) {
        const hypercube = layout.qHyperCube;
        let html = '<table><thead><tr>';
        html += hypercube.qDimensionInfo.map(d => `<th>${d.qFallbackTitle}</th>`).join('');
        html += hypercube.qMeasureInfo.map(m => `<th>${m.qFallbackTitle}</th>`).join('');
        html += '</tr></thead><tbody>';
        html += hypercube.qDataPages[0].qMatrix.map(row => `<tr>${row.map(cell => `<td>${cell.qText}</td>`).join('')}</tr>`).join('');
        html += '</tbody></table>';
        this.element.innerHTML = html;
}

If this looks strange to you, you need to catch up with javascript features like template literals, arrow functions, Array.map and Array.join.

Now just add dimensions and measures to your properties:


"qDimensions": [{"qDef": {"qFieldDefs": ["Customer"] }}],
"qMeasures": [{"qDef":{"qDef":"Sum([Sales Amount])"}}]

And you will get a simple HTML table:

Working, but not something you would use. Let’s make it a bit better with some CSS.

Add some styling

As you might have noticed in the code, requirejs is no longer used, instead you use the import statement. That works for javascript files, but you can actually use it for css files. So we can simply add this line to our index.js:

import './style.css';

We also need a class for our root element, to make sure our styling does not affect other visualizations or the framwork. I’ve called this class ‘nova-table’ (as the extension) and added a root div with this class to the rendering code. And finally I have made sure that numeric fields get the css class ‘numeric’. These are the affected rows:

let html = '<div class="nova-table"><table><thead><tr>';
....
html += hypercube.qDataPages[0].qMatrix.map(row => `<tr>${row.map(cell => `<td${cell.qNum === 'NaN' ? '' : ' class="numeric"'}>${cell.qText}</td>`).join('')}</tr>`).join('');
html += '</tbody></table></div>';   
        

And add a css file (style.css) where I set up some basic styling:

.nova-table {
    font-family: "Source Sans Pro", "Segoe UI", "Helvetica Neue", -apple-system, Arial, sans-serif;
    height: 100%;
    overflow: auto;
}

.nova-table table td,
.nova-table table th {
    text-align: left;
}
.nova-table table td.numeric {
    text-align: right;
}

This gives us something like this:

Conclusion

Qlik nebulajs is a big step forward with it’s support for a modern javascript development environment and good key concepts. If you plan to work on Qlik visualizations in the future it’s definitely worth taking a look at. You can relatively easily get something up and running even though the framework is far from ready.

You find my test project here

Building on Qlik Sense bundled extensions

From the very start when Qlik Sense was launched a recurring question has been ‘how do I modify the xxxx chart and add feature yyyy’. That has never been supported and still isn’t. But with the visualizations provided by Qlik as in the extension bundles it’s another thing. These visualizations are bundled with the installation, but Qlik has also published the source at github.com/qlik-oss, and adding features to those are possible, even encouraged.

The multi-kpi extension

One of the extensions in the visualization bundle is the multi-kpi extension, originally written by Alexander Nerush and published under the very modest name SimpleKPI. Qlik has forked the repository, made some changes, written a few pages of help info and bundled it with Qlik Sense.

The original name SimpleKPI is pretty misleading: this is a powerful KPI object with lots of options and functionality many customers want. I encourage you to take a look at it if you haven’t already.

The requirement

One of these options is the possiblity to connect a sheet to the KPI, so that when the user clicks on the KPI, a sheet showing details behind that number opens. But sometimes you do not only want a sheet to open, but also some selections to be applied. So we want the possibility to connect not only a sheet, but also a bookmark to the KPI.

Adding this to the built-in KPI object is not possible, since you cannot modify it. Writing your own KPI object from scratch is of course possible, but would take some time (certainly if you want the features of the multi-kpi object), but adding it to the multi-kpi object is much easier.

Making the addition

To make the addition you do the following:

  • make a fork of the multi-kpi repository
  • run npm install to get the dependencies needed
  • make your changes to the source code
  • build and test like with any extension

My version of multi-kpi, with the link to bookmark feature is available here. If you feel your feature is of general interest, you might also create a pull request back to Qlik’s version, and let them decide if it’s something worth adding.

Add Sense Chrome 1.1.0: properties as patches, enable copy

Version 1.1.0 of my Add Sense Chrome Extension is now available in Chrome web store. This releas has two new features, one of which is pretty experimental.

Properties in patch mode

Properties in patch mode

From the beginning the extension has allowed you to see properties as an object, that includes all properties. But sometimes you need properties in patch format, with the path to each property and the value. This is the format used in one of the most powerful API calls of the QIX Engine, the applyPatches call.

The list of patches you will see:

  • includes all properties
  • is sorted
  • and can be filtered

Creating the paths for applyPatches calls is one use for it. Another is when you want to for example compare propertries of two objects, like the expressions used (are they the same??) or dimensions. In that case simply open properties for both objects, switch to patch mode and filter on what you are interested in.

Enable copy

Quite frequently you want to copy something from Qlik Sene and paste into a document or another program. This is hard to do. The Enable copy alternative helps you with that by simply turning on standard HTML text select. You can then select and copy text much like in a standard HTML page.

This works well in some cases, but in other it conflicts with the Qlik Sense UI and does not work at all, might even make some stuff unusable. So use it with care and do not have too high expectations.

Checking performance with Add Sense Chrome extension

One of the uses for the Add Sense Chrome extension is to check performance in your Qlik Sense app or mashup. To help you with this the info boxes show you a few numbers, collected as you use the extension. When you select to show the boxes, it will start collecting statistics, and you will se something like this:

Initial display with no values calculated

In this post I am using one of Qliks demo apps (you find it here, try it yourself!), so I don’t really expect to find any performance problems.

The Chrome extension will find all both objects defined in the app and session objects, but only ones that are shown on the page. For all such objects it will start collecting statistics:

  • number of recalculations made
  • latest calculation time in milliseconds
  • maximum calculation time

In the top left corner you will see the handle, that’s a number that the QIX Engine assigns to objects as they are opened. If you want to see the actual messages sent for a specific object in the web socket log you need this number. Since the handles are assigned in sequence, you will also get a grip of how many objects you are using in your solution.

Now to try this lets make some selection. Open the selection tool and click on show in the extension menu (yes the extension works in the selection tool too).Make some selections and you will see something like this (excuse me, it’s in swedish):

Selection tool with the Chrome extension enabled

Confirm your selections and close the selection tool. When you come back to the sheet probably only the app infobox will be visible. When Qlik Sense removes the objects from the page, the infoboxes disappear too. But the extension will continue to collect statistics, so click on show again in the extension menu, and you will se something like this:

Calculations have been made

Note that objects have only been recalculated once. That’s because the client only recalculate objects that are visible on the screen. The text object on the left should not be recalculated at all, since it was not affected by our selections.

So that’s what you need to do in a real situation:

  1. enable timing for objects on the screen with the show command
  2. make your selections, the extension wil register recalculations
  3. if objects have been off screen, reenable the infoboxes with the show command
  4. analyze the figures. Look for long calculation times or too many recalculations

Once you have started the timing, the only way to stop it is by refreshing the page, using the show command will not stop the monitoring.

Qlik Sense developer tool for Chrome now available

The Chrome extension activated in the built-in Qlik Sense client

Some years ago I developed a Qlik Sense developer tool extension, mainly to help mashup developers with the problem of finding object id’s to use in their mashup. Over time we added more features to it, so you could now access script and variables, and use it as a tool (of several) for performance troubleshooting.

But it always had the problem that you needed to add the extension to the Qlik Sense app, something that is not always possible or practical. What if we could do this without needing to add anything to the app?

That’s the reasoning behind the Qlik Sense developer tool now available in the Chrome web store. It’s still an extension, but now a Chrome extension and not a Qlik Sense extension. That means if you install it in your browser, it will always be available.

Another advantage with this approach is that it also works with Qlik Sense mashups, using Qlik Sense Capabilities API.

The Chrome extension activated in a mashup

Still you can find object id’s with the extension, and explore properties for objects, sheets and apps. You can also read the script (provided you have access rights) and the variables. And you can now see what extensions and charts are actually used in an app, something that is not easy with the standard client.

You find more information here.

qsVariable now archived

After four years I have now archived my GitHub repository for the qsVariable extension.

The extension is now included in the dashboard bundle, available with the Qlik Sense installation, and available in Qlik Open Source repositories. It is also supported by Qlik.

This has been a fun project to work on, the functionality it brings was something many customers wanted, and it quickly became one of the most commonly used extensions. It was first realeased over four years ago, in the first days of 2015.

The fact that Qlik now delivers a set of extensions with the product, and supports them, is of course great for customers. One of the few frustrating periods during the last years was when there was a Qlik Sense version that actually broke the extension. That will not happen again.

Thanks to everybody who has contributed to the project with suggestions and in some cases even some code!