Previously I have written about Qliks new nebula.js framework: how to build a visualization and how you can use it in a web page. But what if you want to use it in the Qlik Sense built-in client?

Nebula.js is a new framework, built on an architecture that differs a lot from the old Qlik Sense client and has a new API. This means nebula.js-based visualization won’t work in Qlik Sense.

Add a Qlik Sense wrapper

So to run your visualization in Qlik Sense you need to wrap it in something that supports the Qlik Sense extension API. You also need to add the qext file. Luckily nebula.js can do that for you. Just type:

nebula sense

and nebulajs will wrap your visualization so it can be used as a Qlik Sense extension. You find the result in the [visualization]-ext directory, so for nova -table it will be nova-table-ext. Just copy the entire directory to your Extension directory (assuming you’re using Qlik Sense Desktop) and you will find your extension in the assets panel in sense:

and you can add your nebulaj.s visualization to your Qlik Sense app. You will also get a basic property panel, which works pretty well for the very basic nova-table.

Improving meta data

But the extension name is ‘nova-table’ and the icon used is the default extension icon and there is no description. You might be tempted to fix this in the generated qext file, but don’t do that! There is a better way. Start by getting some help by typing:

nebula sense -help

And you will get the following info:

This means we can add your meta info to a separate file and supply the filename as an argument to the ‘nebula sense’ command. So let’s create a file called meta.json and enter the following:

{
	"name": "Nebula table",
	"icon": "table",
	"description": "Nebula test table wrapped in a Qlik Sense extension"
}

Run the command:

nebula sense --meta meta.json

And copy the update -ext directory to youe Extension directory, overwriting the old version. You will see that the meta data has changed:

Working on the property panel

The wrapper also contains a default property panel, something you might want toimprove, for some visualizations you actually have to improve it to make the extension work. To do that we start with the default propertypanel. It’s in nova-table-ext/extDefinition.js, but it’s minified. To get it in a format that’s easier to work with we use the minify option to turn minification off (default is on):

nebula sense --minify false

And then copy the extDefinition.js file to the src directory. If you look at the file you can see a few things we could improve:

  • the settings section is commented out
  • features as export, exportData and snapshot are not supported

To make this work you also need to remove the define call the build script has wrapped around the definition (it will be added again when you build). So modify the file to:

const extDefinition = {
  definition: {
    type: 'items',
    component: 'accordion',
    items: {
      data: {
        uses: 'data',
      },
      settings: {
        uses: 'settings',
      },
    },
  },
  support: {
    export: true,
    exportData: true,
    snapshot: true,
    viewData: false,
  },
};

export default extDefinition;

And then run the build:

nebula sense --meta meta.json --ext src/extDefinition

Copy over the extension files again and you’ll se that you now have the ‘Appearance’ section (called ‘settings’ in the code) and export etc now works.

patchwork

One of the more powerful calls in the QIX Api is the ApplyPatches call. It allows you to dynamically add, change or remove basically any property of a generic object. With it, you can allow users to dynamically change visualizations: reorder data, change dimensions and measures, add limitation etc etc. But using it in your implementation can be tricky.

Parameters

The ApplyPatches call only has two parameters (plus the handle, identifying which object we are patching:

NameDescriptionMandatoryType
qPatchesArray of patches.YesArray of NxPatch
qSoftPatchPatches are not savedNoBoolean

So it’s an arry and a boolean (true/false) flag. Let’s start with the flag. Setting the qSoftPatch flag to true does mean that the patched property is not saved, but it also has some other consequences:

  • you can change objects that you otherwise cannot, like published objects, or objects in apps you cannot change
  • the changes will only affect you, other users will not see them, they might very well apply a completely different patch to the same object
  • the changes will only affect the current session. If you disconnect and connect again, they will be lost and you will start over with the original version of the object

So if your goal is to allow users to dynamically change visualizations you would want qSoftPatch to be true. While the default value false works OK in Qlik Sense Desktop, it’s generally not something you would want to use in a server environment.

The array of patches

The array is a bit more difficult. It’s an array so you can change several properties with one call. For example if you change dimension or measure you should consider changing the title too, and probably do that in the same call.  Every entry(almost) in the array has three parameters:

NameDescriptionType
qOpOperation to perform. Add, remove or replace.String
qPathPath to the property.String
qValueThe value of the propertyString

The first parameter, qOp, is an easy one. In most cases you want to change a property, use ‘replace’. Occasionaly you want to add a property, use ‘add’. Rarely you want to remove a property, in that case use ‘remove’. 

The second parameter, qPath, is the path to the property, using / between the property names (not . as in javascript). Remember:

  • you’re patching properties, not the layout, so it’s /qHyperCubeDef/… not qHyperCube
  • for arrays you need to include the position in the array, and numbering starts with zero, so first dimension is /qHyperCubeDef/qDimensions/0/ etc
  • property structures can be complicated, speciallly the hypercube..
  • you can use my Chrome extension Add Sense to create the path argument. Show properties for the objects, switch to patch mode, copy the path

The second parameter, qPath, is the path to the property, using / between the property names (not . as in javascript). Remember:

  • you’re patching properties, not the layout, so it’s /qHyperCubeDef/… not qHyperCube
  • for arrays you need to include the position in the array, and numbering starts with zero, so first dimension is /qHyperCubeDef/qDimensions/0/ etc
  • property structures can be complicated, speciallly the hypercube..
  • you can use my Chrome extension Add Sense to create the path argument. Show properties for the objects, switch to patch mode

Some examples

A common case is when you want your users to be able to change the ordering of for example a table dynamically. Ordering of a hypercube is determined by the qInterColumnSortOrder property, an array with the column numbers in the order they should be used for sorting. If you want say the third column (which has number 2, since numbering starts with 0) you move the number 2 to the beginnging of the array and call applyPatches:

model.applyPatches( [{
  qPath: '/qHyperCubeDef/qInterColumnSortOrder',
  qOp: 'replace',
  qValue: JSON.stringify(sortorder)
}], true );

You can find a more complete example in the mashup I showed at Qonnections back in 2015.

Another example is when you want to reverse the ordering of a hypercube. You do this by toggling the boolean value of qReverseSort in the dimension or measure where you want to change the order. For the first dimension this will be:

model.applyPatches( [{
  qPath: '/qHyperCubeDef/qDimensions/0/qDef/qReverseSort',
  qOp: 'replace',
  qValue: JSON.stringify( !reversesort )
}], true );

Nucleus

A while ago I did my first test project with Qlik’s new open source library nebula.js (it’s available here). That time I tested the visualization part, corresponding to extensions in the current product (but hopefully used also for visualizations delivered by Qlik). Now it’s time to try it from the other side, building a web app with these new kind of visualizations.

Terminology first: the term mostly used in Qlik Sense is mashup. That’s because the current javascript API was originally designed for a mashup scenario, where you wanted to add Qlik Sense objects to an existing web page. But what it’s mostly used for today is building web apps, that is pages that include only Qlik Sense objects. And that is what I intend to do in this small test project.

Setting up

Nebula.js includes a small example app. You find it here. It’s just a few files, since it actually downloads the libraries (enigmajs, nebulajs) over the internet. So if we just copy those files to a local directory we have got a starting point.

The example uses a docker installation and a session app, but I want to use my Qlik Sense Desktop and the Consumer Sales app. To fix that I revised (and simplified) the connect.js file so it looks like this:

window.connect = function connect(host, id) {
	return fetch('https://unpkg.com/enigma.js/schemas/3.2.json').
	then(response => response.json()).
	then(schema => window.enigma.create({
		schema,
		url: `ws://${host}/app/${id}`,
	}).open().then(qix => qix.openDoc(id)));
};

And in the index.js file add host and app:

connect('localhost:4848','Consumer Sales.qvf').then((app) => {

and we have got something that’s working. Start Qlik Sense desktop, make sure your test project is served (I use serve), and open the file in the browser. You’ll se a selection toolbar, and an area that displays the simple supernova that’s included:

If you wonder how come my selection toolbar shows selections, that’s because I opened the same app with the same url(use http://localhost:4848/sense/app/Consumer%20Sales.qvf ) and made a selection. Session sharing does the rest.

Load nova-table

OK, so now we have got a page using nebulajs and connecting to an app, and the selction toolbar works. But how do we use supernovas like the one I made in my first test drive? Well first we need to make it available in our project. Just copy the files from the dist directory in nova-table and put it in your project. I have used a sub-directory called novas, since there is actually nothing super about nova-table. You can then load it in index.html:

  <script src="./novas/nova-table.js"></script>
  <script src="connect.js"></script>
  <script src="index.js"></script>

It needs to be before index.js for this to work.

Unlike the current Qlik Sense javascript libraries, nebulajs is not bundled with a module management library like requirejs. Instead it will see if there is a module manager available and use it if there is. If not it will simply add the module as a global function. So we can access it simply as window[‘nova-table’]. In a real project we would not really want everything defined globally, but this is just a quick test.

To make a solution that can be extended with more visualization, we create a novas object, where we can put all our visualizations, and then grab the right one using the type, with the sn from the initial example as the default:

const novas = {
 'nova-table': window['nova-table']()
};
const sn = {
 component: {
  mounted(element) {
   element.textContent = 'Hello';
  },
 },
};

const nebbie = window.nucleus(app, {
 load: (type, config) => config.Promise.resolve(novas[type.name] || sn),
});

Add a table

So now we have got the nova-table module loaded, let’s create one and add to the page. We’ll make a session object, with the definition in the javascript file, so first we change the HTML file to:

  <div class="content">
    <div class="toolbar"></div>
    <div class="object" id="nova01"></div>
  </div>

And in the javascript file we use the create method to create a nova-table object, with properties that contain a hypercube and inject it into the HTML element ‘nova01’:

nebbie.create({
 type: 'nova-table',
}, {
 element: document.getElementById('nova01'),
 properties: {
  qHyperCubeDef: {
   qDimensions: [{
    qDef: {
     qFieldDefs: ['Customer']
    }
   }],
   qMeasures: [{
    qDef: {
     qDef: 'Sum([Sales Amount])'
    }
   }],
   qInitialDataFetch: [{
    qWidth: 5,
    qHeight: 1000
   }]
  }
 }
});

And we get something like this:

A very basic page with a working app connection and displaying some data from our app using the visualization we developed earlier. The project is available here.

A nebula

Qlik recently has released quite a lot of open source software. One of the more interesting of those is nebulajs, described as ‘Product and framework agnostic integration APIs for Qlik’s Associative Engine’.

Nebulajs has three parts:

  • supernova: A component model for Qlik Sense visualizations
  • nucleus: A framework for injecting supernova’s into HTML pages
  • cli: a command line interface for developing supernova’s

This somewhat resembles the current Qlik Sense model, where supernova’s have the same role as extensions, while nucleus replaces the mashup API.

My aim for this test drive is to try the supernova part by building a simple supernova, or visualization. The mashup part will have to wait for later.

Installation and setup

To use nebulajs you need nodejs version 8.0.0 or later and access to Qlik engine. Since I’ll be doing this on Windows, I will use Qlik Sense Desktop, which I have already installed. I also have the right version of nodejs, so I can install nebulajs globally and create my first project:

cd \src\nebula-test
npm install @nebula.js/cli@next -g
nebula create nova-table

When asked about picassojs template I select ‘none’ we will save picasso for another time. This will create a project for me. The project comes with build script configured to use modern javascript tools like babel and webpack.

The default for nebulajs projects is using Qlik Sense Docker, but I will be using Qlik Sense desktop, which means I have to add the Qlik Sense Desktop port number in package.json, under scripts:

"start": "nebula serve --enigma.port 4848",

Now we can start our project:

cd nova-table
npm run start

And a browser window will open with a listing of your Qlik Sense Desktop apps. Select one (I’m in the habit of using the demo app Consumer Sales, but almost anyone will do) and you will get a page with a selection bar at the top and your new supernova displayed in the middle of the screen:

Supernova starting point

Add some functionality

You find the source code of your new supernova in the src directory. Start with the file index.js, which is kind of the main program of your supernova. The component has six methods – to get something working we’ll start with two of them.

The mounted method is where you get the reference to your HTML element. Unlike in the current extension API you will get a plain HTML element, not a jQuery wrapper around it – remember supernova is platform agnostic. The default implementation just sets the content to Hello! – we’ll change that so that we keep a reference to the element (we’ll need that later) and set the content to ‘nova-table’ so that we know that our supernova is running (we’ll soon overwrite this).

mounted(element) {
   this.element = element;
   this.element.innerHTML = 'nova-table';
}

Next up is the render method, which plays the same role as the paint method in todays extension api. But parameters have changed: we do not get the element at this stage, that’s why we need to save it in mounted, and we get a second parameter, called context. While this feels like a good idea, there is not much inside the context yet, so we won’t use it yet. Instead we just add the layout to the element for now, so we can see what we have got:

render({
    layout,
    context,
}) {
    this.element.innerHTML = JSON.stringify(layout);
    console.log('render', this, layout, context);
}

Add a hypercube to get some data

Next step is to add a hypercube to get some data into your supernove. To do this we need to edit object-properties.js and add a hypercube definition:

qHyperCubeDef: {
    qDimensions: [],
    qMeasures: [],
    qInitialDataFetch: [{
      qWidth: 10,
      qHeight: 50,
    }],
  },

When you have done this your supernova will have a hypercube, which you can see in the display, but it will be empty. In Qlik Sense we would use the property panel to add some dimensions and measures, but there is no property panel yet in nebulajs. We will have to edit the properties by hand. If you click the ‘Props’ button you will get the properties (including the empty hypercube definition). Add a dimension to the dimensions array, something like this:

{"qDef":{"qFieldDefs":["Invoice Number"]}}

Note that this is JSON format, so you need to enclose qDef and qFieldDefs in “. When you’ve done this, you will get a hypercube with contents, and it will look something like this:

Obviously it’s time to implement the rendering.

Basic rendering

For this test drive we will just generate some basic HTML from the hypercube data. The framework supports, even forces you to (in it’s standard setup) to use modern javascript with templates etc. That means we can generate a simple HTML table with just a few lines of code, like this:

render({
        layout,
        context,
      }) {
        const hypercube = layout.qHyperCube;
        let html = '<table><thead><tr>';
        html += hypercube.qDimensionInfo.map(d => `<th>${d.qFallbackTitle}</th>`).join('');
        html += hypercube.qMeasureInfo.map(m => `<th>${m.qFallbackTitle}</th>`).join('');
        html += '</tr></thead><tbody>';
        html += hypercube.qDataPages[0].qMatrix.map(row => `<tr>${row.map(cell => `<td>${cell.qText}</td>`).join('')}</tr>`).join('');
        html += '</tbody></table>';
        this.element.innerHTML = html;
}

If this looks strange to you, you need to catch up with javascript features like template literals, arrow functions, Array.map and Array.join.

Now just add dimensions and measures to your properties:


"qDimensions": [{"qDef": {"qFieldDefs": ["Customer"] }}],
"qMeasures": [{"qDef":{"qDef":"Sum([Sales Amount])"}}]

And you will get a simple HTML table:

Working, but not something you would use. Let’s make it a bit better with some CSS.

Add some styling

As you might have noticed in the code, requirejs is no longer used, instead you use the import statement. That works for javascript files, but you can actually use it for css files. So we can simply add this line to our index.js:

import './style.css';

We also need a class for our root element, to make sure our styling does not affect other visualizations or the framwork. I’ve called this class ‘nova-table’ (as the extension) and added a root div with this class to the rendering code. And finally I have made sure that numeric fields get the css class ‘numeric’. These are the affected rows:

let html = '<div class="nova-table"><table><thead><tr>';
....
html += hypercube.qDataPages[0].qMatrix.map(row => `<tr>${row.map(cell => `<td${cell.qNum === 'NaN' ? '' : ' class="numeric"'}>${cell.qText}</td>`).join('')}</tr>`).join('');
html += '</tbody></table></div>';   
        

And add a css file (style.css) where I set up some basic styling:

.nova-table {
    font-family: "Source Sans Pro", "Segoe UI", "Helvetica Neue", -apple-system, Arial, sans-serif;
    height: 100%;
    overflow: auto;
}

.nova-table table td,
.nova-table table th {
    text-align: left;
}
.nova-table table td.numeric {
    text-align: right;
}

This gives us something like this:

Conclusion

Qlik nebulajs is a big step forward with it’s support for a modern javascript development environment and good key concepts. If you plan to work on Qlik visualizations in the future it’s definitely worth taking a look at. You can relatively easily get something up and running even though the framework is far from ready.

You find my test project here

From the very start when Qlik Sense was launched a recurring question has been ‘how do I modify the xxxx chart and add feature yyyy’. That has never been supported and still isn’t. But with the visualizations provided by Qlik as in the extension bundles it’s another thing. These visualizations are bundled with the installation, but Qlik has also published the source at github.com/qlik-oss, and adding features to those are possible, even encouraged.

The multi-kpi extension

One of the extensions in the visualization bundle is the multi-kpi extension, originally written by Alexander Nerush and published under the very modest name SimpleKPI. Qlik has forked the repository, made some changes, written a few pages of help info and bundled it with Qlik Sense.

The original name SimpleKPI is pretty misleading: this is a powerful KPI object with lots of options and functionality many customers want. I encourage you to take a look at it if you haven’t already.

The requirement

One of these options is the possiblity to connect a sheet to the KPI, so that when the user clicks on the KPI, a sheet showing details behind that number opens. But sometimes you do not only want a sheet to open, but also some selections to be applied. So we want the possibility to connect not only a sheet, but also a bookmark to the KPI.

Adding this to the built-in KPI object is not possible, since you cannot modify it. Writing your own KPI object from scratch is of course possible, but would take some time (certainly if you want the features of the multi-kpi object), but adding it to the multi-kpi object is much easier.

Making the addition

To make the addition you do the following:

  • make a fork of the multi-kpi repository
  • run npm install to get the dependencies needed
  • make your changes to the source code
  • build and test like with any extension

My version of multi-kpi, with the link to bookmark feature is available here. If you feel your feature is of general interest, you might also create a pull request back to Qlik’s version, and let them decide if it’s something worth adding.

Version 1.1.0 of my Add Sense Chrome Extension is now available in Chrome web store. This releas has two new features, one of which is pretty experimental.

Properties in patch mode

Properties in patch mode

From the beginning the extension has allowed you to see properties as an object, that includes all properties. But sometimes you need properties in patch format, with the path to each property and the value. This is the format used in one of the most powerful API calls of the QIX Engine, the applyPatches call.

The list of patches you will see:

  • includes all properties
  • is sorted
  • and can be filtered

Creating the paths for applyPatches calls is one use for it. Another is when you want to for example compare propertries of two objects, like the expressions used (are they the same??) or dimensions. In that case simply open properties for both objects, switch to patch mode and filter on what you are interested in.

Enable copy

Quite frequently you want to copy something from Qlik Sene and paste into a document or another program. This is hard to do. The Enable copy alternative helps you with that by simply turning on standard HTML text select. You can then select and copy text much like in a standard HTML page.

This works well in some cases, but in other it conflicts with the Qlik Sense UI and does not work at all, might even make some stuff unusable. So use it with care and do not have too high expectations.

Version 1.0.2 of the Add Sense Chrome extension is now in the Chrome Web Store. This is a small update: no new features, the change is only that it now supports iframe’s.


Add Sense chrome extension used in a page with two iframe’s

This means that if you are using tools like Qlik Sense single or embedding Qlik Sense objects using iframe’s you can now use the extension to troubleshoot it much the same way as in other scenarios.

And that includes Qlik Sense CloudKit, which utses an iframe.

For those of you already using the extension the uppgraderas should be more or less automatic.

One of the uses for the Add Sense Chrome extension is to check performance in your Qlik Sense app or mashup. To help you with this the info boxes show you a few numbers, collected as you use the extension. When you select to show the boxes, it will start collecting statistics, and you will se something like this:

Initial display with no values calculated

In this post I am using one of Qliks demo apps (you find it here, try it yourself!), so I don’t really expect to find any performance problems.

The Chrome extension will find all both objects defined in the app and session objects, but only ones that are shown on the page. For all such objects it will start collecting statistics:

  • number of recalculations made
  • latest calculation time in milliseconds
  • maximum calculation time

In the top left corner you will see the handle, that’s a number that the QIX Engine assigns to objects as they are opened. If you want to see the actual messages sent for a specific object in the web socket log you need this number. Since the handles are assigned in sequence, you will also get a grip of how many objects you are using in your solution.

Now to try this lets make some selection. Open the selection tool and click on show in the extension menu (yes the extension works in the selection tool too).Make some selections and you will see something like this (excuse me, it’s in swedish):

Selection tool with the Chrome extension enabled

Confirm your selections and close the selection tool. When you come back to the sheet probably only the app infobox will be visible. When Qlik Sense removes the objects from the page, the infoboxes disappear too. But the extension will continue to collect statistics, so click on show again in the extension menu, and you will se something like this:

Calculations have been made

Note that objects have only been recalculated once. That’s because the client only recalculate objects that are visible on the screen. The text object on the left should not be recalculated at all, since it was not affected by our selections.

So that’s what you need to do in a real situation:

  1. enable timing for objects on the screen with the show command
  2. make your selections, the extension wil register recalculations
  3. if objects have been off screen, reenable the infoboxes with the show command
  4. analyze the figures. Look for long calculation times or too many recalculations

Once you have started the timing, the only way to stop it is by refreshing the page, using the show command will not stop the monitoring.

The Chrome extension activated in the built-in Qlik Sense client

Some years ago I developed a Qlik Sense developer tool extension, mainly to help mashup developers with the problem of finding object id’s to use in their mashup. Over time we added more features to it, so you could now access script and variables, and use it as a tool (of several) for performance troubleshooting.

But it always had the problem that you needed to add the extension to the Qlik Sense app, something that is not always possible or practical. What if we could do this without needing to add anything to the app?

That’s the reasoning behind the Qlik Sense developer tool now available in the Chrome web store. It’s still an extension, but now a Chrome extension and not a Qlik Sense extension. That means if you install it in your browser, it will always be available.

Another advantage with this approach is that it also works with Qlik Sense mashups, using Qlik Sense Capabilities API.

The Chrome extension activated in a mashup

Still you can find object id’s with the extension, and explore properties for objects, sheets and apps. You can also read the script (provided you have access rights) and the variables. And you can now see what extensions and charts are actually used in an app, something that is not easy with the standard client.

You find more information here.