Category: javascript

Trying Qlik Sense Business API’s

Much of Qlik’s focus is today on their cloud products. Getting up and running with Qlik Sense Business (the name Qlik uses for their entry-level cloud product) is easy. And even if the hub is different and there is no QMC, it feels familiar. After all it’s the same engine behind the scenes. Many (probably most) extension will also work, since the extension API is the same, and the Capabilities API also the same – almost. But even though the methods are all there, some of them actually don’t give the same result.

The global object in the Capabilities API

The differences I have found are in the global object, as should be expected. (documentation is here). So let’s try it.

The first method is getAppList(). In Qlik Sense Desktop or Qlik Sense Enterprise (which is Qlik’s name on the product you install yourself on a windows server) this method will give you a list of all apps the currently authorized user has access to. In Qlik Sense Business it still works, but if you try it, it looks something like this:

In my Qlik Sense Business hub, I’ve got four apps, but getAppList only returns the one I am currently using. So if your extension uses getAppList to create a list of other apps, it won’t work in Qlik Sense Business.

Another frequently used method in the global object is getAuthenticatedUser(). You use it to find out the users id. In Qlik Sense Desktop you’ll get ‘Personal\\Me’, in Qlik Sense Enterprise you’ll get a string containing UserId and UserDirectory. In qlik Sense Business it looks like this:

So, while the format is the same as in Qlik Sense Enterprise, the content is different. The UserDirectory is empty, and the UserId is actually the IdP subject you can find in the cloud console (IdP stands for Identity Provider).

callRepository

The Capabilities APIs has one method designed to call the repository, originally built to call Qlik Sense Enterprise REST API called callRepository(). It is marked as Deprecated, but it is still there in Qlik Sense Enterprise, and looks like it’s there in Qlik Sense Business too. What happens if we try it?

Qlik Sense Cloud has the REST endpoint /api/v1/items to list apps the user has access to. If we try it, it looks like this:

We do get a warning in the console, but the call works, the data array actually contains the four apps I’ve got in my Qlik Sense Business hub. But since it is deprecated we probably shouldn’t use it but use some standard method of making a REST call instead.

Add Sense Chrome 1.3.0: corrections and changed icons

Version 1.3.0 of my Chrome Extension Add Sense is now available here. If you already have it installed, you should get the update automatically. These are the changes

Changed icon set

The extension now uses Qlik’s Leonardo UI instead of material design icons. The loading of Material Design icons does not work without configuration changes in Qlik’s cloud products, while Leonardo UI is already loaded.

Fixed bug when properties where treated like HTML elements

Sometimes expressions containing set analysis where destroyed in the tool. Mostly this happened when there was a letter immediately after the < character. That has now been fixed.

Other changes

If the visualization on screen is in the library, you previously only got the link (containing the id of the master visualization) in the property popup. Now you get the master objects properties.

The default filename when you save properties to file now contains the id of the visualization. You can still use another filename if you want to.

Pros and cons of Qlik Sense mashups

Recently I got the question on if I would recommend using Qlik Sense mashups. That got me thinking, of course the question is not so simple: it all depends. As with most things there are pros and cons…

Why would you use Qlik Sense mashups?

There are several reasons for using Qlik Sense mashups. A very common one is that you want to apply your companys profile and have a look-and-feel more like that you have for other systems. When I started building mashups for customers Qlik Sense hade no support for theming, so if you wanted to change things like fonts you needed to do that in a mashup. Now you can do this in a theme, but I honestly don’t know how much custom themes are used today.

Another common reason is that you want navigation to work another way than it does in Qlik Sense. Perhaps put sheets in a hierarchical structure or have views targeted to different user categories.

You might also mobile support to work differently than it does in standard Qlik Sense. In a mashup you can leverage open source CSS framework like Bootstrap or Bulma which gives you possibilities you do not have in the built-in Qlik Sense client.

Another reason is to simplify the user interface. In a mashup you can focus on the workflow that’s most important for your users (usually analyse data) and skip features like edit mode, storytelling, data load and modelling. This could potentially let your users get started faster and reduce the need for training, a big cost in a large dashboard projects.

You might also want to add features not in the standard Qlik Sense client. Examples could be commenting, writeback, sharing, exports not covered in the standard client, integration with other systems. Features like alternate state and default (startup) bookmark used to fall in this category, but are now included in the standard clent.

Problems with using Qlik Sense mashups

You should realize that mashups require HTML, CSS and javascript knowledge, a skillset most Qlik developers don’t have. You need to make sure you have both web developer skills and Qlik skills in your organization, whether you have that inhouse or work with a partner. And even though there are lots of web developers out there, very few of them know how to work with the Qlik APIs.

And the javascript environment is changing fast. While Qlik script has changed little in the 12 years I have worked with Qlik, the javascript environment has changed a lot. The language has a lot of new features, and frameworks, build tools and libraries has changed even more, so that the environment used in many mashups, with angularjs 1.x, today is pretty old and completely different frameworks (React etc) dominate, at least in new projects.

The tools included with Qlik Sense, like dev-hub, the Mashup Editor and the templates included makes it easy to create a mashup with only drag-and-drop and really no programming, but for production level projects it really is not enough. You would want stuff like multi-page support (including only loading Qlik Sense objects when they are needed), version handling and a dev environment where you can use npm packages, build tools etc.

Some advice on mashup development

So, you have decided that your use case is best solved with mashups. But how should you go about developing it? Here are some advice based on my experience with building mashups.

Recognize that you need web developer skills. The mashups itself is largely a web developer project. You need experience on web development, and on troubleshooting and debugging web apps. Someone with deep knowledge of HTML, CSS, Javascript and Qlik APIs is needed.

Web developer skills is not all. For users the most important part of your mashup is the data. For that you still need Qlik developer skills. In many cases you would have two kind of developers, those building charts and those working on the mashup itself.

Use version control. I used to say that version control is what differentiate professional development from private hacks. Today this is no longer true, even many hobby projects use version control. Set it up early in your project, commit changes often, make suire you know exactly what code runs in production.

Separate Qlik Object definitions from the javascript code. Try to avoid having Qlik Sense object definitions, field names etc in the actual code. Separate object id’s etc into separate configuration files or use tags in the Qlik Sense apps. This improves reusability of your mashup code and simplifies things a lot.

Use a CSS framework. A CSS framework like Bootstrap or Bulma helps you a lot with making your mashup responsive and good-looking. You might want to modify it by overriding the styles with your profile colors, fonts etc, but add those later.

Qlik Sense Global object

Sometimes when you are working with the Qlik Sense APIs you need access to information that is not connected to the Qlik Sense document you are working with, but rather to the QIX Engine running.

You find the apis for this in the global object in the Capabilities APIs, or in the Global class in the Engine API. While the Capabilities APIs only include very few methods, the full list of calls in the Engine API is much longer.

Finding the Global object

While there is a method in the Capabilities API to get the global object, in most cases you should not use it. In fact if you are building an extension you never should. Instead you should use the global property on the app.

That is because the getGlobal() method will open a new web socket to the engine, while app.global property will use the web socket used for the app. There is not reason to open another web socket to the server when you already have one, and in an extension there always is one.

The only situation getGlobal() should be used is when you do not have an open web socket, like in a mashup before the app is opened. And in that case you might be better off with a REST call.

Chrome developer tools in Qlik Sense extension development (part 1)

Perhaps the most important tool for a Qlik Sense extension developer is Chrome developer tools. You can use any text editor to write your code, or even Qlik Sense dev-hub for simple extension, but you really need to use Developer Tools. Debug your code, check loading of javascript, HTML and CSS, inspect HTML structure and CSS rules – there is a lot you need to use them for. So good knowledge of them is essential.

Always run Qlik Sense in a browser

Qlik Sense Desktop is a good tool for your extension development in most scenarios. But you should never use the browser bundled with it for development. Instead use Chrome, and use http://localhost:4848/hub to access the hub. This means you will have the full developer tools available, and also that you will have an up-to-date Chrome installation.

While the browser bundled with Qlik Sense Desktop is built on the same codebase, it is not updated, but will contain an old version. It looks like the Chrome version bundled with Qlik Sense Desktop September 2019 is Chrome/47.0.2526.16 while my browser Chrome version is Chrome/78.0.3904.70, so Qlik Sense Desktop is way behind. BTW Qlik Sense Desktop also reports my Windows version as ‘Windows NT 6.1’ while Chrome proper reports the real value ‘Windows NT 10.0’.

So now that you are happily in Chrome you need to open the Developer Tools. There are different ways to do that:

  • press Shift+Ctrl i (for inspect, this is the method I use)
  • from Chrome menu in the top right corner, under ‘Other tools’

When doing extension development, always keep the console open. We’ll come to why later.

Disable caching

When doing extension development, you want Qlik Sense to allways load the latest version of your code. The browser normally tries to cache loaded files, and not request the file from the server again, it it already has it in memory. The first thing you should do is to disable this caching.It’s easy to do:

  • open developer console
  • switch to network tab
  • click the ‘Disable cache’ checkbox

The disable cache checkbox only affects the browser when the developer console is open. When the developer console is not open, the browser will still cache javascript files. This is why you should always open the browser console when your are working with extension development.

Check installed extensions (and mashups)

There is a lot of information in the network tab. You can se all network requests made by the Qlik Sense client, and since the client is load in chunks there will be a lot of them. There are also requests for images, CSS stylesheets and other requests. To help you filter the requests there is a bar:

Click on XHR and you will get a list of other calls. One of them is of special interest when you are doing extension development, the one called schema (url is really /qrs/extension/schema). Click on that one, and then on ‘Preview’ and you will get a list of all extensions and mashups installed on the system, including all data in the qext file:

This is a good place to check that your extension is installed, and what version (just remember to keep the version number in the qext file up to date). It’s also a good place to start when you are troubleshooting an extension you did not write.

Debugging the extension code

The ‘Sources’ tab is what you use for debugging. If the code is minified (many extensions are) you can click the curly braches in the bottom left corner to get a somewhat more readable format.

The toolbar in the upper right corner can be used to step through your code. The stop button will make the browser pause when an error occurrs, which is very useful to find bugs. Unfortunately it does not really work in extensions. That’s because Qlik Sense catches all extension errors (and silently ignores them).

So to have the browser pause on your errors, you need to check the box ‘Pause on caught exceptions’. If you do that, the browser will pause when an exception is thrown from your code. Unfortunately it will also break on some exceptions (in libraries) that are perfectly OK, so you can’t really turn it on always. Just turn it on when you think there might be an exception thrown from your code and then trigger a repaint, for example by resizin the window.

Check files in extension

The navigator on the left side in the sources tab shows you extension files in a tree structure. But it will only show you javascript files, so CSS and HTML files are not included. If you want to see those, switch to the network tab, and filter on the extension name.

Note that the qext file is not included. That’s because the qext file is not loaded by the client, the information is instead included in the schema.

and there’s more

Of course there is more, but I couldn’t cover everything in one post. Look out for part 2, available when I find the time to write it..

Qlik nebula.js – use in Qlik Sense

Previously I have written about Qliks new nebula.js framework: how to build a visualization and how you can use it in a web page. But what if you want to use it in the Qlik Sense built-in client?

Nebula.js is a new framework, built on an architecture that differs a lot from the old Qlik Sense client and has a new API. This means nebula.js-based visualization won’t work in Qlik Sense.

Add a Qlik Sense wrapper

So to run your visualization in Qlik Sense you need to wrap it in something that supports the Qlik Sense extension API. You also need to add the qext file. Luckily nebula.js can do that for you. Just type:

nebula sense

and nebulajs will wrap your visualization so it can be used as a Qlik Sense extension. You find the result in the [visualization]-ext directory, so for nova -table it will be nova-table-ext. Just copy the entire directory to your Extension directory (assuming you’re using Qlik Sense Desktop) and you will find your extension in the assets panel in sense:

and you can add your nebulaj.s visualization to your Qlik Sense app. You will also get a basic property panel, which works pretty well for the very basic nova-table.

Improving meta data

But the extension name is ‘nova-table’ and the icon used is the default extension icon and there is no description. You might be tempted to fix this in the generated qext file, but don’t do that! There is a better way. Start by getting some help by typing:

nebula sense -help

And you will get the following info:

This means we can add your meta info to a separate file and supply the filename as an argument to the ‘nebula sense’ command. So let’s create a file called meta.json and enter the following:

{
	"name": "Nebula table",
	"icon": "table",
	"description": "Nebula test table wrapped in a Qlik Sense extension"
}

Run the command:

nebula sense --meta meta.json

And copy the update -ext directory to youe Extension directory, overwriting the old version. You will see that the meta data has changed:

Working on the property panel

The wrapper also contains a default property panel, something you might want toimprove, for some visualizations you actually have to improve it to make the extension work. To do that we start with the default propertypanel. It’s in nova-table-ext/extDefinition.js, but it’s minified. To get it in a format that’s easier to work with we use the minify option to turn minification off (default is on):

nebula sense --minify false

And then copy the extDefinition.js file to the src directory. If you look at the file you can see a few things we could improve:

  • the settings section is commented out
  • features as export, exportData and snapshot are not supported

To make this work you also need to remove the define call the build script has wrapped around the definition (it will be added again when you build). So modify the file to:

const extDefinition = {
  definition: {
    type: 'items',
    component: 'accordion',
    items: {
      data: {
        uses: 'data',
      },
      settings: {
        uses: 'settings',
      },
    },
  },
  support: {
    export: true,
    exportData: true,
    snapshot: true,
    viewData: false,
  },
};

export default extDefinition;

And then run the build:

nebula sense --meta meta.json --ext src/extDefinition

Copy over the extension files again and you’ll se that you now have the ‘Appearance’ section (called ‘settings’ in the code) and export etc now works.

Guide to Qlik Sense ApplyPatches

One of the more powerful calls in the QIX Api is the ApplyPatches call. It allows you to dynamically add, change or remove basically any property of a generic object. With it, you can allow users to dynamically change visualizations: reorder data, change dimensions and measures, add limitation etc etc. But using it in your implementation can be tricky.

Parameters

The ApplyPatches call only has two parameters (plus the handle, identifying which object we are patching:

NameDescriptionMandatoryType
qPatchesArray of patches.YesArray of NxPatch
qSoftPatchPatches are not savedNoBoolean

So it’s an arry and a boolean (true/false) flag. Let’s start with the flag. Setting the qSoftPatch flag to true does mean that the patched property is not saved, but it also has some other consequences:

  • you can change objects that you otherwise cannot, like published objects, or objects in apps you cannot change
  • the changes will only affect you, other users will not see them, they might very well apply a completely different patch to the same object
  • the changes will only affect the current session. If you disconnect and connect again, they will be lost and you will start over with the original version of the object

So if your goal is to allow users to dynamically change visualizations you would want qSoftPatch to be true. While the default value false works OK in Qlik Sense Desktop, it’s generally not something you would want to use in a server environment.

The array of patches

The array is a bit more difficult. It’s an array so you can change several properties with one call. For example if you change dimension or measure you should consider changing the title too, and probably do that in the same call.  Every entry(almost) in the array has three parameters:

NameDescriptionType
qOpOperation to perform. Add, remove or replace.String
qPathPath to the property.String
qValueThe value of the propertyString

The first parameter, qOp, is an easy one. In most cases you want to change a property, use ‘replace’. Occasionaly you want to add a property, use ‘add’. Rarely you want to remove a property, in that case use ‘remove’. 

The second parameter, qPath, is the path to the property, using / between the property names (not . as in javascript). Remember:

  • you’re patching properties, not the layout, so it’s /qHyperCubeDef/… not qHyperCube
  • for arrays you need to include the position in the array, and numbering starts with zero, so first dimension is /qHyperCubeDef/qDimensions/0/ etc
  • property structures can be complicated, speciallly the hypercube..
  • you can use my Chrome extension Add Sense to create the path argument. Show properties for the objects, switch to patch mode, copy the path

The second parameter, qPath, is the path to the property, using / between the property names (not . as in javascript). Remember:

  • you’re patching properties, not the layout, so it’s /qHyperCubeDef/… not qHyperCube
  • for arrays you need to include the position in the array, and numbering starts with zero, so first dimension is /qHyperCubeDef/qDimensions/0/ etc
  • property structures can be complicated, speciallly the hypercube..
  • you can use my Chrome extension Add Sense to create the path argument. Show properties for the objects, switch to patch mode

Some examples

A common case is when you want your users to be able to change the ordering of for example a table dynamically. Ordering of a hypercube is determined by the qInterColumnSortOrder property, an array with the column numbers in the order they should be used for sorting. If you want say the third column (which has number 2, since numbering starts with 0) you move the number 2 to the beginnging of the array and call applyPatches:

model.applyPatches( [{
  qPath: '/qHyperCubeDef/qInterColumnSortOrder',
  qOp: 'replace',
  qValue: JSON.stringify(sortorder)
}], true );

You can find a more complete example in the mashup I showed at Qonnections back in 2015.

Another example is when you want to reverse the ordering of a hypercube. You do this by toggling the boolean value of qReverseSort in the dimension or measure where you want to change the order. For the first dimension this will be:

model.applyPatches( [{
  qPath: '/qHyperCubeDef/qDimensions/0/qDef/qReverseSort',
  qOp: 'replace',
  qValue: JSON.stringify( !reversesort )
}], true );

Qlik nebula.js – first web app

A while ago I did my first test project with Qlik’s new open source library nebula.js (it’s available here). That time I tested the visualization part, corresponding to extensions in the current product (but hopefully used also for visualizations delivered by Qlik). Now it’s time to try it from the other side, building a web app with these new kind of visualizations.

Terminology first: the term mostly used in Qlik Sense is mashup. That’s because the current javascript API was originally designed for a mashup scenario, where you wanted to add Qlik Sense objects to an existing web page. But what it’s mostly used for today is building web apps, that is pages that include only Qlik Sense objects. And that is what I intend to do in this small test project.

Setting up

Nebula.js includes a small example app. You find it here. It’s just a few files, since it actually downloads the libraries (enigmajs, nebulajs) over the internet. So if we just copy those files to a local directory we have got a starting point.

The example uses a docker installation and a session app, but I want to use my Qlik Sense Desktop and the Consumer Sales app. To fix that I revised (and simplified) the connect.js file so it looks like this:

window.connect = function connect(host, id) {
	return fetch('https://unpkg.com/enigma.js/schemas/3.2.json').
	then(response => response.json()).
	then(schema => window.enigma.create({
		schema,
		url: `ws://${host}/app/${id}`,
	}).open().then(qix => qix.openDoc(id)));
};

And in the index.js file add host and app:

connect('localhost:4848','Consumer Sales.qvf').then((app) => {

and we have got something that’s working. Start Qlik Sense desktop, make sure your test project is served (I use serve), and open the file in the browser. You’ll se a selection toolbar, and an area that displays the simple supernova that’s included:

If you wonder how come my selection toolbar shows selections, that’s because I opened the same app with the same url(use http://localhost:4848/sense/app/Consumer%20Sales.qvf ) and made a selection. Session sharing does the rest.

Load nova-table

OK, so now we have got a page using nebulajs and connecting to an app, and the selction toolbar works. But how do we use supernovas like the one I made in my first test drive? Well first we need to make it available in our project. Just copy the files from the dist directory in nova-table and put it in your project. I have used a sub-directory called novas, since there is actually nothing super about nova-table. You can then load it in index.html:

  <script src="./novas/nova-table.js"></script>
  <script src="connect.js"></script>
  <script src="index.js"></script>

It needs to be before index.js for this to work.

Unlike the current Qlik Sense javascript libraries, nebulajs is not bundled with a module management library like requirejs. Instead it will see if there is a module manager available and use it if there is. If not it will simply add the module as a global function. So we can access it simply as window[‘nova-table’]. In a real project we would not really want everything defined globally, but this is just a quick test.

To make a solution that can be extended with more visualization, we create a novas object, where we can put all our visualizations, and then grab the right one using the type, with the sn from the initial example as the default:

const novas = {
 'nova-table': window['nova-table']()
};
const sn = {
 component: {
  mounted(element) {
   element.textContent = 'Hello';
  },
 },
};

const nebbie = window.nucleus(app, {
 load: (type, config) => config.Promise.resolve(novas[type.name] || sn),
});

Add a table

So now we have got the nova-table module loaded, let’s create one and add to the page. We’ll make a session object, with the definition in the javascript file, so first we change the HTML file to:

  <div class="content">
    <div class="toolbar"></div>
    <div class="object" id="nova01"></div>
  </div>

And in the javascript file we use the create method to create a nova-table object, with properties that contain a hypercube and inject it into the HTML element ‘nova01’:

nebbie.create({
 type: 'nova-table',
}, {
 element: document.getElementById('nova01'),
 properties: {
  qHyperCubeDef: {
   qDimensions: [{
    qDef: {
     qFieldDefs: ['Customer']
    }
   }],
   qMeasures: [{
    qDef: {
     qDef: 'Sum([Sales Amount])'
    }
   }],
   qInitialDataFetch: [{
    qWidth: 5,
    qHeight: 1000
   }]
  }
 }
});

And we get something like this:

A very basic page with a working app connection and displaying some data from our app using the visualization we developed earlier. The project is available here.

Building on Qlik Sense bundled extensions

From the very start when Qlik Sense was launched a recurring question has been ‘how do I modify the xxxx chart and add feature yyyy’. That has never been supported and still isn’t. But with the visualizations provided by Qlik as in the extension bundles it’s another thing. These visualizations are bundled with the installation, but Qlik has also published the source at github.com/qlik-oss, and adding features to those are possible, even encouraged.

The multi-kpi extension

One of the extensions in the visualization bundle is the multi-kpi extension, originally written by Alexander Nerush and published under the very modest name SimpleKPI. Qlik has forked the repository, made some changes, written a few pages of help info and bundled it with Qlik Sense.

The original name SimpleKPI is pretty misleading: this is a powerful KPI object with lots of options and functionality many customers want. I encourage you to take a look at it if you haven’t already.

The requirement

One of these options is the possiblity to connect a sheet to the KPI, so that when the user clicks on the KPI, a sheet showing details behind that number opens. But sometimes you do not only want a sheet to open, but also some selections to be applied. So we want the possibility to connect not only a sheet, but also a bookmark to the KPI.

Adding this to the built-in KPI object is not possible, since you cannot modify it. Writing your own KPI object from scratch is of course possible, but would take some time (certainly if you want the features of the multi-kpi object), but adding it to the multi-kpi object is much easier.

Making the addition

To make the addition you do the following:

  • make a fork of the multi-kpi repository
  • run npm install to get the dependencies needed
  • make your changes to the source code
  • build and test like with any extension

My version of multi-kpi, with the link to bookmark feature is available here. If you feel your feature is of general interest, you might also create a pull request back to Qlik’s version, and let them decide if it’s something worth adding.