A new version of my Chrome extension Add Sense is now on it’s way out. There is only one new feature in it: measures and dimensions are now validated, and if there is an error, you’ll see it in the list of measures / dimensions.
If you, like me, often start a new app by copying an old one, you might by accident have published an app with invalid measures and/or dimensions. The new feature helps you find thos, so you can fix or remove them. Also pretty useful if you’ve made changes to the data model in the app.
You don’t need to do anything to get the new version, Chrome will update automatically.
Add Sense Chrome version 2.0.0 is now available in the Chrome web store. If you already installed the previous version it should be upgraded automatically.
The main change is that the extension now uses Chrome manifest version 3, a change that was needed since version 2 will soon no longer be supported. As a user that change will not mean much. There are however some other changes too:
if you use many Qlik Sense objects in your page numbers with more than 2 digits will work better
the Active objects list has been updated, number of currently active objects are now displayed in the header, if the are not valid you will see that
the small boxes in the page can now be closed to avoid clutter
ODAG objects work better
the properties button in Active Objects has been corrected
Version 1.3.0 of my Chrome Extension Add Sense is now available here. If you already have it installed, you should get the update automatically. These are the changes
Changed icon set
The extension now uses Qlik’s Leonardo UI instead of material design icons. The loading of Material Design icons does not work without configuration changes in Qlik’s cloud products, while Leonardo UI is already loaded.
Fixed bug when properties where treated like HTML elements
Sometimes expressions containing set analysis where destroyed in the tool. Mostly this happened when there was a letter immediately after the < character. That has now been fixed.
If the visualization on screen is in the library, you previously only got the link (containing the id of the master visualization) in the property popup. Now you get the master objects properties.
The default filename when you save properties to file now contains the id of the visualization. You can still use another filename if you want to.
A new version of my Chrome extension for Qlik Sense, Add Sense, is now available in Chrome Web Store. This is a pretty big version, actually that’s why it has number 1.2.1, because a bug sneaked into 1.2.0 which I had to fix.
New app level operations
You reach the new functions from the app popup box:
In fact, most of the buttons are new. From the left to the right they are used for:
showing the script
listing the tables
exploring the data model (in Catwalk)
listing extensions used
listing active objects
A number of the new functions helps you with the data model. You can see what tables are defined in the app, and list dimensions and measures (and copy their definition into a program of your choice, like Excel or something). But perhaps the best new function is the link to Catwalk. Click on the compass icon, and it will open Catwalk with your current app:
If you don’t know what Catwalk is it’s time to get to know it knopw. It’s an open source tool from Qlik that helps you explore the data model of a Qlik Sense app. If it doesn’t work on your server, you might need to whitelist it. Check the documentation! Originally I planned to do some visualization of the data model, but I realized that I cannot make anything better than Catwalk, so why not reuse it.
List all active objects
The Add Sense extension uses Angular and some CSS selectors to find Qlik objects in your page, whether it’s the standard Qlik Sense client or a mashup. This only works for visualizations that are either Qlik Sense standard visualizations or extensions built with the traditional Angular-based method. But in a custom built web app using Qlik you might choose to visualize the data in some other way. Qlik Sense objects used this way have previously not been available in Add Sense, you could not check their properties, monitor recalculation times etc.
With the new function to list all active objects that has changed. While the little boxes showed on the page start from the HTML DOM and Angular scopes, this function uses enigmajs and lists all objects active in the enigmajs session. It might look something like this:
It still needs to find the enigmajs session, which might not work in all cases. If you want to use the extension for a case where it does not work, please let me know, you might be able to fix that with just a very small change.
Dimensions and measures
The extension can now also list dimensions and measures.
You can use the copy function in the dialog to copy the contants to the clipboard and then into application of your choice, like Excel etc.
And that’s it for the new functions. Please let me know if there is anything you are missing.
As I wrote in Part 1 Chrome Developer tools is perhaps the most important tool for a Qlik Sense extension developer. Some important points in part 1:
you should turn caching off in the network tab, and keep developer tools open to make sure files are not cached
Qlik Sense client catches program errors in your extension, so if you need to see them temporarily check the box ‘Pause on caught exceptions’. You can’t keep this box checked all the time though.
In this post we will continue with some more useful stuff.
Checking web socket traffic
As you probably know communication with the Qlik Engine is over web sockets. You find the web socket in the Network tab, just click on WS and you will find it. Click on the websocket and then on messages, and you will see the communcation, something like this:
Lots of information, lots of transactions. Some things you should know:
messages from client to server has a dark background, messages from server to client a white background
all messages are in JSON format
if you click on a message it will be displayed in a format that’s easier to read in the area below
you can filter messages by typing in text in the field where it says ‘Enter regex..’
Connect request and reply
What you see in the web socket log is simply a list of all messages sent and received, in chronological order. Since commands are processed in paralell, the reply for a specific request might not at all be anywhere near the request. Request and reply are connected with the ‘id’ attribute (that’s a feature in the json rpc protocol, which Qlik Sense is built on). This means we can filter on the id attribute to find the reply for a specific request. If you try typing in “id”:1, in the filter box you will get request #1 and it’s reply:
So now we know how to find a request and it’s reply, but to really understand what’s happening we also need to understand handles.
Handles is a Qlik Sense specific concept, not part of Json RPC but an addition to it, made by Qlik. The QIX protocol is object-oriented, meaning that all commands are sent to an object. When you connect to Qlik Sense engine, the only handle available is the global handle, -1.
The first thing you do in most cases is call OpenDoc to open your Qlik Sense app (or Doc, as its called in the QIX documentation). If that succeeds you will get a handle (called qHandle) back in the reply. You can them send Doc commands to this handle (almost always handle 1). If you type in the filter “handle”:1, you will see all app commands used:
As you can see there is a lot of them. Most of them fetches or creates objects in the app, with GetObject or CreateSessionObject. You do not see the replies, only the requests, since the replies do not contain the handle. But if we instead filter by the id of one of the GetObject calls, (in my case id 13) we can see what happens:
As you can see in the reply there is a new qHandle in the reply, #5. So now we can filter on “handle”:5, and see what commands were sent to this object. If you want to know what handle is used for your extension, there is as far as I know no way to get that information from standard Qlik, but you can use my Chrome Extension Add Sense.
Find commands that invalidates object
Handles are used not only to refer to objects when you send commands, but also for invalidation. When you change selections (make a new selection, clear selection, apply bookmark etc) QIX engine tells you what objects are no longer invalid. It does this with the ‘change’ array (another addition to JSON rpc) which contains handles for the invalidated objects. So if I for example want to find commands that invalidated opbject 13, I can filter on “change”:\[.*13.*\] (note: a regex, since the handle can be at any position in the array):
We only get the replies for commands that invalidated the object, to get the actual command you need to use the id field to find the actual command and handle.
Version 1.1.0 of my Add Sense Chrome Extension is now available in Chrome web store. This releas has two new features, one of which is pretty experimental.
Properties in patch mode
From the beginning the extension has allowed you to see properties as an object, that includes all properties. But sometimes you need properties in patch format, with the path to each property and the value. This is the format used in one of the most powerful API calls of the QIX Engine, the applyPatches call.
The list of patches you will see:
includes all properties
and can be filtered
Creating the paths for applyPatches calls is one use for it. Another is when you want to for example compare propertries of two objects, like the expressions used (are they the same??) or dimensions. In that case simply open properties for both objects, switch to patch mode and filter on what you are interested in.
Quite frequently you want to copy something from Qlik Sene and paste into a document or another program. This is hard to do. The Enable copy alternative helps you with that by simply turning on standard HTML text select. You can then select and copy text much like in a standard HTML page.
This works well in some cases, but in other it conflicts with the Qlik Sense UI and does not work at all, might even make some stuff unusable. So use it with care and do not have too high expectations.
One of the uses for the Add Sense Chrome extension is to check performance in your Qlik Sense app or mashup. To help you with this the info boxes show you a few numbers, collected as you use the extension. When you select to show the boxes, it will start collecting statistics, and you will se something like this:
In this post I am using one of Qliks demo apps (you find it here, try it yourself!), so I don’t really expect to find any performance problems.
The Chrome extension will find all both objects defined in the app and session objects, but only ones that are shown on the page. For all such objects it will start collecting statistics:
number of recalculations made
latest calculation time in milliseconds
maximum calculation time
In the top left corner you will see the handle, that’s a number that the QIX Engine assigns to objects as they are opened. If you want to see the actual messages sent for a specific object in the web socket log you need this number. Since the handles are assigned in sequence, you will also get a grip of how many objects you are using in your solution.
Now to try this lets make some selection. Open the selection tool and click on show in the extension menu (yes the extension works in the selection tool too).Make some selections and you will see something like this (excuse me, it’s in swedish):
Confirm your selections and close the selection tool. When you come back to the sheet probably only the app infobox will be visible. When Qlik Sense removes the objects from the page, the infoboxes disappear too. But the extension will continue to collect statistics, so click on show again in the extension menu, and you will se something like this:
Note that objects have only been recalculated once. That’s because the client only recalculate objects that are visible on the screen. The text object on the left should not be recalculated at all, since it was not affected by our selections.
So that’s what you need to do in a real situation:
enable timing for objects on the screen with the show command
make your selections, the extension wil register recalculations
if objects have been off screen, reenable the infoboxes with the show command
analyze the figures. Look for long calculation times or too many recalculations
Once you have started the timing, the only way to stop it is by refreshing the page, using the show command will not stop the monitoring.