Callejero español 2020 con códigos postales (gratis)

En determinadas ocasiones necesitamos utilizar información geográfica en nuestras aplicaciones. No me refiero tanto a coordenadas geográficas sino simplemente a disponer de una relación de provincias, municipios, distritos, calles y códigos postales.

Desafortunadamente en España parece que los datos de dominio público se encuentran en un estado de disponibilidad muy precario y que cualquier oportunidad es buena para arañar unos cuantos euros a quien esté dispuesto a pagar por ellos.

Ayer tuve la necesidad de incluir información de este tipo en una de mis aplicaciones y cuál fue mi sorpresa cuando al ir a buscarla a internet para adaptarla me encontré con que aparentemente era información ‘de pago’.

En una búsqueda de Google como esta ‘base de datos codigos postales españa‘ aparece como primer resultado un enlace a la página de la empresa Correos en la que nos da instrucciones de cómo acceder a la base de datos de códigos postales previo pago de 687€ (en su versión más básica).

Indagando algo más encontré alguna referencia a reclamaciones realizadas en la página datos.gob.es cuya resolución dice textualmente:

La base de delimitaciones de códigos postales actualmente está sujeta a comercialización. Para adquirir el listado hay que seguir las instrucciones indicadas en la página de CORREOS

Personalmente me parece vergonzoso que este tipo de información esté comercializada por una empresa privada que nació con presupuesto público, pero España es así.

Dejando polémicas a un lado, seguí buscando y encontré una alternativa muy válida en la página del INE. El Callejero del Censo Electoral sirve perfectamente para este propósito ya que contiene información de todos los municipios de España con sus tramos de vías y pseudovías identificados con su correspondiente código postal. En la página de referencia existe un documento con la especificación de formato de cada uno de los ficheros que contienen la información:

  • Tramos de vía
  • Unidades poblacionales
  • Vías
  • Pseudovías

He realizado una importación y tratamiento de los ficheros para MySQL dejando únicamente la información esencial para poder montar un callejero con información de códigos postales.

La base de datos está disponible aquí. Una vez descomprimida ocupa 164MB.

Tras ejecutar el script SQL, encontraremos una base de datos llamada codpos que contendrá las siguientes tablas:

  • provincia
  • unidad_poblacional
  • via
  • pseudovia
  • tramo

Tramo es la tabla que relaciona el resto y corresponde a los tramos de vía correspondientes a cada una de las secciones y subsecciones censales (información que he omitido de la base de datos).

Un ejemplo de utilización sería la siguiente SQL:

select t.cpro as `codigo_provincia`,
       pr.dpro as `provincia`,
       up.dmun as `municipio`,
       up.nentsic,
       up.nnucle,
       ifnull(pv.dpsvia, concat(v.tvia, ' ', v.nvia)) as `via`,
		 t.cpos as `codigo_postal`
from tramo t left outer join via v 
on(t.cpro = v.cpro and t.cmum = v.cmum and t.cvia = v.cvia)
left outer join pseudovia pv 
on(t.cpro = pv.cpro and t.cmum = pv.cmum and t.cpsvia = pv.cpsvia)
left outer join unidad_poblacional up 
on(t.cpro = up.cpro and t.cmum = up.cmum and t.cun = up.cun)
left outer join provincia pr
on(t.cpro = pr.cpro)
where v.nvia like 'velazquez%'
  and 58 between t.ein and t.esn
  and pr.cpro = 28
  and t.tinum = 2

que nos devolverá todos aquellos tramos de vía con denominación que empiece por ‘velazquez’, que contengan un número 58 entre sus pares (tinum = 2) y que estén en la provincia de Madrid (28).

Read a Xiaomi Mi Smart scale using an ESP32

Xiaomi Mi Smart is a digital scale with BLE interface. This allows it to broadcast your weight to any device like your smart phone and incorporate the measurements to an application like Samsung Health.

Xiaomi Mi Smart Scale

 Two years ago I was trying to read BLE messages from an ESP32, but Arduino BLE libraries didn’t work very well at that moment and all I got were some headaches. I wasn’t even able to match the Weight Measurement remote characteristic, so once the ESP32 connected to the scale it wasn’t able to implement the callback functions.

Today I’ve tried again, and I made it work!!!

Calculate the UUIDs

I have used the BLE_Client example (under ESP32 dev board samples).  In order to connect and read values from any device you need two unique identifiers (16 bit UUID alias):

Now, with this two short uuids, we will calculate the long (128 bits) uuid by using the next formula:

128bit_base_uuid = 16bit_uuid_alias * 2^96 + Bluetooth_Base_UUID

where the Bluetooth_Base_UUID is 1000800000805F9B34FB (00000000-0000-1000-8000-00805F9B34FB)

In our case we will be using the string version of the 128bit uuid and that means all we have to do to calculate the long uuid for the service and the characteristic is concatenating the uuids (aliases and Bluetooth Base one) in the following way:

  • Service UUID: 0000181D-0000-1000-8000-00805F9B34FB
  • Characteristic UUID: 00002A9D-0000-1000-8000-00805F9B34FB
Finding our weight scale

Bring your Xiaomi scale next to your ESP32 MCU and load the next sketch using Arduino IDE:

/**
 * An ESP32 BLE client to find Xiaomi Mi Smart weight scale
 * Author Pangodream
 * Date 2020.05.31
 */

#include "BLEDevice.h"
//Base UUIDs
//Weight Scale service
static BLEUUID srvUUID("0000181d-0000-1000-8000-00805f9b34fb");

/**
 * Callback class for each advertised device during scan
 */
class deviceCB: public BLEAdvertisedDeviceCallbacks {
 //Called on each advertised device
  void onResult(BLEAdvertisedDevice advertisedDevice) {
    // We have found a device, let us now see if it contains the service we are looking for.
    if (advertisedDevice.haveServiceUUID() && advertisedDevice.isAdvertisingService(srvUUID)) {
      if(advertisedDevice.getName() != "MI_SCALE"){
        Serial.print("Weight scale (no MI_SCALE) device found with address ");
        Serial.print(advertisedDevice.getAddress().toString().c_str());
        Serial.print(" and name ");
        Serial.println(advertisedDevice.getName().c_str());
      } else {
        Serial.print("Xiaomi Mi Smart weight scale found with address ");
        Serial.println(advertisedDevice.getAddress().toString().c_str());
        BLEDevice::getScan()->stop();
        Serial.println("End of scan");
      }
    } else {
      Serial.print("Non weight scale device found with address ");
      Serial.println(advertisedDevice.getAddress().toString().c_str());
    }
  } 
};

void setup() {
  Serial.begin(115200);
  Serial.println("Devices scan");
  BLEDevice::init("");
  BLEScan* pBLEScan = BLEDevice::getScan();
  pBLEScan->setAdvertisedDeviceCallbacks(new deviceCB());
  pBLEScan->setInterval(1349);
  pBLEScan->setWindow(449);
  //Set active scan
  pBLEScan->setActiveScan(true);
  //Scan during 5 seconds
  pBLEScan->start(5, false);
}

void loop() {

  delay(1000);
}

The sketch starts an active scan to find all the BLE servers (devices) that are advertising themselves at that moment.

The scan results are shown at the Serial Monitor (115200 bauds) and if a Xiaomi Mi Smart scale is found (and its name is MI_SCALE) something like the next should appear in your monitor:

If your weight scale was present in the scan and you can read the message “Xiaomi Mi Smart weight scale found with address xx:xx:xx:xx:xx:xx” then you can continue with our next goal, which is to retrieve data from weight measurement.

Retrieving data

When the ESP32 is scanning for devices, it only reads the service beacons sent by each BLE server to advertise itself, but when we use the scale and it broadcasts the calculated weight, it doesn’t send the same type of message (it is not a service message, but a characteristic one).

Note that we haven’t used yet the characteristic UUID we calculated before and we’ve only used the service one.

Let’s load a new sketch into the ESP32 MCU to retrieve some data from the characteristic:

/**
 * An ESP32 BLE client to retrieve data from the Weight Measurement characteristic  
 * of a Xiaomi Mi Smart weight scale
 * Author Pangodream
 * Date 2020.05.31
 */

#include "BLEDevice.h"
//Base UUIDs
//Weight Scale service
static BLEUUID srvUUID("0000181d-0000-1000-8000-00805f9b34fb");
//Weight Measurement characteristic
static BLEUUID chrUUID("00002a9d-0000-1000-8000-00805f9b34fb");

static BLEAdvertisedDevice* scale;
static BLERemoteCharacteristic* remoteChr;
static boolean doConnect = false;
static boolean connected = false;
static int year = 0;

/**
 * Callback function for characteristic notify / indication
 */
static void chrCB(BLERemoteCharacteristic* remoteChr, uint8_t* pData, size_t length, bool isNotify) {
    //Console debugging
    Serial.print("Received data. Length = ");
    Serial.print(length);
    Serial.print(". - Data bytes: ");
    for(int i =0; i< length; i++){
      Serial.print(pData[i]);
      Serial.print("  ");
    }
    Serial.println(" ");
    //End of console debugging
    
    //Parsing the received data and calculate weight
    boolean temporary = true;
    int rcvdYear = pData[3];
    //If we received a year for the first time, store it in the year variable
    //The first year we receive indicates a temporary measurement
    if(year == 0){
      year = rcvdYear;
    }else{
      //If year has been previously defined and the year we have received is
      //greater than it, then the measurement is not temporary, is the final one
      if(rcvdYear > year){
        temporary = false;
      }
    }
    double weight = 0;
    weight = (pData[1] + pData[2] * 256) * 0.005;
    
    Serial.print("Weight: ");
    Serial.print(weight);
    Serial.print(" Kg - ");
    if(temporary){
      Serial.println(" (Provisional)");
    }else{
      Serial.println(" (Definitive)");
    }
}

/**
 * Callback class for each advertised device during scan
 */
class deviceCB: public BLEAdvertisedDeviceCallbacks {
 //Called on each advertised device
  void onResult(BLEAdvertisedDevice advertisedDevice) {
    // We have found a device, let us now see if it contains the service we are looking for.
    if (advertisedDevice.haveServiceUUID() && advertisedDevice.isAdvertisingService(srvUUID)) {
      if(advertisedDevice.getName() != "MI_SCALE"){
        Serial.print(".");
      } else {
        Serial.println("  Found!");
        BLEDevice::getScan()->stop();
        Serial.println("Stopping scan and connecting to scale");
        scale = new BLEAdvertisedDevice(advertisedDevice);
        doConnect = true;
      }
    } else {
      Serial.print(".");
    }
  } 
};
/**
 * Callback class for device events
 */
class ClientCB : public BLEClientCallbacks {
  void onConnect(BLEClient* pclient) {

  }

  void onDisconnect(BLEClient* pclient) {
    Serial.println("Disconnected. Reconnecting...");
    connected = false;
  }
};

bool connectToScale() {
    Serial.println("Stablishing communications with scale:");
    BLEClient*  pClient  = BLEDevice::createClient();
    Serial.println("    BLE client created");

    pClient->setClientCallbacks(new ClientCB());

    // Connect to the remove BLE Server.
    pClient->connect(scale);
    Serial.println("    Connected to scale");

    // Obtain a reference to the service we are after in the remote BLE server.
    BLERemoteService* pRemoteService = pClient->getService(srvUUID);
    if (pRemoteService == nullptr) {
      Serial.println("    Error: Failed to find service");
      pClient->disconnect();
      return false;
    }
    Serial.println("    Service found");

    remoteChr = pRemoteService->getCharacteristic(chrUUID);
    if (remoteChr == nullptr) {
      Serial.print("    Failed to find characteristic");
      pClient->disconnect();
      return false;
    }
    Serial.println("    Characteristic found");
    Serial.println("    Setting callback for notify / indicate");
    remoteChr->registerForNotify(chrCB);
    return true;
}

void setup() {
  Serial.begin(115200);
  Serial.println("Searching for MI_SCALE device");
  BLEDevice::init("");
  BLEScan* pBLEScan = BLEDevice::getScan();
  pBLEScan->setAdvertisedDeviceCallbacks(new deviceCB());
  pBLEScan->setInterval(1349);
  pBLEScan->setWindow(449);
  //Set active scan
  pBLEScan->setActiveScan(true);
  //Scan during 5 seconds
  pBLEScan->start(5, false);
}

void loop() {
  if(doConnect && !connected){
    connected = connectToScale();
  }
  delay(1000);
}

Reboot the MCU and take a look to the serial monitor. If everything goes right, something similar to this should be shown:

The sketch performs these tasks:

  1. Scan devices to find the Xiomi scale
  2. Connects to the scale
  3. Search for the Weight Scale service
  4. Instantiates the service
  5. Search for the remote characteristic
  6. Instantiates the remote characteristic
  7. Sets the callBack function for the characteristic

In this way, when the scale begins to measure weight, our callback function will be invoked receiving the data.

At this point, the ESP32 is waiting to receive something in the callBack function. Put some weight over the scale and you should see the provisional and definitive measurements at the serial monitor.

The callback function always receive 10 bytes of information. These bytes can be parsed to get the weight and timestamp values in the following way:

Parsing the XIOMI Scale BLE data

The way we can differentiate between the provisional and the definitive measurements is looking at the year in the timestamp. While measurements are provisional, the scale sends a fake year and not the current one. So, all we hve to do is taking note of the value for the year in the first data packet we receive and then compare to the rest of the years we receive.

When the received year is greater than the first year we received that is the definitive measurement and we can get the calculated weight.

Why BLE_client sketch doesn’t work

If yoy try the sketch located at File / Examples / ESP32 BLE Arduino / BLE Client it won’t work and will not receive any chararteristic message even when you change the UUIDs.

The reason is this piece of code

if(pRemoteCharacteristic->canNotify())
      pRemoteCharacteristic->registerForNotify(notifyCallback);

To set the callback function for the remote characteristic, first it checks the characteristic can be notifiable (method canNotify()) and our characteristic is not.

If you study the BLE connection using any app for that purpose (I use nRF Connect from Nordic Semiconductor) you will notice that the characteristic we are using is not notifiable but ‘indicate’ type:

So, if you comment out the condition to avoid checking if the characteristic is notifiable or not, and set the callback function anyway, you will received calls in the callback function with the parameter isNotify set to false.

ESP32 MCU + MAX30100 not working

First of all, please read the post title. This doesn’t work on an ESP32 MCU.

I only made it work on an ESP8266 MCU.

This afternoon I received a couple of MAX30100 sensors that I ordered some days before (44 days, China, COVID19, …)

MAX30100 Sensor Board
MAX30100 Sensor Board

The board is quite easy to connect: it has an I2C interface and you even don’t need the INT pin, so all you have to do is wiring the SDA and SCL pins.

I used D22 and D21 pins in my ESP32 MCU, as they are also used for I2C_SDA and I2C_SCL interfacing (checkout pinout diagram here)

Summarizing, the pins you need to connect are:

ESP32 GND to SENSOR GND

ESP32 3.3 to SENSOR VIN

ESP32 D22 to SENSOR SDA

ESP32 D21 to SENSOR SCL

Once you have all the pins connected it’s software time. 

In Arduino IDE, select Manage Libraries and install the Sparkfun MAX3010X Library

Sparkfun MAX3010X Library

After that, you are ready to open any of the examples provided with the library and test the module.

Probably you will notice that doesn’t work correctly. It seems to initialize the module correctly and even it starts to make some measurements. You even  would think that is working for a few seconds, but it isn’t.

The behaviour, at least in my case, is the next: 

  • The module initialize with a SUCESS status
  • It lights on the red led
  • You put your finger over the sensor
  • It starts measuring, but after a second the red led goes off
  • It seems to measure intermittently

If this is your case, don’t waste your time. I was searching for troubleshootings and all I found was replacing the inbuilt 4K7 resistors or bypassing the voltage limiter in the sensor board. Don’t do it, it is not a problem of pull-up resistors.

I decided then to try with an ESP8266 MCU and this is the result:

The sensor board works fine, so probably it is a problem of compatibility between the library and ESP32.

The sensor measures heart rate and SpO2. I only used the heartBeat event to light the led to show that it works.

Google Earth Studio: a filming drone in your hands

I can’t stop amazing about what Google can do with data.  If you like switching to terrain view in Google Maps to look to the actual appearance of places, then you have to visit Google Earth Studio

A short promotional video is shown at the home page and the first time I saw it I thought: This can’t be real. I immediately pressed the “Try Earth Studio” button and… and nothing: you need to register and explain your reasons to try the product. I did it, but honestly I thought I would never receive their approval.

I was entirely wrong. Today I received a link to try the product, and I have to say it is amazing. 

It was a matter of minutes to make a video showing “La Puerta de Alcalá” in Madrid (where I live) from the air like being filmed by a camera drone.

The video is very simple, but what I needed to produce it was even simpler.

The only thing I didn’t like was that the platform doesn’t produce a video ready to be downloaded, but all the video frames (jpeg format) compressed in a zip file and you have to compose the video with them.

Well, if you use default options (30 fps) and you want to produce an H264 video all you have to do is:

  • Unzip the file containing the frames
  • Access the footage folder
  • Execute ffmpeg specifying these options
ffmpeg -i "videoname_%03d.jpeg" -c:v libx264 -vf fps=30 -pix_fmt yuv420p out.mp4

Of course, you’ll have to replace videoname with the prefix name of your jpeg files.

Make your WOL configuration TRULY sticky

Introduction

The typical steps to make your Ubuntu server wake on LAN are:

  • Find your network card interface name
  • Check your network card capabilities
  • Use ethtool to set “Wake-on” option to “g” value

And that’s all, then you put your server in suspend or hibernate mode and wake it up remotely. It works like a charm, but then you try a second time, you hibernate the server again and… it doesn’t wake remotely.

What happened, is that you didn’t repeat the third step to set again the “Wake-on” option to “g” value. The value you set for the network interface is volatile and you have to repeat the third step on each boot… unless you make it sticky.

Setup the network interface to work just once

1.- Find your network card interface name

sudo ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: enp3s0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc fq_codel state DOWN group default qlen 1000
    link/ether e8:94:f6:08:5a:60 brd ff:ff:ff:ff:ff:ff
3: eno1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether c8:9c:dc:2b:aa:48 brd ff:ff:ff:ff:ff:ff
    inet 192.168.16.126/24 brd 192.168.16.255 scope global noprefixroute eno1
       valid_lft forever preferred_lft forever
    inet6 fe80::ca9c:dcff:fe2b:aa48/64 scope link
       valid_lft forever preferred_lft forever

In my case, the server has three interfaces:

1: lo (the local loopback)

2: enp3s0: one 100Mbps ethernet card (not being used)

3: eno1: one 1Gbs ethernet card (this is the one I want to use to wake the system remotely, as it is the one configured to connect to my LAN). I will copy two values:

Interface name: eno1 (be aware of one (1) and lowercase L (l)). Usually interface name ends with a number, not a letter.

MAC address: e8:94:f6:08:5a:60

Now we know the interface name, we will check the Wake-on capabilities:

sudo ethtool eno1
Settings for eno1:
        Supported ports: [ TP ]
        Supported link modes:   10baseT/Half 10baseT/Full
                                100baseT/Half 100baseT/Full
                                1000baseT/Full
        Supported pause frame use: No
        Supports auto-negotiation: Yes
        Supported FEC modes: Not reported
        Advertised link modes:  10baseT/Half 10baseT/Full
                                100baseT/Half 100baseT/Full
                                1000baseT/Full
        Advertised pause frame use: No
        Advertised auto-negotiation: Yes
        Advertised FEC modes: Not reported
        Speed: 1000Mb/s
        Duplex: Full
        Port: Twisted Pair
        PHYAD: 1
        Transceiver: internal
        Auto-negotiation: on
        MDI-X: off (auto)
        Supports Wake-on: pumbg
        Wake-on: d
        Current message level: 0x00000007 (7)
                               drv probe link
        Link detected: yes

Take a look at the last lines. We are looking for two different lines:

Supports Wake-on: pumbg

and

Wake-on: d

The “Wake-on” mode configured by default is “d”, which means that the network card will not switch on the server when it receives a magic packet but, as the network interface supports “g” mode (it is one the letters in pumbg) we can set the value of “Wake-on” to “g”.

We will use ethtool for this. If it is not already installed on your system, do it:

sudo ethtool -s eno1 wol g

Now, if you repeat the step to check your network card capabilities (ethtool eno1) you shoud see the “Wake-on” option set to “g” value.

That means your server is ready to sleep and wake remotely.

Put the server into hibernation mode:

sudo systemctl hibernate

And now wake it remotely using one of the many available tools. Depending on the platform you will use an Android, Windows, Linux, … tool for this purpose and the only thing you will need is the MAC address you copied some steps above.

If everything went right, your server has woken, but what if you repeat the previous steps? (hibernate – remotely wake) It doesn’t work.

As I mentioned in the introduction, the value you configure in the “Wake-on” option of your network card is volatile. Each time you reboot your server it resets its value (usually to “d”).

Make your configuration sticky

We will create a system service to set the “Wake-on” value to “g” each time the server boots or restart.

There are a lot of recipes for these, but most of them didn’t work in my case. I’ll tell you one configuration line that did the trick for me.

1.- Create the .service file using your favourite editor

sudo nano /etc/systemd/system/wol.service

Now, copy the next content inside the file (change the name of the interface card and set the description you prefer):

[Unit]
Description=Activate WOL on eno1 network card
After=network-online.target

[Service]
Type=oneshot
ExecStart=/sbin/ethtool -s eno1 wol g

[Install]
WantedBy=basic.target

Save the file (^O + ENTER + ^X)

Now we will start the service for the first time

sudo service wol start

And check its status

sudo service wol status

● wol.service - Activate Wake On LAN
   Loaded: loaded (/etc/systemd/system/wol.service; enabled; vendor preset: enabled)
   Active: inactive (dead) since Sat 2020-05-09 12:55:26 CEST; 2min 8s ago
  Process: 1706 ExecStart=/sbin/ethtool -s eno1 wol g (code=exited, status=0/SUCCESS)
 Main PID: 1706 (code=exited, status=0/SUCCESS)

may 09 12:55:26 estudios-srv systemd[1]: Starting Activate Wake On LAN...
may 09 12:55:26 estudios-srv systemd[1]: wol.service: Succeeded.
may 09 12:55:26 estudios-srv systemd[1]: Started Activate Wake On LAN.

You will notice the service is dead or inactive. This is normal because it is not actually a service and it is not running like a daemon, it starts,  do whatever it has to do and finishes.

If we restart the server now, our service entry will not run at startup because we haven’t enabled it. To do so:

sudo systemctl enable wol.service

Now, you can restart the server and it will wake remotely because “Wake-on: g” should be already set when it boots.

The explanation to “TRULY sticky”

But, why did I titled my post with a “TRULY sticky”?. Well, the reason is that all the recipes I’ve found to do this didn’t work. After rebooting, always the “d” value was set for the “Wake-on” option.

In fact it is not a problem of executing the configuration or not. Although the service entry run on every reboot, it was doing it before the network card was available to be configured.

So, the problem is when to run the network card configuration.

That’s the reason you should put this line in you .service file:

After=network-online.target

To make sure it configures the network card when it’s really available.

I hope it to work for you too.

How to invoke a Qlik Sense task from Node.js

If you just want to start a Qlik Sense task from the command line using curl you shoud visit this other post

How to invoke a qlik sense task from command line

But if you need to check status after execution, get details of the task or any other entity of Qlik Sense (user, app, license, etc) then you have to use a library or package that performs all that kind of funcionality.

I wrote a simple npm package qlik-sense-qrs that can help you to write more complex scripts that will allow you to interact with Qlik Sense in a more efficient way.

The package is available at github and there you will find some basic documentation on how to install and use it.

Basically, all you will need is adding the package to your project

npm install qlik-sense-qrs --save 

Once the package and its dependencies are installed you are ready to use it.

You will need a Qlik Server certificate to authenticate to the QRS API. There are some basic directions on how to get your server certificate in the post mentioned above.

Put both files containing the certificate and the key in a secured folder to make sure only your application has access to them.

Initialization

Now you are ready to import and configure the library to access your server from your Node.js application:

const qsqrs = require('qlik-sense-qrs');

//Set an array of options
let options = {
    host: 'https://yourserver:4242/qrs',
    //Path to your server certificates
    certFile: '../../qlik/client.pem',
    certKeyFile: '../../qlik/client_key.pem'
};
//Apply defined configuration
qsqrs.config.apply(options);
console.clear();
//Show the current configuration
console.log(qsqrs.config.getConfiguration());

That is the simplest way to set the connection options: you create an object containing three mandatory parameters and pass it to the apply method of the config object.

The next two lines are optional. They just show the applied configuration to you, like this:

{
  host: 'https://yourqliksenseserver:4242/qrs',
  certFile: '../../qlik/client.pem',
  certKeyFile: '../../qlik/client_key.pem',
  xrfKey: '43f0a598891d850d',
  certPassword: ''
}

You will notice there are two more parameters that you didn’t specify: xrfKey and certPassword. Let’s explain what they are.

  • xrfKey: This is a 16 characters long string used to prevent CSFR attacks. The key will be sent in the query string and also as a header of the request. If you don’t specify a key, the library will create a random one for you, but if you want to use your own (can’t imagine a reason) all you have to do is adding it to the configuration object or invoking the specific method to set it:
qsqrs.config.setXrfKey('myownqliksense_k');
  • certPassword: when exporting your server certificate you can specify a password to improve the security. At the moment of writing this post the library hasn’t implemented this option, but probably in the next version will be available.

You can set and get any of the configuration parameters by using their corresponding method in the config object (qsqrs.config):

  • qsqrs.config.setHost(‘host url’)
  • qsqrs.config.getHost()
  • qsqrs.config.setCertFile(‘certificate file location’)
  • qsqrs.config.getCertFile()
  • qsqrs.config.setCertKeyFile(‘certificate key file location’)
  • qsqrs.config.getCertKeyFile()
  • qsqrs.config.setXrfKey(‘Your own key’)
  • qsqrs.config.getXrfKey()
Accessing the entities of Qlik Sense

There are two different types of objects in the wrapper. All of them are implemented in the entities object.

  • genericEntity: Most of the objects of QRS API have a get (list or read) endpoint and the structure of the subjacent request is the same for all of them. What the genericEntity does is receiving the name of the Qlik Sense entity as a parameter and perform the requested method. For instance, I don’t know what the odagrequest object is used for, but using the genericEntity object I can perform a call like the following and get some results:
let data = await qsqrs.entities.genericEntity.list('odagrequest');

You can check some more examples in the Github page or inside the test code.

There is an array inside generic-entity.js file that defines which objects you can invoke to:

const allowedEntities = [
    'about', 'analyticconnection', 'app', 'appavailability', 'appcomponent',
    'appcontentquota', 'appseedinfo', 'appstatus', 'binarydelete', 'binarydownload',
    'binarysyncruleevaluation', 'compositeevent', 'compositeeventoperational', 'compositeeventruleoperational',
    'contentlibrary', 'custom', 'custompropertydefinition', 'dataconnection',
    'engineservice', 'event', 'eventoperational', 'executionresult', 'executionsession',
    'extension', 'externalprogramtask', 'externalprogramtaskoperational', 'fileextension', 'fileextensionwhitelist',
    'filereference', 'health', 'license', 'licenseaccessusage', 'lock',
    'mimetype', 'odagenginegroup', 'odaglink', 'odaglinkusage', 'odagmodelgroup',
    'odagrequest', 'odagservice', 'printingservice', 'proxyservice', 'proxyservicecertificate',
    'reloadtask', 'reloadtaskoperational', 'repositoryservice', 'schedulerservice', 'schemaevent',
    'schemaeventoperational', 'selection', 'servernodeconfiguration', 'servernodeheartbeat', 'servernoderole',
    'servicecluster', 'servicestatus', 'sharedcontent', 'staticcontentreference', 'staticcontentreferencebase',
    'stream', 'syncsession', 'systeminfo', 'systemnotification', 'systemrule',
    'tag', 'task', 'taskoperational', 'tempcontent', 'user',
    'userdirectory', 'userdirectoryconnector', 'usersynctask', 'usersynctaskoperational', 'virtualproxyconfig',
];
  • task: By now, this is the only object that implements a method different of a list or read one. It implements a start method, which is the one we are naming since the first line of this post.
Putting it all together

The example below shows how to search for a task based on its name. Then, retrieve its id to invoke the task and after that it waits for 5 seconds and shows the execution result.

//Initialization
const qsqrs = require('qlik-sense-qrs');
let options = {
    host: 'https://yourserver:4242/qrs',
    //Path to your server certificates
    certFile: '../../qlik/client.pem',
    certKeyFile: '../../qlik/client_key.pem'
};
qsqrs.config.apply(options);
//End of initialization

(async() => {
    try {
        let name = 'My task name';
        console.log('Retrieve task data by name');
        let data = await qsqrs.entities.task.list(`name eq '${name}'`);
        let id = data[0].id;
        console.log(`Task id is ${id}\nStart task specifying its id`);
        data = await qsqrs.entities.task.start(id);
        console.log('Task started. Wait for 5 seconds....');
        await qsqrs.util.wait(5000);
        console.log('Retrieve task data by id');
        data = await qsqrs.entities.task.list(`id eq ${id}`);
        console.log('Show execution / progress result');
        let execData = data[0].operational.lastExecutionResult;
        console.log(`\tStatus:\t${execData.status}\n\tStart:\t${execData.startTime}\n\tStop:\t${execData.stopTime}`);
    } catch (err) {
        console.log(err);
    }
})();

Connect from Robo 3T to Atlas

A quick recipe to set the connection options in Robo 3T when accessing to an Atlas MongoDB:

1.- Reveal the connection string from Cluster Atlas console (Connect button at your cluster)

Select the “Connect your application” option and a pop-up window will open

Click on Copy button to copy the connection string.

2.- Open a new connection dialog in Robo 3T

3.- Paste the connection string you copied in step 1 into the textbox close to “From SRV” button and don’t press any button

4.- Now, replace the <password> with your actual password for the specified user

5.- Press now “From SRV” button to import the connection string settings. All the configuration parameters will be applied.

6.- Make sure SSL and SSH options are not checked.

7.- Press Test button and check that the connection works

8.- Give a name to your connection and save it

Laravel + Apache + Windows = 403 Forbidden

Just some advice to prevent “403 – Forbidden” error when loading the root page of a new fresh Laravel installation on Windows/Apache:

1.- Do not modify .htaccess file: the issue is not there.

2.- Use virtualhosts configuration in your httpd.conf file

3.- Try to keep your configuration clear and simple. Once you make it work you can add extra features.

This could be a good starting point to configure your VistualHost entry, and please pay special attention to the DirectoryIndex entry:

 

<VirtualHost *:80>
    DocumentRoot "/var/www/html/yoursite/public"
    DirectoryIndex index.php
    ServerName yoursite.local
    <Directory "/var/www/html/yoursite/public">
        Options Indexes FollowSymLinks
        AllowOverride All    
        Require all granted
    </Directory>
</VirtualHost>