Setup MQTT Broker, IoT Devices and Security
Some time ago I heard an interesting talk:
The S in IoT stands for Security.
There is no ‘S’ in IoT? Correct, and exactly this is very often the problem. Even if you think “My IoT devices are all behind my firewall, I’m safe”: No, because most attacks happens from inside the network, by malware, social engineering attacks, unhappy employees or bad friends, …
So my goal is, to setup my MQTT broker which is as safe as possible. So you need to authenticate the client to publish or subscribe to topics. And of course we don’t want that the password is transferred in plaintext over the network, so we provide TLS encryption.
Knowing about the problem, that most of my IoT devices are not able to use TLS 😒 (especially my Shelly devices), the challenge is to setup the MQTT broker in a way that it works with TLS and without. And additional it is necessary to put this IoT devices into a separate network, where nobody else has access too.
In the last blog I talked already how to setup Prometheus & Grafana as containers, now I need a MQTT broker, since Prometheus is not able to scrape most of my IoT devices. Again, the MQTT broker should run in a container like Prometheus and Grafana, managed by podman kube and run of course on openSUSE MicroOS.
So the big picture is, depending on the device:
- IoT device sends data to the MQTT Broker
- Grafana uses Prometheus and InfluxDB as datasources
Why Prometheus and InfluxDB? Many of my devices send only a new MQTT message with new values, if the difference to the old value is big enough. This can take several hours (or even days), something where I have big problems to handle that with Prometheus, which scrapes the metrics in a regular interval. In my experience, InfluxDB is also better suited if you want to measure the power created by our balcony power plant.
I used Mosquitto as MQTT Broker. The mosquitto process does not run as root, but a special user in the container, like we had already with Grafana. So let’s create a system user for this user ID, so that it does not come to a clash and somebody else is allowed to read and modify the MQTT data:
useradd -u 1883 -r mqtt -d /srv/mosquitto
For the certificate I use Let’s Encrypt, since I had already one for the machine. Not using a self signed certificate has the advantage, that the possibility that an IoT device will accept it is much higher. Else you need to teach the IoT device about your own key so that it trust it.
My directroy structure looks like:
├── mosquitto/config -> for configuration and password file
├── mosquitto/data/ -> for the persistent data
├── mosquitto/log/ -> for the log files
└── certbot/... -> for the certificate
My mosquitto configuration file
/srv/mosquitto/config/mosquitto.conf looks like:
log_dest file /mosquitto/log/mosquitto.log
It’s good to know, that the main directory of mosquitto inside the container is
/mosquitto, so everything is mapped into subdirectories of that directory.
At first we listen to port
1883. This is the default port for unencrypted connections.
8883 is the standard port for TLS encrypted connections. So we need to specify, where to find the certificate to use for this. In my case, everything for this is stored in
/srv/ceertbot/etc, so I mount this directory inside the container on
As I don’t want to loose data if I restart the container or reboot the server, I set
persistence to true, which means all connection, subscription and message data will be written to the disk at the location specified by
The log messages are printed on stdout and in a log file, which can be found on disk in
To have an overview, who connects to the server, connections are logged, too.
The last entry is the password file.
MQTT user account with password
The MQTT broker should be configured to require client authentication via username and password, so that not everybody can connect to it. The username and password combination is transmitted in clear text, that’s why it is so important to secure the connection with TLS (see above).
However having username/password authentication does provide an easy way of restricting access to the broker.
The username can also be used in restricting access to topics using an ACL (Access control list). For this reason it is important to not only generate one username/password pair, but devide your MQTT clients in usefull groups and give them an own username/password, depending on the access they really need.
To create accounts, we need to create at first an empty password file (we could let
mosquitto_passwd do that, but if you call that several times to create different accounts, there is the risk to overwrite all existing entries). Afterwards we run
mosquitto_passwd from the mosquitto MQTT container to create the entry. With this command, the file
password.txt with the username/password pairs can be found in
podman run --rm -it -v /srv/mosquitto/config:/mosquitto/config docker.io/eclipse-mosquitto mosquitto_passwd /mosquitto/config/password.txt new_user
Password: <enter password>
Reenter password: <repeat password>
For various features I need Podman as container runtime. Podman comes with a very nice feature: podman pod and podman kube, which uses kubernetes yaml files, at least if they don’t use too advanced features.
podman kube play file looks like:
- name: broker
- containerPort: 1883
- containerPort: 8883
- mountPath: /mosquitto/config
- mountPath: /mosquitto/data
- mountPath: /mosquitto/log
- mountPath: /mosquitto/certs
This configuration uses the upstream mosquitto MQTT broker container. We open port 1883 for unencrypted MQTT connections and port 8883 for MQTT over TLS. Additional we have four volumes:
- for configuration and password file
- for persistent data
- for log files
- for the Let’s Encrypt certificates
That’s already all we need.
Now we just need to start the containers:
podman kube play mqtt.yaml
podman pod ps should show you at least one pod:
POD ID NAME STATUS CREATED INFRA ID # OF CONTAINERS
3e2f71431b37 mqtt Running 16 seconds ago bb105b7a6122 2
Start container with every boot
While the containers are now running, we need to make sure that they will be started with the next reboot, too. For this, podman comes with a very nice and handy systemd service:
This service will not only start the pod, but also makes sure, that the containers are current and update them if necessary.
The configuration file with complete path is passed as argument. The path needs to be escaped, but for this there is a
So the final command to enable the systemd service would be:
systemctl enable "podman-kube@$(systemd-escape /<path>/mqtt.yaml).service"
Configure IoT Device (Shelly Plug S)
We have a MQTT broker, now we need an IoT device, which sends something to it 😉 I used a Shelly Plug S for the beginning. Below “Internet & Security” and “ADVANCED - DEVELOPER SETTINGS” you can enable MQTT.
Fill out the following fields:
- Username: use the account you created before
- Password: the same, use as created above
- Server: the name or IP and port of your MQTT server
- Use custom MQTT prefix: for easier identification and better parsing of MQTT topics, I give all my IoT devices names which allows me to identify the device.
Now, that we have a device sending MQTT messages, we should verify that they reach the MQTT Broker. For this we start a commandline tool and subscribe to the topics:
mosquitto_sub -v -h mqtt.example.com -p 8883 -t "#" -u <account name> -P <password>
-p 8883 port option means, we use MQTT over TLS. This is a good first test, if the certificate setup and configuration is correct. The
-t "#" means, we subscribe to all topics.
Shortly you should see something like:
Which means: it’s working 😄
In the next blogs I will explain how to setup an InfluxDB timeseries database, how to store the data in the database and how to visualize them in Grafana.