How to communicate an ESP32 with a web page via MQTT

New post about communication with the ESP8266 and ESP32. This time we’ll see how to communicate a webpage served from the ESP32 via MQTT.

We have already seen How to use asynchronous MQTT and How to send text in Json format via MQTT. Those communications were between devices, between backends. Now we want to communicate it with a webpage served to the client, with the frontend.

This is not usually (or shouldn’t be) so common, because we have many ways to communicate an ESP8266/ESP32 acting as a backend with the frontend. We have seen Ajax calls and communication via Websockets.

But, if our system is using MQTT, it can be useful to use the same system to inform the frontend. So, in short, we want to make a web client for the MQTT protocol, and serve this client from the ESP32 itself.

This is where we encounter the first problem: how can we communicate a webpage, which lives in its “web world” of HTTP calls, with communication via MQTT.

Communicating a Webpage via MQTT

A webpage communicates via HTTP, which is a protocol that operates on the Application layer (layer 7 of the OSI model). This, in turn, operates over TCP which acts on the transport layer (layer 4). That is, with few exceptions or future changes (HTTP 3?), a webpage can only communicate via HTTP requests, websockets, or WebRTC.

And how does MQTT fit into this? MQTT is also an Application layer protocol (layer 7) that generally operates over TCP/IP. In summary, MQTT goes directly against a socket and will not enter an HTTP request.

So can’t we connect them? Well, here come Websockets to the rescue. We can configure MQTT to work over Websockets, which as we said, we can manipulate from a webpage.

But first we need to configure our broker to accept Websockets. Whether this is possible, or how to configure it, will depend on the broker we are using. In the rest of the post, we will see it as if we had a local installation of Mosquitto as we saw in this post.

And, on the other hand, we will need a JavaScript library that allows us to read this MQTT communication via Websockets from the served webpage.

Configuring Mosquitto to Use Websockets

If we are using Mosquitto as an MQTT broker, communication via Websockets is disabled by default. So we must activate it in the configuration to be able to communicate from a webpage.

To do this, edit the ‘mosquitto.conf’ file, which you will have in the folder where you installed Mosquitto. Here, look for the “Extra listeners” section and uncomment the Websockets lines by removing the ’#’ symbol from the beginning of the line.

This way we have added an extra listener for Mosquitto, via Websockets acting on port 9001.

Paho JavaScript Client Library

The other component we need for our communication is a JavaScript library for MQTT. Several libraries exist, but the most commonly used is the Paho JavaScript Client library from Eclipse.

It is a library available in several languages, including Java, JavaScript, Python, C/C++, Rust, and Golang. More information on the project page https://www.eclipse.org/paho

Code Example

Now that we have configured our broker to accept communication via Websockets, and with our Paho JS Client library, it’s time to put it all together to see an example.

The program we will upload to the ESP32, responsible for MQTT communication, is basically the same as in the previous post. Therefore, we will only comment on the differences, and in case of doubts, you can consult the previous tutorial.

#include <WiFi.h>
#include <SPIFFS.h>
#include <ESPAsyncWebServer.h>
#include <AsyncMqttClient.h>
#include <ArduinoJson.h>

#include "config.h"  // Replace with your network data
#include "Server.hpp"
#include "MQTT.hpp"
#include "ESP32_Utils.hpp"
#include "ESP32_Utils_MQTT_Async.hpp"

void setup(void)
{
  Serial.begin(115200);
  SPIFFS.begin();

  delay(500);

  WiFi.onEvent(WiFiEvent);
  InitMqtt();

  ConnectWiFi_STA();
  InitServer();
}

void loop()
{
  PublishMqtt();

  delay(1000);
}

AsyncWebServer server(80);

void InitServer()
{ 
  server.serveStatic("/", SPIFFS, "/").setDefaultFile("index.html");

  server.onNotFound([](AsyncWebServerRequest *request) {
    request->send(400, "text/plain", "Not found");
  });

  server.begin();
  Serial.println("HTTP server started");
}

Now we go to what’s really new, which is the page we are serving, and which must act as a MQTT communication client.

<!doctype html>
<html lang="">

<head>
  <title>ESP32 MQTT</title>
  <meta charset="utf-8">
  <meta http-equiv="x-ua-compatible" content="ie=edge">
  <title></title>
  <meta name="description" content="">
  <meta name="viewport" content="width=device-width, initial-scale=1">
</head>

<body>
  <div id="app">
  </div>

  <script type="text/javascript" src="./vendor/nativeWs.min.js"></script>
  <script type="text/javascript" src="./vendor/mqttws31.min.js" ></script>

  <script type="text/javascript" src="./js/API.js"></script>
  <script type="text/javascript" src="./js/app.js"></script>
</body>

</html>

function onConnect() {

  var options = {
    qos: 0,
    onSuccess: onSubSuccess,
    onFailure: onSubFailure
  };
  client.subscribe('hello/world', options);
}

function onFailure(message) {
  console.log(message)
}

function onConnectionLost(responseObject) {
  if (responseObject.errorCode !== 0) {
    console.log("onConnectionLost:" + responseObject.errorMessage);
  }
}

function onMessageArrived(message) {
  console.log(message)
  var topic = message.destinationName;
  var payload = message.payloadString;

  let json = JSON.parse(payload);

  var element = document.getElementById('app');
  var newElement = document.createElement('div');
  newElement.appendChild(document.createTextNode(json.data));
  element.appendChild(newElement);
}

function onSubFailure(message) {
  console.log(message)
}

function onSubSuccess(message) {
  console.log(message)
}

function createGuid() {
  return 'xxxxxxxx-xxxx-4xxx-yxxx-xxxxxxxxxxxx'.replace(/[xy]/g, function (c) {
    var r = Math.random() * 16 | 0,
      v = c === 'x' ? r : (r & 0x3 | 0x8);
    return v.toString(16);
  });
}

client = new Paho.MQTT.Client("192.168.1.150", 9001, createGuid())

var options = {
  onSuccess: onConnect,
  onFailure: onFailure
};

client.onConnectionLost = onConnectionLost;
client.onMessageArrived = onMessageArrived;

client.connect(options);

Result

With this, we upload both the program and the web to the ESP32, run everything, and access the web page served from the ESP32. If everything has gone well, we will see the received messages appear as they are added to the served web page.

In this example, it simply contains the value of Millis(). Obviously, in a real project, the JSON-formatted data would contain more information. We will see this in the last entry of the series. But first, in the next one, we will see how to do this using VueJS. Until next time!

Download the code

All the code from this post is available for download on Github.

Version for the ESP8266: https://github.com/luisllamasbinaburo/ESP8266-Examples

Version for the ESP32: https://github.com/luisllamasbinaburo/ESP32-Examples