Example of custom web app in HTML5 and plain JavaScript: face detection on a video stream from a camera in the network

Some of you may be surprised to know that the UV4L Streaming Server not only provides its own built-in web applications and services discussed extensively in other tutorials, but also allows the user to deploy her own custom web apps. It’s “two servers in one” (listening to two different ports).

In this example we will see how to use the Streaming Server to serve a custom web application which does the following in the browser:

  • contact the WebRTC web service on the server itself to ask for a live 30fps video stream from a camera;
  • embed the video stream into an HTML5 page in the browser;
  • process the video in real-time (face detection in this case) with the OpenCV.js library in plain JavaScript.

No third-party plugin’s for the browser are required, as this is entirely based on standards.

For the sake of simplicity, in this example we will deploy this web app on a Raspberry Pi with a camera module connected to it, although the procedure would be the same with any other supported architectures or devices. It will be left as an exercise for the reader to remove the post-processing part, if it is not needed.

First of all, install all the packages required according to these instructions: uv4l, uv4l-server, uv4l-webrtc, uv4l-raspicam, uv4l-raspicam-extras and uv4l-demos.

Log into the Raspberry Pi, edit the default configuration file /etc/uv4l/uv4l-raspicam.conf, comment out the following options and leave them as follows:

server-option = --enable-www-server=yes
server-option = --www-root-path=/usr/share/uv4l/demos/facedetection/
server-option = --www-index-file=index.html
server-option = --www-port=80
server-option = --www-webrtc-signaling-path=/webrtc

To know what those options mean please refer to the server manual. Restart UV4L with the following command:

$ sudo service uv4l_raspicam restart

You can now connect to the Streaming Server with the browser. The URL should be something like http://<rpi_address>, where <rpi_address> has, of course, to be replaced with the real hostname or IP address of the Raspberry Pi in your network. A page like the following should appear in the browser:

facedetection

As you are downloading this web app from server, the Web Service URL field should already be filled with its correct address, so just press the Start Streaming button to receive and see the live stream inside the white box and press the Toggle Face Detection button to turn on and off face detection on-the-fly on the received video. Whenever a face is detected, a red rectangle surrounding it should appear in the image:

facedetection2

The source code of the web app is installed in the filesystem under the path /usr/share/uv4l/demos/facedetection/, which consists of:

  • index.html, an HTML5 page for the UI elements (e.g. <video> and <canvas>);
  • main.js, boilerplate code for callbacks onuser actions (e.g. start the streaming on mouse click);
  • signalling.js, the WebRTC signaling protocol over web socket (e.g. negotiate the resolution, the H264 compression and other options) ;
  • face-detection.js, code that makes use of the OpenCV.js library to detect faces on the canvas;
  • cv.js, cv.data, the OpenCV.js library itself.

The signaling aspect is probably the most interesting part, in that is the key for UV4L to be integrated with other third-party WebRTC-based frameworks (remember that WebRTC is not limited to browsers).

Of course, this is just the tip of the iceberg. You can enhance or fork this simple web app in order to support, for example, audio processing in the browser as well or, more in general, bidirectional streaming of audio/video/data that allows to implement funny things like FPV robots as shown in this demo. For a more comprehensive example about 2-way audio/video/data you can always take inspiration from source code of the built-in page available at http://<rpi_address>:8080/stream/webrtc by default.