Build

DIY Dashboard Camera with Raspberry Pi and AngularJS Part 2

Michael Carroll on Jul 5, 2016
DIY Dashboard Camera with Raspberry Pi and AngularJS Part 2

In part 1 of the Raspberry Pi Dashcam Series, we talked about how to create a powerful and low-cost embedded computing platform for “always on” live dashboard camera broadcast using the Raspberry Pi 2 Model B and the Raspberry Pi 8MP Camera Module. With this hardware setup, we can broadcast high- or low-quality images at a fixed interval and have them immediately picked up by our receiver user interface. To make things fun on the UI side, we’ve decided to use AngularJS to highlight PubNub’s new and improved AngularJS SDK – we’ve had a great time using it so far.

Raspberry Pi Dashcam

AngularJS is a great choice for small and large web applications (although Angular aficionados will definitely have their share of war stories to tell). In this case, our sample app is just about 80 lines of HTML and JavaScript code. This makes it easy for beginners to pick up the basics of real-time image streaming, and also give experienced AngularJS developers an indication of how to scale the techniques to larger applications (by setting up routes, adding more controllers, etc).

Overall, we think it’s pretty awesome that developers can assemble a mobile computing platform and user interface in less than 150 lines of total code (65 lines of Node.js on the Raspberry Pi, and 85 lines of HTML+JavaScript for the UI). Hopefully you’ll think so too!

The Mobile DashCam Platform

Just to recap from the previous blog entry, here’s what our mobile DashCam looks like:

The Mobile Platform:

Not too shabby considering this whole platform can be assembled for around $65-$100 depending on how fancy you want to get – the possibilities are just about endless for mobile data collection and intelligence.

Raspberry Pi Setup

How it Works

Given an autonomous compute platform including power and real-time data connectivity, we now have the capability to deploy our DashCam app to vehicles that:

  • Takes images at a pre-defined interval (say, every 10-30s)
  • Splits them up as necessary to fit the PubNub data stream model (or uploads them to an alternative Cloud Storage provider)
  • Publishes the images in real-time to any number of subscribers, from one to tens of millions.
  • Performs additional image processing as desired, including feature detection (such as facial recognition or license plate recognition)

Sound interesting? Check out the previous article including the sample Raspberry PI DashCam NodeJS code if you haven’t already!

Getting Your PubNub Keys

The first thing to note is that you’ll want to create a free developer account at PubNub and create a new app (which will include your unique publish and subscribe keys) if you don’t have those yet. You’ll be able to run all the examples out-of-the box, plus much much more. NOTE: make sure your sender and receiver are using publish and subscribe keys from the same app!

Background Techniques

Before we go any further, let’s review a couple underlying techniques we’ll use in our DashCam UI web app. We use two ideas that folks might not have seen before when working with real-time image feeds: Data URIs and Chunking (or splitting).

  • Data URIs are a nifty browser innovation intended to make it easier to load web page resources by embedding a base64-encoded binary string and MIME type directly into a web page, obviating the need for a remote data request. (You may have seen this used before for fonts, sprites, and other resources.)
  • Chunking is just the practice of splitting a large image data string into smaller strings of a given length (or smaller). In our case, images greater than 320×240 pixels are typically greater than 32k base64-encoded (remember that the base64 encoding in data URIs inflates the original size by 33%), so we need to split the data URI for the image into chunks of 28k or so to fit within PubNub’s message size.

Alternatively, if you don’t wish to use Data URIs, you can just publish images to an alternative hosting provider (such as Cloudinary, Amazon S3, or Google Cloud Storage), and send URLs in your messages instead. (This also obviates the need for chunking.)

Hopefully this helps in understanding the upcoming code. If not, feel free to reach out and ask for help!

The Dashcam User Interface

The dashcam user interface web application is pretty lightweight, weighing in at < 85 lines of HTML and JavaScript code (CoffeeScriptand Jade aficionados will see an opportunity to shave at least another 20-30 lines off of that). We create a file calleddashcam_ui.html on our favorite code hosting platform.

The basic flow of the user interface is:

  • Create an unordered list of image slots to contain the latest image and timestamp from each device
  • Initialize PubNub and subscribe to the photo update channel
  • Make a call to the PubNub channel message persistence to pre-populate the latest image (optional)
  • When new messages come in:
    • Take note of the UUID and the image chunk’s position within the current set of chunks
    • If it’s the first chunk (i == 0), start building a new data-uri for the image
    • If it’s a middle chunk (i < n – 1), append it to the in progress data-uri
    • If it’s the last chunk (i == n – 1), append it to the in-progress data-uri and set the image to be the latest (so the view will pick it up)

That’s it!

As we said, if we wanted to avoid the whole data URI and chunking thing, we could have published our image content to a cloud provider such as Cloudinary, Amazon S3, or Google Cloud Storage. In this case, we decided to publish image chunks directly to PubNub because we love the idea of having the content in-line and ready for browsers to display, plus the added control of usingPubNub’s built-in encryption and/or Access Manager in the future if we want to restrict permissions in one place.

Ok, with that said, here’s the code (also available as a gist and codepen for your convenience):

<!doctype html>
<html>
<head>
  <script src="https://cdn.pubnub.com/pubnub-3.15.1.min.js"></script>
  <script src="https://ajax.googleapis.com/ajax/libs/angularjs/1.5.6/angular.min.js"></script>
  <script src="https://cdn.pubnub.com/sdk/pubnub-angular/pubnub-angular-3.2.1.min.js"></script>
  <link rel="stylesheet" href="//netdna.bootstrapcdn.com/bootstrap/3.3.6/css/bootstrap.min.css">
</head>
<body>
<div class="container" ng-app="PubNubAngularApp" ng-controller="DashCamCtrl">
<h4>Online Devices</h4>
<ul class="list-unstyled list-inline">
  <li ng-repeat="(uuid, data) in received">
    <img style="width:400px;height:300px" ng-src="{{data.image}}" />
    <br />
    <b>{{data.uuid}}</b>
    <br />
    {{data.ts}}
  </li>
</ul>
</div>
<script>
//
// Set Up Your Angular Module & Controller(s)
//
angular.module('PubNubAngularApp', ["pubnub.angular.service"])
.controller('DashCamCtrl', function($rootScope, $scope, Pubnub) {
  $scope.received   = {};
  $scope.inprogress = {};
  // make up a channel name
  $scope.channel  = 'dashcam-demo';
  if (!$rootScope.initialized) {
    Pubnub.init({
      subscribe_key: 'YOUR_SUB_KEY',
      publish_key: 'YOUR_PUB_KEY',
      ssl:true
    });
    $rootScope.initialized = true;
  }
  var msgCallback = function(payload) {
    $scope.$apply(function() {
      if (payload.uuid) {
        if (!$scope.inprogress[payload.uuid]) {
          $scope.inprogress[payload.uuid] = payload;
        }
        if (payload.i == 0) {
          $scope.inprogress[payload.uuid].imageTmp = payload.imgPart;
        } else {
          $scope.inprogress[payload.uuid].imageTmp = $scope.inprogress[payload.uuid].imageTmp + payload.imgPart;
        }
        if (payload.i == (payload.n - 1)) {
          $scope.received[payload.uuid] = $scope.inprogress[payload.uuid];
          $scope.received[payload.uuid].image = $scope.inprogress[payload.uuid].imageTmp;
          $scope.inprogress[payload.uuid] = null;
        }
      }
    });
  };
  var multiMsgCallback = function(payloads) {
    payloads[0].forEach(function(x) { msgCallback(x); });
  };
  // Subscribe to the Channel (to receive updates)
  Pubnub.subscribe({ channel: $scope.channel, callback: msgCallback });
  // Retrieve history (to initialize with most recent history)
  Pubnub.history({ channel:$scope.channel, limit:16, callback: multiMsgCallback });
});
</script>
</body>
</html>

Okay, that’s a lot of code to digest all at once – suffice it to say, it has all of the code necessary for the view and behavior of the app. This is a cute sample app for < 85 lines of code; In a real app, you’d probably split the JS code and partial HTML views out for a bit nicer modularity. Moving right along, let’s take a look at the UI code one section at a time.

<!doctype html>
<html>
<head>
  <script src="https://cdn.pubnub.com/pubnub-3.15.1.min.js"></script>
  <script src="https://ajax.googleapis.com/ajax/libs/angularjs/1.5.6/angular.min.js"></script>
  <script src="https://cdn.pubnub.com/sdk/pubnub-angular/pubnub-angular-3.2.1.min.js"></script>
  <link rel="stylesheet" href="//netdna.bootstrapcdn.com/bootstrap/3.3.6/css/bootstrap.min.css">
</head>

Ok, this should be familiar to all the frontend developers out there. We’re just declaring an HTML document and setting up our includes. They are:

Now that we have our header and required resources out of the way, let’s look at the view code:

<body>
<div class="container" ng-app="PubNubAngularApp" ng-controller="DashCamCtrl">
<h4>Online Devices</h4>
<ul class="list-unstyled list-inline">
  <li ng-repeat="(uuid, data) in received">
    <img style="width:400px;height:300px" ng-src="{{data.image}}" />
    <br />
    <b>{{data.uuid}}</b>
    <br />
    {{data.ts}}
  </li>
</ul>
</div>
<script>

Wow, that’s pretty nice! Our body element just contains the standard bootstrap container DIV, and declares ng-app andng-controller attributes for compact linking up with the upcoming AngularJS code. In a more complicated app, we’d probably leave out the ng-controller attributes in favor of using Angular Routes. On that note, let’s move on to the Angular code!

//
// Set Up Your Angular Module & Controller(s)
//
angular.module('PubNubAngularApp', ["pubnub.angular.service"])
.controller('DashCamCtrl', function($rootScope, $scope, Pubnub) {
  $scope.received   = {};
  $scope.inprogress = {};
  // make up a channel name
  $scope.channel  = 'dashcam-demo';
  if (!$rootScope.initialized) {
    Pubnub.init({
      subscribe_key: 'YOUR_SUB_KEY',
      publish_key: 'YOUR_PUB_KEY',
      ssl:true
    });
    $rootScope.initialized = true;
  }

These lines shouldn’t look too scary – we’re just declaring an Angular module for our app (named PubNubAngularApp) containing one controller (named DashCamCtrl). The DashCamCtrl controller needs three resources to do its job: $scope – which we all know and love as the controller scope object (providing data and functions to our view), $rootScope – the longer-lived cousin of $scope(providing a root-level long-lived context for the entire app), and PubNub – our friendly real-time data streaming service in Angular.

We initialize two collections in the scope for incoming image data: inprogress – which holds temporary image data in progress as we concatenate chunks, and received – which holds the latest completed image for each device. In each of these collections, the first level of key will be the device UUID (this is set up in the mobile dashcam.js code for each device).

The Pubnub.init() call is very straightforward. We provide our publish and subscribe keys (actually, for this receiver app, we just need subscribe), specify ssl:true for security purposes, and set $rootScope.initialized = true to make sure we only create one PubNub connection.

Now that we have a connection to PubNub, how do we use it? The first thing we should think about is what happens when a message comes in. So we declare a function called msgCallback that contains the logic for processing an incoming message.

var msgCallback = function(payload) {
    $scope.$apply(function() {
      if (payload.uuid) {
        if (!$scope.inprogress[payload.uuid]) {
          $scope.inprogress[payload.uuid] = payload;
        }
        if (payload.i == 0) {
          $scope.inprogress[payload.uuid].imageTmp = payload.imgPart;
        } else {
          $scope.inprogress[payload.uuid].imageTmp = $scope.inprogress[payload.uuid].imageTmp + payload.imgPart;
        }
        if (payload.i == (payload.n - 1)) {
          $scope.received[payload.uuid] = $scope.inprogress[payload.uuid];
          $scope.received[payload.uuid].image = $scope.inprogress[payload.uuid].imageTmp;
          $scope.inprogress[payload.uuid] = null;
        }
      }
    });
  };

Thanks to Angular’s processing model, we need to wrap our logic in $scope.$apply() so that the UI can pick up scope data changes.

  • First, we ensure that the message contains a uuid for the remote device; otherwise, we won’t be able to link the data up properly.
  • Then, if this is the first chunk, we start a new inprogress image data object with the payload.imgPart data
  • Otherwise, we append the payload.imgPart data to the current inprogress data for the uuid
  • Then, if this is the last chunk of the set, we set the received data for the uuid to the specified image and clear out theinprogress object in preparation for the next one

Still following? Hopefully that wasn’t so bad! The next feature to consider is initialization – what happens when we have just loaded the app, but before the remote device has published an image? If we’d like to avoid the wait, we can employ PubNub’s persistence feature to replay the latest messages as soon as we initialize the app. To do that, we’ll need a bit of glue code to handle processing that first “bunch” of messages that comes in.

var multiMsgCallback = function(payloads) {
    payloads[0].forEach(function(x) { msgCallback(x); });
  };

This key thing here is that since the persistence function returns a whole bunch of messages at once as an array, we need to look at the zero-th element of the payloads object to make sure we’re processing that entire array. If you’re interested in diving deeper, the persistence call also returns time tokens that can be used for pagination (for example, if we’d like to extend our UI with a “rewind” feature).

Now that we have the callback logic covered and ready to use, we just hook it up to PubNub’s live data stream functionality.

 // Subscribe to the Channel (to receive updates)
  Pubnub.subscribe({ channel: $scope.channel, callback: msgCallback });
  // Retrieve history (optional: to initialize UI with most recent history)
  Pubnub.history({ channel:$scope.channel, limit:16, callback: multiMsgCallback });
});

Since we’ve declared our single and multiple message callbacks as handy stand-alone functions, we have everything in a nice form to make things easy to connect on the PubNub side.

For the live image feed, we call Pubnub.subscribe() with the data feed channel (make sure this is the same channel as the remote device!), and provide the msgCallback function we described earlier.

For our optional initialization handling, we call Pubnub.history() specifying the data feed channel and a small number of image chunks (we chose 16, since most images we came across did not have more than 16 chunks). We pass in the multiMsgCallbackdescribed earlier, which takes care of invoking the msgCallback once for each historical image.

Finally, we just close out the script tag and HTML content.

</script>
</body>
</html>

That’s it! Too easy? We hope so – not bad for about 80 lines of HTML & JavaScript!

If everything is working properly, you’ll start seeing images come in for each remote device that is running the dashcam.js code, just like this:

Next Steps

Once you have a good feel for the techniques in this example, it’s easy to consider extending the sample web app with additional features.

  • History/Rewind. To implement a history of image data for each device, we could use an intermediate array to hold multiple image data objects as well as the current image index to display. We could then update the view code to display the latest image by default, including buttons to move the image index earlier or later in the series.
  • Time-lapse/”Playback” looping. This is where the fun starts! Once we have an array of images, we can animate the image slots for each device using a JavaScript setInterval() timer and advancing the image index accordingly. When the index moves beyond the array limit, we just reset it to zero to implement the video “loop”.
  • Communcation back to the vehicle. We have the full power of real-time data streams here, why not use it? We could easily send chat messages back to the vehicle, and even use text to speech on the Raspberry Pi for distraction-free delivery.

One thing to mention about the first two advanced image features above is that they can lead to significant memory consumption in the browser. It’s worth making the history arrays fixed-length and coming up with a strategy for pruning images (once per minute, etc) to reduce the burden on the client browser.

What’s happening under the hood

So, what’s happening under the hood here? PubNub is providing our dashcam UI with real-time data streaming around the globe – each message will be received by any subscribers within 250ms of being sent. It’s pretty amazing to say that with PubNub’s scalability, it doesn’t matter if you’re tracking one vehicle or one million vehicles, and/or have one listener or tens of millions of camera feed subscribers.

We hope you enjoyed this blog entry and caught a glimpse of the power available in the Raspberry Pi 2 platform when combined with PubNub real-time data streaming capabilities. Please reach out if you have any questions, or just feel like sharing your latest cool app!

Resources