How to Smooth Your Location Data & Snap to Roads

We have many customers who use PubNub to exchange geolocation data (latitude, longitude) and track assets in real-time. For example, they can track food delivery, manage a fleet of vehicles, or monitor when riders will arrive safely at their destination.

All delivery solutions face a common set of challenges, and some of the common questions we receive from developers are:

“How do I stop location fixes from ‘jumping around’?”
“How do I snap the vehicle to the road?”
“What if the vehicle GPS location is bad? Can I smooth that data?”

This how-to aims to answer all these questions.

TL;DR

PubNub Functions allow you to transform raw location data in real-time to conform to the road. Start the demo below to see how Real GPS data gathered from a moving vehicle is corrected in real-time Since a single point at an intersection is ambiguous, previous points in your route are considered to ensure the right road is followed.

The application is also available at https://pubnub-roads-api-demo.netlify.app/ | Source Code.

Please note the front-end is always drawn with Google Maps, only the back end logic will switch between Google and Mapbox when you select the appropriate radio button.

For an explanation of how this application works, jump to the ‘Snap to the Nearest Road’ section.

Is this real data, or faked? The GPS coordinates are real, I recorded them myself from my Android phone whilst riding in an Uber. The corrected location is generated live everytime you run the demo by either Google or Mapbox, it is not predefined - feel free to check the code :)

Where to Start?

A naive Google search for “How to smooth GPS data” will return mostly mathematical answers and can potentially mislead developers. You are quickly led down the path of signal filtering and creating a Kalman filter to smooth your GPS track, and it is very easy to lose sight of the end goal.

Using a Kalman Filter

Many blogs that discuss the problem of noisy location data will make a sweeping statement such as “create a GPS point smoothing function, such as a simple Kalman filter,” and the Wikipedia article even includes an ‘Example application’ section that discusses using a filter to determine the precise location of a truck. Developers going down this route will then probably stumble on this ubiquitous Stack Overflow post or open source projects such as KalmanJS which appear suitable at first glance.

Developers quickly come up against practical issues in their implementation however; for example, the KalmanJS project is a 1D filter and so not ideal for location data, as emphasized by the author in his accompanying blog. The Stack Overflow code example is ideal for assets moving at a constant speed but breaks down as the velocity changes between updates (if you dive into the comments of that post, there is a discussion about why you cannot just naively change the velocity value based on the asset’s current speed)

So, why do all resources about smoothing location talk about using a Kalman Filter? In some situations, a Kalman filter is ideal. Consider a vehicle entering a tunnel or driving down a street with an intermittent line of sight to satellites. Using a combination of dead reckoning based on the last known position, speed, and velocity of the vehicle, along with GPS coordinates, a reliable tracking algorithm can be developed based on Kalman Filtering. A careful reading of the Example application from Wikipedia mentioned earlier shows that this is the exact scenario being discussed, NOT extrapolating the location based purely on GPS readings.

The maths required to create a reliable multi-dimensional filter is beyond the scope of this article and probably overkill for most developers creating a solution to track moving assets.

Location APIs

Developers have a wide choice of location APIs depending on their chosen platform, for example, the HTML5 geolocation API for web, the Android Fused location provider, or Apple’s Core Location Framework.

Location technologies and mobile phones (where most people first encounter location hardware) have been around for a long time. It is important to understand how the technology has evolved, especially since many search results on Stack Overflow are from nearly a decade ago.

Early location APIs only returned data from a single source to your application. For example, you could request data from the GPS chip, but then it was up to you as the developer to decide how best to handle that data.

The most significant enhancement to location accuracy came by amalgamating different location sources, not from smoothing data. For example, on Android, the Fused location provider’ will amalgamate the location data from GPS with other sources such as nearby WiFi, cellular signals, and Bluetooth signals.

Do I need to smooth data from the Location API? No, high-level APIs such as those returned by Android and iOS are already smoothed and processed, so it is not necessary to apply further smoothing. Even ‘raw’ GPS data is subject to smoothing by the chip firmware that reports its data to the higher-level OS, so no benefit will be gained from smoothing it further.

If all location APIs are smoothed, why does my location still jump around?

Moving at a low speed will never give a stable position, no matter how much the data is smoothed owing to technological limitations.

If you are tracking mostly stationary assets, the best way to avoid ‘jittery’ location fixes is to ignore any small changes that the API reports. Some high-level APIs make this especially easy with dedicated methods to set the minimum update distance before a location update is triggered. This will not enhance the accuracy of the location fix but will make it appear more stable.

What if you are tracking a moving asset, and the route traced by the location fixes is not smooth? The most straightforward approach to this problem is a pragmatic one, assuming the asset you are tracking is a vehicle moving along a road, just snap its location to the most likely road they are following.

Snap to the Nearest Road / Map Matching

There are two popular choices for snapping your location to the nearest road:

Google offers a set of backend Roads APIs that take care of all the difficult road network logic for you. The Snap to Roads API will take a series of points and return the most likely route that the vehicle followed along a road or road segment
Mapbox offers an entire suite of APIs for all aspects of locationing. Their Map Matching API will snap fuzzy, inaccurate GPS coordinates to the OpenStreetMap road network.

By providing a series of points to the API, it is able to disambiguate which road the vehicle followed at intersections or where the vehicle crosses flyovers and underpasses.

The downside is that each client is responsible for smoothing its own location data, increasing traffic, and complicating the solution.

Snap to the Nearest Road with PubNub

Let’s say your clients are reporting their location data with PubNub by sending the last set of recorded location fixes

1
2
3
4
5
6
7
  pubnub.publish({
    channel: 'loc.vehicle',
    message: {
      provider: 'google',  //  or 'mapbox' 
      'points': [recordedPoints]  //  Array or [{"lat":.., "lng"...}]
    }
  })

Create a PubNub Function

Log into the PubNub admin portal and select the ‘Functions’ tab
Select the application and keyset you wish to use
Click ‘+ CREATE NEW MODULE’
Give the module and name and description and hit ‘create’.
Select the module you just created
Select ‘+ CREATE NEW FUNCTION’
Give the function a name and select the Before Publish or Fire event type
Choose a channel name (this example uses loc.*) then hit ‘create’.
This will bring you to the function creation window.
Specify the test payload as follows:

1
2
3
4
[{
  "lat": -35.27801,
  "lng": 149.12958
}]

Copy the following code into the function:
- Replace the GOOGLE_API_KEY with your own Google API key. For information on setting up your own Google API Key, see the Google instructions. Please ensure that the key has the Roads API enabled.
- Replace the MAPBOX_API_KEY with your own Mapbox API key. See Mapbox.com for more information. By default, your key will have access to the required Map Matching API

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
const xhr = require('xhr');
const GOOGLE_API_KEY = "YOUR_GOOGLE_API_KEY"
const MAPBOX_API_KEY = "YOUR_MAPBOX_API_KEY"

function roadsAPI(path) {
  var pathParam = "path="
  for (var i = 0; i < path.length; i++)
  {
    if (i > 0) {pathParam += "%7C"}
    pathParam += path[i].lat
    pathParam += "%2C"
    pathParam += path[i].lng
  }
  const url = "https://roads.googleapis.com/v1/snapToRoads?" + pathParam + "&key=" + GOOGLE_API_KEY + "&interpolate=true";
  const httpOptions = {
    method: 'GET',
  };
  return xhr.fetch(url, httpOptions).then((resp) => JSON.parse(resp.body))
}

function mapboxMapMatchingAPI(path) {
  var pathParam = ""
  for (var i = 0; i < path.length; i++)
  {
    if (i > 0) {pathParam += "%3B"}
    pathParam += path[i].lng
    pathParam += "%2C"
    pathParam += path[i].lat
  }
  const url = "https://api.mapbox.com/matching/v5/mapbox/driving/" + pathParam + "?access_token=" + MAPBOX_API_KEY;
  const httpOptions = {
    method: 'GET',
  };
  return xhr.fetch(url, httpOptions).then((resp) => JSON.parse(resp.body))
}

export default (request) => {
  var provider = request.message.provider;
  var points = request.message.points;
  if (!(points[0].lat && points[0].lng))
  {
    console.log("Message does not have the expected format of {'lat':xxx,'lng':xxx}")
    return request.ok()
  }
  else if (provider == "mapbox" && points.length < 2)
  {
    console.log("Only one point provided")
    return request.ok()
  }
  else
  {
    if (provider == "google")
    {
      return roadsAPI(points)
        .then(roadsResponse => {
          console.log("Google Roads API response: ", roadsResponse);
          if (roadsResponse && roadsResponse.snappedPoints && roadsResponse.snappedPoints[0])
          {
            request.message = roadsResponse;
          }
          return request.ok()
        })    
      }
    else 
    {
      return mapboxMapMatchingAPI(points)
        .then(mapMatchingResponse => {
          console.log("Mapbox Map Matching API response: ", mapMatchingResponse);
          if (mapMatchingResponse && mapMatchingResponse.tracepoints && mapMatchingResponse.tracepoints[0])
          {
            //  Since I have already written the front end to understand the Roads API format, convert the response
            var convertedResponse = {"snappedPoints": []}
            for (var i = 0; i < mapMatchingResponse.tracepoints.length; i++)
            {
              if (mapMatchingResponse.tracepoints[i] != null && mapMatchingResponse.tracepoints[i].location != null)
              {
                var newPoint = {location: {latitude: mapMatchingResponse.tracepoints[i].location[1], longitude: mapMatchingResponse.tracepoints[i].location[0]}}
                convertedResponse.snappedPoints.push(newPoint);
              }
            }
            request.message = convertedResponse;
          }
          return request.ok()
      })    
    }
  }
};

How-to - Smooth Location Data - Image 02

How does it work?

When a PubNub Message is received containing the last known positions of the vehicle, that data is sent to either the Google Snap to Roads API or the Mapbox Map Matching API. The lat / long data in the original message is replaced with the lat / long data returned by the External API, including interpolated points if available, before it is distributed to listening clients. This means that a client displaying the vehicle location will only draw points along the nearest road.

The demo will draw two polylines on the map to clarify what is happening, but this is not required.

Summary

Need help getting started?

We have a How-to and Tutorial that explains how to create a real-time delivery application.
See our Delivery Demo for inspiration.
Check out our delivery solutions page for customer testimonials and use cases.

Questions? Feel free to reach out to the DevRel team at devrel@pubnub.com or contact our Support team for help with any aspect of your PubNub development.

How to Smooth Your Location Data & Snap to the Nearest Road