Triggering Webhooks with XHR (Advanced Functions Techniques)

Tutorial Overview

The “Search Bot” is an excellent demonstration of the Functions XHR module, an efficient HTTP request library that integrates directly with Functions handlers. In this tutorial, we dive into an example of intercepting data stream message events to implement a “Search Bot” using custom serverless event handling in a real-time Angular2 web application

We'll extend our Angular2 messaging app with a “Search Bot” handler to process message events and retrieve search results via a Web Service API. Although the example is small (using fixed commands and limited data), it could easily be extended to call additional web services for message persistence, natural language detection, sentiment analysis, bridging to external message systems and more.

We'll create a PubNub BLOCK of JavaScript that runs entirely in the network and implements the “Search Bot” feature to obtain relevant search results so that the web client UI code can stay simple and just display the data.

As we prepare to explore our sample Angular2 web application with a “Search Bot” feature, let’s check out the underlying PubNub API features that make it all possible.

PubNub makes implementing real-time systems much easier by integrating enterprise-grade security, presence, storage and other advanced features into a huge (and growing!) number of client toolkits. Check out the PubNub SDK page to see them all.

The core of PubNub is Data Streams – real-time message channels that allow an unbounded numbers of devices to publish and subscribe to messages. The Functions feature allows developers to attach custom code to those message events in one of two ways, either as a “before publish or fire” event which allows modification of message attributes, or as an “after publish or fire” event which takes place after the message has been published to the message channel. (There is also an “after presence” event for implementing Presence-related functionality). Check out our Event Handler Types post for more on those.

Using the XHR module, we create a BLOCK that intercepts the “before publish or fire” events in a message channel to implement the “Search Bot” functionality. The big question is, how do we obtain search results in the context of data streams? This is a prime use case for the Functions XHR module, which offers scalable HTTP request functionality for real-time applications. Here’s how our BLOCK works:

• We examine each message for a “text” attribute.
• Based on the text value, we perform a search GET request from a third-party API.
• We decorate the message with a “search_results” attribute including the result JSON.

The XHR module is extremely configurable, so it’s easy to create functionality to access just about any API you can think of. Here are a few of the considerations where XHR works best:

• The API speaks HTTP (ideally HTTPS for security).
• The API is publicly accessible (otherwise Functions can’t access it).
• The API accepts/produces JSON/form-encoded/plain text (you can do XML, it’s just slightly harder to integrate in Functions JavaScript).
• The API responds in a reasonable time, say 1-5s (the max XHR timeout is 10s).
• The API uses a reasonably easy auth scheme (such as API key, Basic Auth, etc.).

If your API follows those rough parameters, there are a ton of places where XHR can be configured to integrate:

• XHR supports HTTP methods GET, POST, PUT, OPTIONS, DELETE, PATCH.
• XHR supports getting/setting Request and Response headers.
• Codecs are available for query string, base64, and basic auth encoding.
• XHR supports configurable timeouts up to 10s for APIs that need a bit more time.

You’ll want to grab the 35 lines of BLOCK JavaScript and save them to a file, say, pubnub_xhr_block.js. It’s available as a Gist on GitHub for your convenience.

First up, we create a function to handle incoming messages. We include a dependency on the ‘xhr' module for HTTP/HTTPS API access, ‘codec/query_string' for query string encoding. Other common codecs to require are ‘codec/auth' for HTTP Basic Auth encoding, and ‘codec/base64' for base64 encoding.

export default (request) => {
    const xhr    = require('xhr');
    const query  = require('codec/query_string'); // for HTTP URL encoding
    const auth   = require('codec/auth');         // for HTTP Basic Auth
    const base64 = require('codec/base64');       // for other custom schemes

Next, we configure the API URL, query string options, and HTTP options.

const apiUrl = "https://api.duckduckgo.com/?";
var queryString = query.stringify({
    q : request.message.text,
    format : 'json'
});
var httpOptions = {
    method  : 'GET', // allowed GET/POST/PUT/OPTIONS/DELETE/PATCH
    timeout : 5000,  // max 1000 (10s)
    headers : {
       'Accept' : "application/json"
       // 'Content-Type' : ...
       // 'Authorization' : ...
       // 'If-Modified-Since' : ...
    },
    body    : null,  // form body, JSON string, XML ...
};