Your Guide To Messaging APIs Web 3.0 Internet Communication

Given all the promise of the Internet, we initially experienced it only as an information dump with millions of websites containing billions of web pages hyperlinked to one another. With Web 2.0 we envisaging a semantic web powered by dynamic web applications where information was stored and retrieved structurally. Now with the emergence of Web 3.0, we are witnessing yet another transformation of the Internet in the form of cognitive apps and human-machine interaction.

As applications become more engaging and interactive, so does the need for messaging APIs and a variety of techniques such as long polling, that enable the underlying communication mechanisms. As a developer, how do you adapt to these rapid strides in the evolution around internet and communication technologies?

This blog post will take you through the journey of exploring the various web messaging frameworks and technologies, both legacy and emerging, that will power the apps of today and tomorrow. We will also look at how the PubNub Data Stream Network leverages these technologies to enable developers like you to build the most reliable and scalable apps over the Internet.

1. HTTP

HTTP (Hyper Text Transfer Protocol) was the first protocol that defined the specification for Internet communication during the Web 1.0 era. This was the way to transfer hypertext documents over the Internet, and request for external media resources linked to the documents for rendering in the browser.

HTTP follows a simple request-response pattern, wherein the client requests the hypertext document, addressed by an URL from the server and the server obliges by sending the document as a response to the client’s request.

Over the years, HTTP has been enhanced to bring in additional features to support newer applications requirements and security features (HTTPS). If you consider yourself a protocol nerd then you can check out the HTTP1.1 specification in IETF. The newer version, HTTP2 is available here.

2. REST (REpresentational State Transfer)

REST is a methodology for creating HTTP/HTTPS enabled applications that adhere to a structured means of communication over the Internet. In a broader sense, REST mandates that the communication should happen through well-defined web services that define their own APIs (Application Programming Interface).

REST is a powerful concept and its emergence has led to the dominance of “as-a-Service” models of application delivery over the Internet vis-a-vis the traditional methods of installing applications on your local computer. If you use Google docs or spreadsheets then you are already using REST APIs to store and edit documents in the cloud. Or if are an outer space aficionado, then you might be interested to explore the NASA open APIs which give you access to NASA images of space and other planets.

Understanding REST

To understand REST better, let's take a step back in the history. Back in the days of Web 1.0 when websites were just a bunch of .html files, each webpage had an URL that was formed by combining the domain name followed by the file path of the HTML file. Users had to type in or click that URL to retrieve the web page. These web pages were the endpoints and the URLs were the APIs for website visitors to interact with the site.

As web applications became more engaging, the web servers evolved to define URLs for logical endpoints instead of the actual HTML files. These logical endpoints allowed client applications to engage with Web servers in a modular way, through a bunch of services.

Let’s consider a simple Todo app. The operations for this app can be structured as different services that operate on ToDo list, Todo items, and users. With REST, the server functions of the app should be arranged in a structured and modular way.

While the underlying protocol driven by HTTP/HTTPS remains the same, the REST APIs now becomes an embodiment of the end user application and its entities, namely the users, todo lists, and todo Items. This way REST provides a framework to think and define the application services in an abstract way. This leads to a more concrete and sensible implementation of the application functionality.

If you are curious, here is a light read on a brief history of the evolution of REST.

GraphQL: The Next Evolution of REST

While REST provides a standardized structure to APIs, it does have some limitation. Consider the REST APIs for the ToDo app above. All the constituent entities of the app, namely the todo list, the todo item, and the users are linked in some way. That is, the users create todo lists which contain a bunch of to-do items. However, the REST APIs are designed to retrieve these entities separately. If you have to display all the todo lists of a user, then you have to make multiple API calls to retrieve the user and then retrieve the user’s todo lists.

GraphQL is an open standard that leverages the semantic association between the entities to retrieve multiple resources in a single query. It defines a query language and a runtime that can sit in the front of an API endpoint such that you can query and retrieve all the todo lists of a user in one shot.

3. WebSockets

WebSockets bring a spark to web applications by enabling real-time, bi-directional internet communication between applications. Again, back to the Web 1.0 days, you can probably recall that the world wide web was all about fetching web pages from servers. That's the reason why HTTP was designed as a request/response protocol.

With Web 2.0, we have a whole new way of interacting with websites, that have now turned into single page web applications. These applications are real-time, user-centric and highly engaging. Take the case of Facebook for example. HTTP can still be used for driving such web applications, however, it is inefficient for such real-time interactions.

WebSocket allows you to have a persistent bi-directional communication, instead of the frequent request/response message exchanges.

Metaphorically, we could think of this as an army with a chain of command. Let’s make the web server the General, and all the browsers as the soldiers on the ground waiting for orders. Using HTTP/REST, if every soldier has to ask the General if there are any new orders, it burdens the General tremendously, particularly when there isn’t anything new. That means the General is saying “no, nothing new” most of the time.

If your servers are the General, their resources are being wasted mostly saying that there is nothing new.

Using the same metaphor, WebSockets being connected are like each soldier having a radio, and when the General has a new order, he can broadcast that order into the radio. You can learn more about the technical details of WebSockets from the official W3C specification.

There are many implementations of WebSockets across programming languages. One of the popular libraries is Socket.IO. However, to build applications at scale and with a high degree of reliability, it is not just enough to have bi-directional communication between two ends.

At PubNub, we embrace scalability and real-time performance as a way of life. Hence we have wrapped this concept of WebSocket around our own data streaming technology that empowers you to scale out your apps without spending extra effort in development.

4. Publish/Subscribe

Publish/Subscribe is more of a paradigm or a communication pattern. With the advent of Web 3.0, we have entered into a new realm of collaborative applications that are no longer merely human-centric. We are now talking about human-to-machine and machine-to-machine interfaces because we are increasingly interacting with cyber-physical systems that are smart and have cognitive capabilities.

The scale of such interaction always fans out as a multi-party communication. That’s where we need something more than a mere request-response or bidirectional mechanism. The Publish/Subscribe approach provides a way out. Imagine, you have to send commands to multiple devices at once or a device should update its status to a web dashboard as well multiple app screens. All of this can be achieved seamlessly with Publish/Subscribe.

You can think of Publish/Subscribe as a bi-directional, multi, request/response system. Confused? Let take an analogy.

Think of your grandfather’s telephone. You could dial another person’s phone number and talk to him. But how difficult was it to bring in more people into the conversation? Now, think about Google Hangouts, or WhatsApp group. In essence, the communication is now driven by a virtual entity acting as a Hub, with end users acting as publishers or subscribers of messages.

A web messaging API that supports Publish/Subscribe mechanism, allows multi-party collaboration. It is a flexible way of communication and there already exist several standardized implementations of this. Take the case of MQTT, which is the preferred protocol used for IoT. In a typical IoT application, data coming from a sensor needs to be forked out to multiple places.

Publish/Subscribe mechanism provides the basis for achieving all this without the overhead of multiple request/response interactions. In this case, all the messages are orchestrated by a MQTT broker, which is the equivalent of the hub in the previous example.