Kafka Bridge

PubNub's Kafka Bridge integrates the real-time messaging capabilities of PubNub and the streaming data platform of Apache Kafka.

It allows for receiving Kafka events in PubNub and streaming events from PubNub to Kafka, maximizing the benefits of both platforms for developing highly responsive, scalable, and real-time applications.

Receive events from Kafka into PubNub

PubNub Kafka Sink Connector enables streaming events from an Apache Kafka cluster into PubNub to deliver Kafka events to the broad network of PubNub with minimal latency.

It's handy for applications requiring immediate action or notification based on the data processed in Kafka, such as messaging, live updates, or IoT signaling.

Setting up PubNub Kafka Sink Connector involves:

1. Compiling the connector from the source.
2. Configuring the connector with PubNub publish and subscribe keys, Kafka topic(s) details, and connection information.
3. Deploying the connector to Kafka Connect ensures it effectively bridges Kafka events with the specified PubNub channels.

Stream events from PubNub to Kafka

Conversely, the Kafka Action feature within PubNub's Events & Actions framework lets you send data from PubNub channels to an external Apache Kafka cluster. This action is agnostic of how or where the Kafka instance is hosted, ensuring flexibility in deployment scenarios.

This capability is handy for applications that require real-time processing of messages or events in Kafka for analytics, storage, or further processing in big data pipelines.

To configure this action, you need to:

1. Have your Kafka environment set up with information like topic, key, broker URL, and authentication details.
2. Use the PubNub Admin Portal to configure the Kafka action, specifying details such as the topic and key, authentication mechanism, username, password, and broker URLs.

You can create a Kafka action regardless of whether you have an existing Kafka cluster or are setting up a new one. Also, various hosting configurations are supported, including self-hosted Kafka, Amazon MSK clusters via Terraform, and Confluent Cloud.