This is part 1 of a 2-part series on why PubNub Functions are Awesome. The second part features two more reasons.
PubNub Functions are serverless code triggered as events pass through on the PubNub platform. Functions let you write code or leverage existing integrations to transform, re-route, augment, filter, and aggregate data.
PubNub Functions do not require you to spin up a dedicated backend. They are distributed, meaning as your application grows, they will automatically scale so your application can enjoy low latency and 99.999% uptime. Functions support several built-in libraries as well as custom integrations that offer preconfigured connections to third-party services and partners.
How do I Moderate Chat Messages and Censor Profanity?
Assuming you exchange chat messages between clients using PubNub, Functions gives you complete control over how those messages are moderated, filtered, censored, or stored for later moderator analysis.
Although this section will focus on PubNub Functions, we also offer moderation capabilities as part of our BizOps Workspace feature, explained later in this article.
Detecting whether a message contains content that requires moderation is a nontrivial problem. Beyond profanity, your application might need to identify sexual messages, harassment, hateful or threatening speech, violent threats, the potential for self-harm, or more nuanced categorization.
How you approach moderation likely falls into one of 3 broad categorizations, all of which we can support:
Based on a list of banned words, with some kind of fuzzy or RegEx word matching, this is the simplest approach to implement but equally easy for a determined user to circumvent.
A build-your-own approach based on AI or Machine Learning. The rise of Chat GPT and open-source machine learning (ML) models, accessible through REST APIs, make it trivial to detect and classify different kinds of hateful speech in only a few lines of code.
A dedicated content moderation service such as Tisane.ai provides the most powerful and configurable approach, supporting features not available with open source AI or ML approaches, such as different language support or more structured insights.
In all cases, you will likely also want to send a copy of the original message for legal reasons and moderator review.
To diagram these different approaches, your client application sends a message to PubNub, which a PubNub Function intercepts. Moderation based on a banned-word list can be achieved entirely within the Function, introducing only a negligible delay as your message passes through. To take advantage of AI, ML, or a dedicated moderation service requires an external service and we offer many of these services as a custom integration, for example, our Tisane.ai or HuggingFace integrations. Even if we do not offer a custom integration, you can access any external REST API from a PubNub Function through the XHR module, making Functions infinitely flexible.
Set up Functions for Moderating Chat Messages
To create a PubNub Function, first log into the PubNub Portal at admin.pubnub.com (if you don’t have an account, you can sign up for free). Create an application to house your Function by selecting ‘Apps’ from the left-hand menu, then click ‘+ Create app.’ At the time of writing, we are in the process of transitioning to a new version of Functions - please follow the documentation that matches your situation, either Functions v1 or Functions.
Be sure to specify the ‘Before Publish or Fire’ event type since moderation requires changing messages in flight.
Alternatively, many of our custom integrations have a ‘Try it’ button. This button will guide you through creating the correct type of Function on your keyset and prepopulate it with a code template you can modify as needed.
Function Source Code for Moderating Chat Messages
PubNub Function that implements a bad word list
To implement a simple bad word list, you can use our Chat Message Profanity filter integration and customize the list as needed. For example, a Harry Potter chat might choose to filter words using a PubNub Function as follows:
PubNub Function that moderates using Chat GPT
To moderate messages based on Chat GPT, using that service’s moderation endpoint, the code will look as follows:
This code was taken from the Chat Moderation with OpenAI blog, so please see that article for more context and information.
Use BizOps Workspace to Moderate Chat Messages
BizOps Workspace is the collective name for PubNub’s data management capabilities, with new features being added to it frequently. One feature of BizOps Workspace worth highlighting here is the ability to monitor channels and moderate messages. After a message has been identified that requires moderation, a moderator has a number of next steps they can take such as editing / deleting the message or banning / muting the sender. I go into a lot more detail on this topic in my blog about securely moderating messages and users
Next Steps: Moderating Chat Messages & Censor Profanity
You might also find these blogs interesting:
Guide to Using AI Models on PubNub using HuggingFace. HuggingFace has several ML models to identify harmful messages.
Add Moderation to your Unity Game . This has an example of moderation using the Tisane.ai API
We also have the following custom integrations for message moderation: Chat Message Profanity Filter, Lasso, OpenAI, Tisane.ai, Sift
Sign up for a free PubNub account to start exploring Functions today, or check out our Functions documentation.
How do I Analyze the Sentiment of my Users’ Messages?
Understanding the sentiment of what your users are talking about can give insight into your wider application and let you take immediate action. The standard example is a customer support chat app where angry or impatient customers can be escalated or handled differently, but you also get insights into your application based on what your customers are talking about in chat. Developing a game with in-app chat where players are continually negative? There are probably ways you can tweak your game to improve their experience. Maintaining a live event chat where participants from one geographical region are overly positive? Understanding why can help you spread that positivity worldwide.
PubNub can provide additional value. Unlike other solutions, which allow you to analyze sentiment after the fact, PubNub allows you to monitor sentiment in real time, as chats are happening, and even potentially take action based on the user sentiment—to continue the gaming example, it would be technically possible to make the game easier for players who are expressing frustration in chat.
To analyze message sentiment, you should configure a PubNub Function to query an external service through a REST API. Don’t worry, we have several custom integrations that offer this functionality, and I strongly recommend you read through our blog about Sentiment Analysis with PubNub and HuggingFace, which provides open source machine learning models for this purpose.
To diagram the overall approach, your client application sends a message to PubNub, which a PubNub Function detects but does not block. The message is sent to your chosen external service, and the sentiment is returned.
With an understanding of whether the message is positive or negative, the next step will depend on your application. Most customers choose to store that message for offline analysis using a service such as Amazon S3, Amazon Kinesis, or Apache Kafka - our Events & Actions service is designed for exactly this use case and can stream to all of these products at any scale. Alternatively (or in addition), Events & Actions can also trigger a custom webhook, enabling you to take immediate and custom action on the message and sentiment.
Set up Functions for Analysing Message Sentiment
To create a PubNub Function, first log into the PubNub Portal at admin.pubnub.com (if you don’t have an account, you can sign up for free). Create an application to house your Function by selecting ‘Apps’ from the left-hand menu, then click ‘+ Create app.’ At the time of writing, we are in the process of transitioning to a new version of Functions - please follow the documentation that matches your situation, either Functions v1 or Functions.
Be sure to specify the ‘After Publish or Fire’ event type since you do not need to make changes to the message in flight.
Alternatively, many of our custom integrations have a ‘Try it’ button. This button will guide you through creating the correct type of Function on your keyset and prepopulate it with a code template you can modify as needed.
Function Source Code for Analysing Message Sentiment
PubNub Function that uses HuggingFace for Sentiment Analysis
The following code is taken from the Sentiment Analysis with PubNub and HuggingFace blog, it shows how to use the distilbert/distilbert-base-uncased-finetuned-sst-2-english model to analyse the sentiment of provided text, positive or negative.
PubNub Function that redirects a copy of messages to PubNub Events and Actions
As mentioned earlier, PubNub’s Events & Actions service can be used to store a copy of your message in Amazon S3, Amazon Kinesis, Apache Kafka, and more. The easiest way to achieve this is to republish the message to a dedicated channel
Next Steps: Analysing Message Sentiment
You might also find these blogs interesting:
Guide to running AI models on PubNub using HuggingFace. You are not limited to the models discussed in the first link, but you can use any of HuggingFace’s growing library.
OpenAI GPT API Integration with Functions and Build a Chat Bot with PubNub and OpenAI. These are not specifically about sentiment analysis but show how to ask ChatGPT a question from a Function, which could easily be “What is the sentiment of this message?”
We also have the following related custom integrations: HuggingFace, OpenAI, Tisane.ai (focused on message moderation, but Tisane also offer sentiment analysis)
Sign up for a free PubNub account to start exploring Functions today, or check out our Functions documentation.
And Finally: Recent Updates to PubNub Functions
PubNub has recently implemented a slew of enhancements to our Functions to make them easier to use, with additional capabilities:
A Functions API makes it easier to deploy & manage new and existing Functions
We have enhanced our in-browser editor to allow on-the-fly editing
We now retain a history of previous deployments, so you can easily roll back changes if something is not working.
We have added new built-in libraries to support UUID, JWT, and JSONPath Plus.
Sign up for a free PubNub account to start exploring Functions today, or check out our Functions documentation for more information.
