Auto Moderation

Beta feature

Auto Moderation is in beta and available upon request. Contact PubNub Support or Sales.

Related moderation approaches

Auto Moderation is one of several moderation solutions offered by PubNub. For a complete overview of all moderation options and help choosing the right approach, see the Moderation Overview. You can also combine Auto Moderation with Channel Monitor for manual oversight, Chat SDK moderation features for user-initiated reports, or custom Functions for advanced logic.

The Auto Moderation module works with PubNub Channel Monitor. It automates message moderation in selected conversations.

Instead of monitoring messages one by one, use Auto Moderation Functions.

Auto Moderation provides a policy editor that creates Auto Moderation Functions to check every message on your selected channels.

Auto Moderation Functions can:

• Block or report messages flagged as promotional spam or scam (Spam detection Function, English only).
• Block or report messages that violate toxicity thresholds (Profanity filter Function).
• Mask words you define as forbidden (Word masking Function).

Functions dependency and limits

Auto Moderation relies on Functions v2 and is available to Functions v2 users. The Free tier covers a limited number of executions per month. If you exceed the limit, upgrade to maintain continuity.

Create configuration

To use Auto Moderation, start by creating a moderation policy:

1. Log in to the Admin Portal.

2. In the left sidebar, go to BizOps Workspace.

3. Select Auto Moderation.

   If it's the first policy you create, you'll see an empty page when you open the Auto Moderation module. That empty state still uses configuration; later screens use policy.

4. Click Create configuration or Create policy, depending on which label the button shows, to open the policy editor.

   

Auto Moderation with an existing Before Publish Function

If a Before Publish or Fire Function already runs on a channel, you cannot activate another of the same type. Choose Generate code for Functions when activating to add moderation to the existing Function.

1. Once the policy editor opens, enter a policy name and a description at the top of the page.

   

2. In the Select features sidebar, enable the moderation features you want to apply and configure each one.

Select moderation behavior

In the Select features sidebar, choose which moderation features you want to apply to messages on your app's channels.

1. Click a feature in the sidebar to enable it. This opens its configuration panel in the main area.
2. Configure the feature options in the panel.
3. To disable a feature, use the toggle next to its name in the configuration panel.

Spam detection

Choose how AI treats English-language messages in selected channels when it flags promotional spam or scam content.

Flagged messages can be blocked before publish or reported to a moderator for action.

To proceed, you must agree to the PubNub beta license terms of service.

Handle blocked messages

All three functions (Spam detection, Profanity filter, and Word masking) are Before Publish or Fire. They check messages before publish. If a message is blocked, it does not appear in Channel Monitor and the client publish returns an error. Handle this error in your client UI to inform users about applied moderation.

Profanity filter

Choose how the Profanity filter treats messages in selected channels when it detects a violation.

Flagged messages can be blocked before publish or reported to a moderator for action.

To proceed, you must agree to the PubNub beta license terms of service.

Languages

Select the languages the filter should analyze. Supported languages are English, Spanish, French, German, Portuguese, Italian, Dutch, Polish, Japanese, Korean, Chinese, Arabic, Hindi, Russian, and Turkish.

Thresholds

Set the detection sensitivity for each violation category using the OFF/LOW/HIGH slider. The available categories are:

Category	Description
Toxicity	Hostile, abusive, inflammatory, or harmful language.
Harassment	Targeted harassment, insults, or abusive behavior toward individuals.
Hate speech	Slurs, discrimination, or attacks based on identity.
Sexual content	Explicit adult content, sexual acts, or pornography.
Spam	Unwanted, repetitive, promotional, or deceptive content.
Profanity	Offensive, vulgar, or obscene language and swearing.
Self-harm	Content involving self-injury, suicide, or encouragement of self-harm.
Doxxing	Sharing, requesting, or threatening to expose private personal information.
Scam	Fraudulent schemes, deceptive practices, or attempts to steal money or information.
Impersonation	Pretending to be another person, organization, or authority.
Raiding	Coordinated efforts to harass, disrupt, brigade, or overwhelm a target community.
Political abuse	Political abuse, extremist rhetoric, radicalization, or propaganda targeting political views.

Word masking

Define which words to mask in messages on selected channels. When a sensitive word is detected, each letter is masked with an asterisk (*).

Word masking limitations

Word masking works on initial publish only. Later message actions, such as editing, can change masked words.

Select Mask word as the action to take when a sensitive word is detected, then define a word list (or choose an existing one) for Auto Moderation to check.

Creating a word list

Give your word list a name, then add entries to the Restricted words (patterns) field, one entry per line. You can add plain words or regular expressions.

You can also populate the list by importing a .txt or .csv file (max 32KB). Drag and drop it into the import panel, or click Select file to browse. Use the Generate button to populate the list with a built-in set of regex patterns for common profanity, including leet-speak variations (for example, @ for a, $ for s, 1 for i).

Shared word lists

Auto Moderation and Channel Monitor share word lists. This means you can reuse a word list previously created in Channel Monitor (either active or inactive) and apply it in the Auto Moderation policy. You can also do the other way round, but remember to activate the word list in Channel Monitor for it to work.

Configure message path

To moderate messages, PubNub needs to know which part of your JSON payload contains the text. Click Configure message path in the top-right corner of the policy editor to set this.

By default, Auto Moderation checks the $.message.text field. Update this path if your payload is structured differently.

This setting applies to all three moderation features (Spam detection, Profanity filter, and Word masking).

Different message content paths

Auto Moderation and Channel Monitor do not share message path configuration. You can set different paths in each module.

Test policy

You can test your policy at any time while configuring it by clicking Test policy in the top-right corner of the policy editor.

This opens the Test your policy panel, which shows:

• Your features — the moderation features currently enabled in the policy.
• Select message for testing — three pre-built test messages to click, or a free-text field where you can enter your own message and click Test.
• Results — displayed inline next to each tested message, showing the action taken and which feature triggered it. The final result is determined by the most severe action triggered across all enabled features.
• Testing details — the moderation sequence (for example, Spam detection → Profanity filter → Word masking) and a per-feature result breakdown.

Mocked publish call

This testing option only lets you locally check if the Auto Moderation mechanism works, but it does not publish an actual message on any of the channels within the selected keyset.

Save or activate policy

When you're done configuring and testing, use the buttons at the bottom of the policy editor:

• Create policy — saves the policy without activating it and takes you to the Moderation policies overview page. From the overview page, you can activate any saved policy at any time by clicking Activate policy next to it.

• Create & activate policy — saves the policy and immediately launches the activation wizard.

Once activated, each policy displays a label in the Activations column indicating its deployment type: PubNub managed or Code for Functions.

Activate policy

Whether you click Create & activate policy or Activate policy from the overview, the same activation wizard opens. Start by choosing a deployment method.

Deploy as PubNub managed

Choose this option if you want PubNub to automatically deploy and manage the moderation logic using PubNub Functions. You can toggle it on/off instantly.

You can deploy the same policy to many channels across multiple keysets, but you add them one keyset at a time.

1. Under Add channels from keysets, click Select keyset and choose one keyset. You can search by app, keyset, or subkey, and filter the list by app. Channels are listed only after you select a keyset. After you choose a keyset, the Select keyset button is replaced by the name of the selected keyset.

   

2. Select one or more channels from that keyset. The Selected for deployment panel shows the app, keyset, and channels you have chosen.

   

   To cover a large number of channels without exceeding the deployment limit (30), add a Channel pattern. Denoted channel patterns are treated as one deployment. Click + Add to define a pattern.

3. To include channels from another keyset, click the button that now shows the current keyset name, choose the next keyset, and select its channels. Repeat until every keyset and channel you need is listed in Selected for deployment.

4. Click Deploy policy. The policy is now active.

After deployment, the policy detail page opens. PubNub managed and Code for Functions are listed in the left sidebar.

When PubNub managed is selected, Channel deployments lists each keyset, app, channel count, and status. Search keysets, click Manage channels to change the deployment, and use the row actions to restart or stop it. The sidebar also shows which features are enabled and the Message path used for moderation.

Generate code for Functions

Choose this option if you already have a Before Publish or Fire Function running on a channel and cannot deploy another of the same type, or if you want to combine moderation with your own custom logic. PubNub generates the moderation code for you to copy and integrate into your existing Function.

1. Click Select keysets and choose the keyset for which you want to generate the code.

   

2. Click Generate code for Functions.

3. Copy the generated code from the panel.

   

   The generated code calls ugc.moderateMessage with your policy's configId and handles blocking, reporting, and message transforms (such as word masking):

   1const ugc = require('ugc');
   2
   
   3export default async (request) => {
   4    try {
   5        const moderationResult = await ugc.moderateMessage({
   6            configId: '<YOUR_CONFIG_ID>',
   7            message: request.message,
   8            userId: request.params.uuid,
   9            channel: request.channels[0],
   10        });
   11
   
   12        if (moderationResult.flagged && moderationResult.actions.includes('block')) {
   13            return request.abort('Moderated');
   14        }
   15
   
   16        if (moderationResult.flagged && moderationResult.actions.includes('report')) {
   17            // reporting is handled externally, but you can use this block for additional actions
   18        }
   19
   
   20        if (moderationResult.transform) {
   21            // Transform message (e.g. for word masking)
   22            if (moderationResult.transform.message) {
   23                request.message = moderationResult.transform.message;
   24            }
   25
   
   26            // Transform meta (e.g. to include report metadata)
   27            if (moderationResult.transform.meta) {
   28                const existingMeta = JSON.parse(request.params.meta || '{}');
   29                const transformedMeta = { ...existingMeta, ...moderationResult.transform.meta };
   30                request.params.meta = JSON.stringify(transformedMeta);
   31            }
   32        }
   33
   
   34        return request.ok();
   35    } catch(err) {
   36        console.log('Moderation Error:', err);
   37        return request.ok();
   38    }
   39};

   show all 39 lines

4. Go to the Functions module in the Admin Portal and select the active Before Publish or Fire Function to which you want to add the moderation code.

   

5. Add the moderation code to the Function's body. The way you integrate it differs depending on your existing Function logic.

   For example, if your Function already detects @mentions, here is how the original and combined versions compare:

   Original Function:

   1// Before Publish Function entry point; runs before the message is delivered.
   2// You can modify `request.message` or `request.params.meta` here.
   3export default (request) => {
   4    // Regex to detect @mentions like @alice or @user_123 (case-insensitive)
   5    const mentionPattern = /\B@[a-z0-9_-]+/gi;
   6
   
   7    const text =
   8        typeof request.message === 'string'
   9        ? request.message
   10        : (request.message && request.message.text) || '';
   11
   
   12    const mentions = Array.from(new Set((text.match(mentionPattern) || []).map(m => m.toLowerCase())));
   13
   
   14    let meta = {};
   15    try {
   16        meta = typeof request.params.meta === 'string' ? JSON.parse(request.params.meta) : (request.params.meta || {});
   17    } catch (_) {
   18        meta = {};
   19    }
   20
   
   21    meta.mentions = mentions;
   22    meta.hasMentions = mentions.length > 0;
   23
   
   24    request.params.meta = JSON.stringify(meta);
   25
   
   26    return request.ok();
   27};

   show all 27 lines

   Combined Function with moderation:

   1const ugc = require('ugc');
   2
   
   3export default async (request) => {
   4    // Parse existing meta (string → object)
   5    let meta = {};
   6    try {
   7        meta = typeof request.params.meta === 'string'
   8        ? JSON.parse(request.params.meta)
   9        : (request.params.meta || {});
   10    } catch (_) {
   11        meta = {};
   12    }
   13
   
   14    // Run moderation (block or apply transforms)
   15    try {
   16        const res = await ugc.moderateMessage({
   17        configId: '<YOUR_MODERATION_CONFIG_ID>',
   18        message: request.message,
   19        userId: request.params.uuid,
   20        channel: request.channels[0],
   21        meta,
   22        });
   23
   
   24        if (res.flagged && res.actions.includes('block')) {
   25            return request.abort('Moderated');
   26        }
   27
   
   28        if (res.flagged && res.actions.includes('report')) {
   29            // reporting is handled externally, but you can use this block for additional actions
   30        }
   31
   
   32        if (res.transform?.message) request.message = res.transform.message;
   33        if (res.transform?.meta) meta = { ...meta, ...res.transform.meta };
   34    } catch (err) {
   35        console.log('Moderation Error:', err);
   36    }
   37
   
   38    // Extract @mentions from the (possibly transformed) message
   39    const mentionPattern = /\B@[a-z0-9_-]+/gi;
   40    const text = typeof request.message === 'string'
   41        ? request.message
   42        : (request.message && request.message.text) || '';
   43    const mentions = Array.from(
   44        new Set((text.match(mentionPattern) || []).map(m => m.toLowerCase()))
   45    );
   46
   
   47    meta.mentions = mentions;
   48    meta.hasMentions = mentions.length > 0;
   49
   
   50    request.params.meta = JSON.stringify(meta);
   51
   
   52    return request.ok();
   53};

   show all 53 lines

   This Function moderates via your configId (blocks or applies transforms), extracts @mentions, updates request.params.meta, then publishes the message unless blocked.

6. Save the changes as a new revision and redeploy the Function.

Activated policy

Once activated, you'll see the policy on the overview page with the dashboard showing statistics for moderated messages across all active and inactive policies.

Only an activated policy deploys an Auto Moderation Function of type BizOps (as seen in the Functions module) which runs Auto Moderation on the selected channels within the keyset. Each activated policy creates one Package with one Deployment and a number of Functions equal to the number of channels selected during activation.

Every word from the word list included in a message published on the selected channels within the scope of the given keyset will now be masked with asterisks.

Every spam message will be either blocked before it is published on a channel or reported to an admin for further actions.

Every message that violates the Profanity filter thresholds will be either blocked before it is published or reported to an admin for further actions.

Provide feedback on auto moderated messages

You can help us improve the mechanism behind auto moderation by reviewing messages that were either blocked or reported. Read the Review auto moderated messages section for details.

The reported message will still be visible on the message list on the channel but marked with the Reported label.

Moderation during service disruptions

During Function-related disruptions (for example, service communication errors), Auto Moderation publishes all messages instead of moderating.

Edit or delete a policy

To edit or delete a policy, click the ... menu next to it on the Moderation policies overview page and select Edit or Delete.

Editing a policy opens the same policy editor. You can only modify a policy from the Auto Moderation section in BizOps, not from the Functions module in the Admin Portal.

You can only have one policy running on a given channel. Activating an edited policy will automatically restart the previously running one. However, if you want to activate a new policy on the same channel, you will have to manually stop the previously running one.

Policy and Function conflicts

Auto Moderation validates channels when you create or edit a policy to make sure there is only one policy and the corresponding Before Publish Function running on a given channel.

When creating or editing a policy, you can encounter the following conflicts:

• Internal conflict (within the Auto Moderation space in the Admin Portal) — there is already an active Auto Moderation policy running on the selected channel.
• External conflict (related to the Functions space in the Admin Portal) — there is already an active Before Publish Function running on the selected channel.

When conflicts occur, the UI shows a banner and per-row badges with instructions on actions you can perform to resolve these conflicts.