Auto Moderation

Beta feature

This feature is in beta and available upon request. Test in your environment before production use. To get access to Auto Moderation, contact PubNub Support or Sales.

The Auto Moderation module works with PubNub Channel Monitor. It automates message moderation in selected conversations.

Instead of monitoring messages one by one, use Auto Moderation Functions.

Auto Moderation provides a setup wizard that creates Auto Moderation Functions to check every message on your selected channels.

Auto Moderation Functions can:

• Block messages or publish and report them to an admin (Spam detection (AI) Function powered by the PubNub‑hosted AI model).
• Mask words you define as forbidden (Word masking Function).

Functions dependency and limits

Auto Moderation relies on Functions v2 and is available to Functions v2 users. The Free tier covers a limited number of executions per month. If you exceed the limit, upgrade to maintain continuity.

Create configuration

To use Auto Moderation, start by defining the Function's configuration:

1. Log in to the Admin Portal.

2. In the left sidebar, go to BizOps Workspace.

3. Select Auto Moderation.

   If it's the first configuration you define, you'll see the empty page when you open the Auto Moderation module.

4. Click the Create configuration button to open the setup wizard.

Auto Moderation with an existing Before Publish Function

If a Before Publish or Fire Function already runs on a channel, you cannot activate another of the same type. Choose Generate Code for Functions to add moderation to the existing Function. See Auto Moderation with an existing Before Publish Function for steps.

1. Once the wizard opens, enter a configuration name and description.

After you create your configuration, the wizard guides you to select moderation behaviors.

Select moderation behavior

In the next step, you can decide what type(s) of Auto Moderation Function(s) you want to apply to messages on your app's channels.

Spam detection (AI)

Choose how AI treats messages in selected channels when it flags spam.

Spam can be blocked before publish or reported to a moderator for action.

To proceed, you must agree to the PubNub beta license agreement.

Handle blocked messages

Both functions (Spam detection (AI) and Word masking) are Before Publish or Fire. They check messages before publish. If you block a spam message, it does not appear in Channel Monitor and the client publish returns an error. Handle this error in your client UI to inform users about applied moderation.

Message content path

By default, Auto Moderation Functions look for the message content to moderate under the text key in each message payload JSON structure. However, if your message structure has a different format, you can point Auto Moderation to the correct path in your messages:

Different message content paths

Auto Moderation and Channel Monitor do not share message path configuration. You can set different paths in each module.

Word masking

Define which words to mask in messages on selected channels.

Word masking limitations

Word masking works on initial publish only. Later message actions, such as editing, can change masked words.

Define a word list (or choose an existing one) for Auto Moderation to check.

If you choose this Auto Moderation Function type, you must define a list of words (or choose an existing one) you want the Auto Moderation Function to be on the lookout for.

When such a word is detected in a message, each letter in this word will be masked with an asterisk (*) character.

Shared word lists

Auto Moderation and Channel Monitor share word lists. This means you can reuse a word list previously created in Channel Monitor (either active or inactive) and apply it in the Auto Moderation configuration. You can also do the other way round, but remember to activate the word list in Channel Monitor for it to work.

Test configuration

In the next step you can check whether your configuration will work when activated.

To do that:

1. Click Test moderation behavior to see moderation running on the three test messages or enter and send a sample text message (JSON object, array, or literal) that either includes a word from the previously defined word list or should be considered spam.

   

   Mocked publish call

   This testing option only lets you locally check if the Auto Moderation mechanism works, but it does not publish an actual message on any of the channels within the selected keyset.

2. Check in the console if Auto Moderation worked as expected. Auto moderated messages should be assigned proper labels (Message blocked or Message reported and Words masked).

   

   You can also test your saved/activated configuration from the general Auto Moderation view once it's activated.

Select keyset

On the next configuration wizard page:

1. Choose a keyset on which you want the Auto Moderation Function(s) to run.
2. Press Next to proceed.

Select channels

Select channels on which you want Auto Moderation Functions to run for all messages and detect spam or mask forbidden words.

If you want to choose a large number of channels and not exceed the deployment limit (50), specify channels using wildcards.

If you notice the message below when trying to select a channel on which a Function should be deployed, this means you already have a Before Publish or Fire Function running on this channel and cannot deploy another.

In such a case, select the Generate code for Functions option to follow the alternative path in which you create moderation configuration but add it to the existing Function on your own.

Save and activate configuration

When you're done configuring and testing Auto Moderation, you can save the settings (Save) or activate the configuration straightaway (Save and activate).

You'll see the active Auto Moderation Function and the dashboard showing statistics for the moderated messages in all active and inactive Functions.

Only an activated configuration deploys an Auto Moderation Function of type BizOps (as seen in the Functions module) which runs Auto Moderation on the selected channels within the keyset. Each activated configuration creates one Package with one Deployment and a number of Functions equal to the number of channels selected during the configuration setup.

Every word from the word list included in a message published on the selected channels within the scope of the given keyset will now be masked with asterisks.

Every spam message will be either blocked before it is published on a channel or reported to an admin for further actions.

Provide feedback on auto moderated messages

You can help us improve the mechanism behind auto moderation by reviewing messages that were either blocked or reported. Read the Review auto moderated messages section for details.

The reported message will still be visible on the message list on the channel but marked with the Reported label.

Moderation during service disruptions

During Function‑related disruptions (for example, service communication errors), Auto Moderation publishes all messages instead of moderating.

Edit configuration

An activated Auto Moderation can still be modified, but only from the Auto Moderation section in BizOps, not from the Functions module in the Admin Portal.

You can only have one configuration running on a given channel.

Activating an edited configuration will automatically restart the previously running one. However, if you want to activate a new configuration on the same channel, you will have to manually stop the previously running one.

Auto Moderation with an existing Before Publish Function

Auto Moderation supports an edge case in which you already have a Before Publish or Fire Function running on a given channel and would like to enable moderation on it.

Follow these steps:

1. Select Generate code for Functions in the initial Create configuration step.

2. Select moderation behavior.

3. Test configuration.

4. Select a keyset and save configuration.

5. Copy the moderation code from the pop-up modal.

   

   Copied code with explanation:

   1// Imports the UGC moderation module. For more details, read the docs: https://www.pubnub.com/docs/serverless/functions/functions-apis/ugc-module
   2const ugc = require('ugc');
   3
   
   4// Entry point for the Function; async so we can await the moderation call
   5export default async (request) => {
   6    try {
   7        // Calls the moderation service with context about this publish
   8        const moderationResult = await ugc.moderateMessage({
   9            // ID of your Auto Moderation configuration
   10            configId: '0346b6d9-2183-4de3-ae29-1d98cb5e9784',
   11            // Full message payload the client sent
   12            message: request.message,
   13            // Publisher's UUID from the request
   14            userId: request.params.uuid,
   15            // Target channel for this publish
   16            channel: request.channels[0],
   17        });
   18
   
   19        // If moderation configuration requires blocking, it stops the publish
   20        if (moderationResult.flagged && moderationResult.actions.includes('block')) {
   21            return request.abort('Flagged'); // Prevents the message from being published
   22        }
   23
   
   24        // Applies any suggested transformations before publishing
   25        if (moderationResult.transform) {
   26            // Replaces the outgoing message if a transformed version is provided (e.g., word masking)
   27            if (moderationResult.transform.message) {
   28                request.message = moderationResult.transform.message;
   29            }
   30
   
   31            // Merges moderation-provided meta (e.g., report identifiers) into existing meta
   32            if (moderationResult.transform.meta) {
   33                // Parses existing meta (arrives as a string in Functions)
   34                const existingMeta = JSON.parse(request.params.meta || '{}');
   35                // Merges existing and moderation meta objects
   36                const transformedMeta = { ...existingMeta, ...moderationResult.transform.meta };
   37                // Assigns back as a string for the runtime to forward
   38                request.params.meta = JSON.stringify(transformedMeta);
   39            }
   40        }
   41
   
   42        return request.ok(); // Proceeds with publish (possibly with transforms)
   43    } catch (err) {
   44        // On moderation errors (service/network), logs and allows the publish to proceed
   45        console.log('Moderation Error:', err);
   46        return request.ok();
   47    }
   48};

   show all 48 lines

6. Go to the Functions module on the Admin Portal and select the active Before Publish or Fire Function to which you want to add moderation configuration.

   

7. Add the previously copied moderation configuration code to the Function's body. The way you modify the existing Function's code differs depending on the Function logic.

   For example, this is an original body of a Function that tags detected @mentions in meta so downstream code/clients can notify mentioned users, highlight mentions, or apply mention-specific rules—without re-parsing the text:

   1// Before Publish Function entry point; runs before the message is delivered.
   2// You can modify `request.message` or `request.params.meta` here.
   3export default (request) => {
   4    // Regex to detect @mentions like @alice or @user_123 (case-insensitive)
   5    const mentionPattern = /\B@[a-z0-9_-]+/gi;
   6
   
   7    // Normalize the incoming message to a text string:
   8    // - if it's a string, use it directly
   9    // - if it's an object, try to read the `text` field
   10    // - otherwise, default to an empty string
   11    const text =
   12        typeof request.message === 'string'
   13        ? request.message
   14        : (request.message && request.message.text) || '';
   15
   
   16    // Extract mentions, lowercase them for consistency, and de-duplicate
   17    const mentions = Array.from(new Set((text.match(mentionPattern) || []).map(m => m.toLowerCase())));
   18
   
   19    // Parse existing meta (Functions pass meta as a JSON string).
   20    // If missing or invalid, start with an empty object.
   21    let meta = {};
   22    try {
   23        meta = typeof request.params.meta === 'string' ? JSON.parse(request.params.meta) : (request.params.meta || {});
   24    } catch (_) {
   25        meta = {};
   26    }
   27
   
   28    // Add mention info to meta for downstream consumers (clients, After Publish Functions)
   29    meta.mentions = mentions;                  // e.g., ["@alice", "@bob"]
   30    meta.hasMentions = mentions.length > 0;    // quick boolean flag
   31
   
   32    // Persist meta back to the request as a string
   33    request.params.meta = JSON.stringify(meta);
   34
   
   35    // Allow the publish to continue
   36    return request.ok();
   37};

   show all 37 lines

   Code with embedded moderation configuration:

   1const ugc = require('ugc');
   2
   
   3export default async (request) => {
   4    // Parse existing meta (string → object)
   5    let meta = {};
   6    try {
   7        meta = typeof request.params.meta === 'string'
   8        ? JSON.parse(request.params.meta)
   9        : (request.params.meta || {});
   10    } catch (_) {
   11        meta = {};
   12    }
   13
   
   14    // Run moderation (block or apply transforms)
   15    try {
   16        const res = await ugc.moderateMessage({
   17        configId: '<YOUR_MODERATION_CONFIG_ID>',
   18        message: request.message,
   19        userId: request.params.uuid,
   20        channel: request.channels[0],
   21        meta,
   22        });
   23
   
   24        if (res.flagged && res.actions.includes('block')) {
   25        request.status = 403;
   26        return request.abort('Flagged');
   27        }
   28
   
   29        if (res.transform?.message) request.message = res.transform.message;
   30        if (res.transform?.meta) meta = { ...meta, ...res.transform.meta };
   31    } catch (err) {
   32        console.log('Moderation Error:', err);
   33    }
   34
   
   35    // Extract @mentions from the (possibly transformed) message
   36    const mentionPattern = /\B@[a-z0-9_-]+/gi;
   37    const text = typeof request.message === 'string'
   38        ? request.message
   39        : (request.message && request.message.text) || '';
   40    const mentions = Array.from(
   41        new Set((text.match(mentionPattern) || []).map(m => m.toLowerCase()))
   42    );
   43
   
   44    // Annotate meta for downstream consumers
   45    meta.mentions = mentions;
   46    meta.hasMentions = mentions.length > 0;
   47
   
   48    // Persist meta back as string
   49    request.params.meta = JSON.stringify(meta);
   50
   
   51    return request.ok();
   52};

   show all 52 lines

   Now this Function moderates via your configId (blocks or applies transforms), extracts @mentions, updates request.params.meta with mentions and hasMentions, then publishes the message unless blocked.

8. Save the changes (as a new revision) and redeploy the Function.

Configuration and Function conflicts

Auto Moderation validates channels when you create or edit a moderation configuration to make sure there is only one configuration and the corresponding Before Publish Function running on a given channel.

When creating or editing a configuration, you can encounter the following conflicts:

• Internal conflict (within the Auto Moderation space in the Admin Portal) — there is already an active Auto Moderation configuration running on the selected channel.
• External conflict (related to the Functions space in the Admin Portal) — there is already an active Before Publish Function running on the selected channel.

When conflicts occur, the UI shows a banner and per‑row badges with instructions on actions you can perform to resolve these conflicts.

Possible conflicts when creating a configuration:

Possible conflicts when editing a configuration:

Follow these steps to resolve configs:

• For the channel duplication on the configuration level, save and activate the new configuration and the currently active one will be automatically stopped.

  

  

• For the channel duplication on the Function level, follow the steps to generate moderation code and then embed it into an existing Before Publish Function.