Automatic image moderation

File Sharing

To use image moderation, enable and configure File Sharing for your chat app keyset on the Admin Portal. Check Required Configuration for details.

Moderation Dashboard also supports image moderation that administrators can use to detect and filter any unwanted content in images in real time. Once image moderation is set, it will automatically block any inappropriate images sent by a user of the moderated chat app.

This functionality, similarly to text moderation, is enabled by Functions which can perform the real-time transformation of published images before they are distributed to subscribers. Once the image moderation is configured, the Moderation Dashboard GUI calls the PubNub provisioning API and deploys the configured logic as the Before Publish File function. You can view this function in your keyset for the moderated app on the Admin Portal.

Image moderation is enabled through the integration with a third-party moderation provider called Sightengine which requires an account with such details as API User, API Secret, and Workflow ID. Follow the Image Moderation Workflow instructions to configure a workflow where you can specify the type of content you want to moderate.

To configure image moderation, expand the Settings menu on the left-hand navigation of the Moderation Dashboard GUI and select Image Moderation. Just like with text moderation, this option has the on/off toggle at the top. Make sure to enable this toggle at the start and save the configuration after you're done.

Available settings

With image moderation, admins can:

• Select either a channel name or a channel pattern in which they want to apply image moderation in their app. When enabled, image moderation forces you to use the same channel pattern as you used for text moderation and inherits the channel pattern specified there.
• Adjust the risk factor for the moderation that SightEngine performs to block the complete image if it passes the minimum threshold.
• Choose to reroute the message to a *.banned channel, same as with automatic text moderation.