How to Monitor and Moderate Conversations with BizOps Workspace

Restrict what actions a user can perform:

If a User has misbehaved, you will want to restrict their ability to post (mute) or restrict their ability to interact with anyone else at all (ban)

On the server side, you need some way for a moderator to mute or ban a User, as well as a way for that user to be unmuted or unbanned.  You also need to put rules in place that ensure messages cannot be sent or received to users who should not receive them - doing this on the server is far more secure than relying on client-side moderation, which a pernicious user could bypass.

The Chat SDK exposes the setRestrictions() method, which will 'mute' or 'ban' a user on a specific channel, as well as allow you to define a 'ban reason.' The setRestrictions() API itself does NOT apply any limitations on what the client can do; it just notifies registrants that a restriction has been applied.  It is up to both the client and server to listen for 'moderation' events through the Chat SDK's listenForEvents() method.

Listen for changes in user restrictions:

When a user's permissions change, it is important these changes are applied immediately and reflected throughout your solution.  When a user is banned (for example, by a moderator using a web interface), the user's client app shows a message telling that user they are banned.  Any attempt to circumvent the ban by the user, for example, by "hacking" the client application to bypass client-side checks, should immediately fail owing to additional checks on the server side.

The Chat SDK's listenForEvents() method will notify any interested party that a user's permissions in a specified channel have been updated.  The client should listen for these changes to update the user experience and request a new access token from the server.  On the server, any previous access token should be revoked using revokeToken(), and a new token should be generated with the updated access rules using grantToken().

Query what restrictions are in effect:

You need to be able to query what restrictions are in place for users within your solution.  If you are creating a moderation dashboard that runs on a server, then you need to be able to query whether a user is already manned or muted to display this state to the moderator.

On the client side, you need the ability to query whether a message recipient is muted or banned so your app can warn the user that their message will not be received and cannot be sent.

The Chat SDK exposes a family of APIs to retrieve the current restrictions: getChannelRestrictions() and getUserRestrictions(), which act on the User, and getUsersRestrictions(), which acts on the Channel.

Reporting messages or users for inappropriate content:

To provide a safe and enjoyable chat experience for all your users, they should feel empowered to report problematic users or messages to a moderator for review and action.

On the client side, your chat application will need some way for a user to highlight a message as problematic and provide some optional additional feedback.  The Chat SDK provides the message.report() and user.report() methods for this purpose, which the client should call.

On the server side, you need to listen to these message reports and display them to the administrator so they can take action.  They might choose to edit/delete the message itself, or if the issue persists, perform some action on the user, such as muting or banning them.  The Chat SDK provides the listenForEvents() method to receive reported messages, usually handled by the server.

Where does the Channel Monitor fit into this moderation architecture?

The Channel Monitor provides a dashboard to mute or ban users based on a manual review of the messages they are sending.  The dashboard will take care of calling setRestrictions() for you and always presents you with the latest information on which of your users are banned within each Channel.

What remains for you, as the developer, is to integrate these ban and mute requests into your wider application architecture.  Specifically, as a PubNub customer who has deployed a production application, you are already using the PubNub Access Manager to control access to your data, ensuring that users are only permitted the RW access they need for each channel.  Based on the 'mute' and 'ban' updates you receive from listenForEvents(), call into your existing Access Manager server component to revoke and grant new Access tokens for the affected User.

You will also need to ensure that your client application is taking the appropriate steps based on the updates it receives from listenForEvents() or the getXRestrictions() family of APIs.  Do not allow the user to send messages to a banned user since they are destined never to be received.

Muting and Banning Users: Demo using the Sample application

As explained in the previous section, the Channel Monitor is one piece of your robust moderation solution.

This article will show how you can take action on the client when a user is muted or banned from the dashboard, though a production application will also comprise server logic that integrates with the PubNub Access Manager.

The default sample application does not implement the listenForEvents() for API, so we need to make a small change.

In HomeScreen.tsx, add the TimetokenUtils and Alert imports as shown: