As software becomes more easily accessible than ever before, you need to ensure your chat application can successfully handle and support typical user behavior, as well as unintentional, odd, or even malicious behavior that can cause issues for your app. Although developers may be able to fix the issues after deploying applications, poor software quality in the US alone recently exceeded $2.41 trillion. These issues are attributed to rushed deadlines, substandard security, incorrect design choices, and scalability problems.
Reduce your Testing Burden with PubNub
Before we dive into the nuances of testing a chat application, let's acknowledge that testing is both a lot of work, and distinctly unglamorous. Ideally, you would focus testing on the key functionality of your app, and rely on message functionality and infrastructure that "just works".
Enter PubNub. PubNub allows you to build direct (1:1), group, and large-scale chat for your users including read receipts, typing indicators, threads, group invitations, user profiles and channel metadata, user presence indicators, message reactions, file sharing, @Username mentions and more.
To understand what PubNub can deliver, see the embedded demo below, designed for desktop use, which is also available at https://www.pubnub.com/demos/chat/. Choose a username to get started.
Convinced?: Sign up for a free trial to get started or Follow a tour to better understand what PubNub is.
Want to speak with someone about your testing needs?: Our Support team are on hand to answer all your questions.
The need for testing your chat applications
As previously mentioned, poor software quality is costing companies trillions of dollars worldwide. Thoroughly testing your chat application is essential to ensure you don’t lose hundreds, if not millions of dollars on errors that can be prevented with careful software testing.
In October 2021, the entirety of Facebook’s infrastructure (Instagram, Messenger, and WhatsApp) went down due to a single command issued by an engineer during a maintenance check on their servers. This command was initiated to determine the health of their network server capacity, which unintentionally took down all connections in the network.
Due to the outage that lasted for roughly six hours, the company lost an estimated 100 million dollars in ad revenue and had to reimburse companies for this loss of revenue. This issue could have been prevented by thoroughly testing commands and testing scripts for end-to-end testing, to ensure commands like these are caught by testing frameworks and tools. Even though Facebook has run drills that simulate issues like these, training team members to handle critical errors such as these and having safeguards in place can prevent these kinds of errors from occurring again.
Create test cases for your chat application based on goals and usage
Because there are different types of software testing, you’ll need to ensure you create tests that make sense for your chat application. Although some tests can be applied to chat applications regardless of audience, many tests need to be tailored toward the type of chat application you are implementing, as chat applications have different usages and audiences.
Is your chat application meant to be the main focus of the product, such as Messenger, WhatsApp, Discord, and Slack? Or is it meant to be used as an auxiliary with online video games, customer support for e-commerce applications like eBay and Amazon, or streaming platforms such as Twitch or YouTube? The style of your application will drive the direction of your test cases.
You’ll need to then consider what devices your users will be operating your chat application on. Is it strictly mobile or are you offering a web, desktop, and mobile cross-platform experience? This is especially important as you’ll need to implement end-to-end, acceptance, and performance tests based on the different browsers, desktop, and mobile communication between them.
Within the application, you’ll need to test to ensure your users will have a good user experience. Users will send messages to other users, groups of users, and even servers. These messages can be text, emojis, GIFs, web links, mentions of other users, reactions to other messages, and even files. Users should be able to organize these different groups as they see fit and allow them to change settings for these different types of messages and chat application settings.
Create a short list of test scenarios
The specific test cases for your chat application depend on the type of application, business requirements, audience, and design. However, there are other general areas you want to ensure you create test cases for.
You can organize these areas and determine if the tests you will create will be for manual testing or automated testing:
Functionality: Test the essential features of the chat application. Users must be able to send, receive, and view messages. They should also be able to open web links, files, and GIFs. Users should be able to open their friend list, manage direct/group/server messages and call other users via video and audio all with ease.
Security: Messages sent across the network should be protected with TLS and AES256 encryption. Users signing into the application should be encrypted and ensure that any SSO options are properly functioning. Permission rights should be working as intended, which means that users have to be invited to join conversations, friend groups, or servers.
Platform: The chat app should function properly on different devices. Accessing the application on desktop (Windows, macOS, Linux), mobile (Android and iOS), and web (major browsers) devices should be seamless. Ensure devices can communicate cross-platform properly.
Practicality: Test the chat application’s usability, ensuring users have a seamless experience in operating the application.
Performance/Scalability: Test the application under heavy load. Populate with multiple users at a time, sending multiple messages from different users, making requests to the server, etc.
Accessibility: Ensure the application’s localization features are properly supported and translated correctly. Ensure accessibility options (such as for color vision deficiency users) properly work.
Integration: Ensure the application easily connects with third-party applications, databases, and cloud services.
Safeguards: Run test cases for when the application loses network connectivity, or if the network servers supporting the application fall offline.
Test cases for chat applications
Based on the general area discussion previously, you can create more detailed test cases based on the following areas. You should create the unit tests and determine how to best create the detailed version of these tests for your unique application when you perform your functional testing, application testing, and performance testing. This list is not exhaustive and is meant as a starting point for you to expand upon and create detailed test cases unique to your application.
Functionality
Users can send basic text messages to other users. A delivery receipt should show the message has been sent.
Users can send emojis and GIFs to other users.
Users can receive and view new messages from other users.
Unread messages should be highlighted or marked in some way that shows that it is unread.
Read messages should no longer be highlighted or marked as such. A read receipt should display for the original sender to show that the message has been read.
Missed messages when not using the application should be indicated on mobile devices with push notifications to alert the user (banner, badge icon, etc). If there are settings to send an email to offline users, these emails should work based on the appropriate settings.
Messages should be able to insert hyperlinks to web pages and create a preview of these links when the message is sent.
Users should be able to react to messages with emoticons.
Ensure users can copy and paste text and links in the message box.
Users can send and receive files within the size limit of files defined by the application.
Ensure the appropriate message is displayed when users try to send files too large or the extension is not supported.
Ensure the file types supported by the application can be opened and sent.
If files are supported and drag and drop is supported, ensure this functionality works properly.
Status icons for users should be displayed appropriately (green for online, grey or red for offline) and allows users to change if the functionality allows.
Users should be able to open their friends list to view the online status of friends.
Users should be able to search for and add friends, as well as receive and accept incoming friend requests.
Users should be able to manage their friends in the friend list, such as sending them a message, promoting their status of friendship (starred, best friend, etc), or removing the friend.
Users should be able to search for other users, previous messages, discussions, and links.
Users should be able to change their profile picture and enable settings to change the functionality of the application.
Attempt to send messages with special characters (especially related to SQL injection).
Ensure the application’s profanity filter when sending a message is properly functioning.
Users should be able to send messages to individual users, add other users to a group chat, and manage these relationships.
Users should be able to create and invite other users to servers/organizations.
Users should be able to make audio and video calls to other users. They should be able to adjust their audio/video settings while on these calls, as well as turn on/off their cameras, and mute their microphones.
Security
Ensure that messages sent across the network are protected with TLS and AES256 by using network monitoring tools and watching the messages as they are sent/received.
Ensure that messages are being stored in the correct region for data privacy laws and responsiveness of obtaining the data.
Test the login process for users. Ensure that users are securely authorized and cannot be circumnavigated.
If users can log in using SSO or third-party applications, ensure this login process functions correctly with your system.
Users should be able to manage their friends, group chats, and servers appropriately. Server admins/owners should be able to manage other users, granting specific permissions unique to that server/group chat. Private conversations between users should not be able to be intruded upon by a third party unless invited.
Platform
Every feature in the chat application runs properly across every supported browser.
Two users using the application in separate browser windows should be able to send, receive, and view messages between each other.
Users downloading the desktop version of the chat application from the web or software store should be in a file format that makes sense for the desktop.
The chat application on either web or desktop windows should be able to respond to different size screens, from mobile internet browsers to ultra-wide monitors.
Users should be able to download the chat application on mobile devices through major online stores, such as Google Play and the App Store.
The native mobile application should be properly formatted to run on different mobile device screens, from smartphones to tablets.
Practicality
Installing the application should be a seamless process with minimal information necessary.
Users should be able to easily create an account or log in using a username/password combination, SSO options, or third-party integration (such as with Google, Facebook, etc).
Determine the usability of the different features of the application. Test the difficulty in adding friends, sending messages, adding friends to servers, etc.
Test how easy it is to find resources that explain the different features of the application.
Test the different UI or alert settings in the application, such as the theme, notifications, and text font/sizes.
Performance/Scalability
Perform different actions quickly across all the application platforms for one user. The application should be optimized for a smooth, responsive experience.
Repeat the previous test but with two users accessing resources from the network server at the same time.
Populate the application with multiple users online at the same time. The application should be able to scale and handle all of the incoming requests.
Repeat steps 1-3 but with hundreds/thousands of simulated users.
The web and desktop versions of the chat application should not consume unnecessary browser and computer resources.
The web and desktop versions of the chat application should allow for multitasking with other applications while in use.
The mobile version of the chat app should not consume unnecessary resources.
The mobile version of the chat application should be allowed to be placed in the background without having to restart the application.
Accessibility
Ensure the application can be downloaded in the different regions that are supported.
Test the application’s localization features that are offered. Ensure that the application’s settings, messages, and features are translated into the appropriate language.
Messages sent between users in different languages should be translated appropriately and/or display that language’s characters correctly.
Test the different accessibility options provided by the application. Test that the options for color vision deficiency users display the correct color spectrum and persist throughout the application, as well as when logging into the application on another platform.
Integration
Test signing into or creating an account in the chat application works if using SSO and third-party services, such as a Google or Facebook account.
Users should be able to add third-party applications into the chat application itself, such as chatbots, searching for a library of GIFs, video/audio calling, etc.
Ensure that this integration manager for these third-party services allows users to search for these features, and how users can create or add their integrations.
Recovery
Ensure the appropriate action takes place in the chat application when a user is completely disconnected from the internet.
When in the web and desktop versions of the application, test the appropriate action that takes place when a user is disconnected from LAN but is connected to WiFi.
When in the web and desktop versions of the application, test the appropriate action that takes place when a user is disconnected from WiFi but is connected to LAN.
In the mobile version of the chat app, ensure that the appropriate action takes place when a user is disconnected from WiFi but is connected to mobile LTE.
Test what happens when a user’s application suddenly gets disconnected, freezes up, or shuts down due to a device issue.
Simulate what happens when the network server that operates the network backbone of the chat application goes down.
Getting Started with PubNub for your chat app
Creating and implementing test cases for chat applications is a vital aspect of ensuring your chat app can handle correct user interactions, proper security measures are implemented to protect your users and company, and an infrastructure to handle the scalability of your chat app to ensure a reliable and secure experience.
As you continue to build your application, you’ll need to keep in mind the different types of interactions and features your users require. Users will want to be able to join different servers or organizations that are uniquely created for their hobbies or professional use. They will want to be able to collaborate on different channels or topics, as well as be able to have group chats or even 1:1 conversations with friends and coworkers in private chat rooms. Users should be able to access a member list that shows if their friends or channel members are online. When typing messages, users should be able to see when other users are typing and responding and allow for message reactions and emoji support to keep conversations lively.
Powering these features is by no means an easy task. You have to set up an infrastructure to maintain the communication platform that allows for messages to be delivered to users, detect an online presence system, load messages when users come online, and maintain these features securely for your users, on top of building the UI for your chat app. While it is possible to create this infrastructure yourself, this will take time, resources, and upkeep that can be spent elsewhere. This is where PubNub can help.
As mentioned earlier, PubNub allows you to build direct, group, and large-scale chat for your users including delivery and read receipts, typing indicators, user and channel metadata, user presence indicators, message reactions, file sharing, and more.
Learn how you can get started with PubNub for chat applications with the following resources:
See how other customers depend on PubNub to power their chat applications.
Follow a tour to learn the basics of PubNub.
See our chat demos that cover both web and mobile, including how to create a chat app in 10 lines of code with JavaScript.
Follow our step-by-step tutorials to set up and build applications with PubNub, including building chat applications with PubNub SDKs.
Learn how to build chat and other industrial applications with PubNub.
Feel free to reach out to the Developer Relations Team at devrel@pubnub.com for any questions or concerns.
