Performing client testing is a vital part of our continuous efforts to improve the quality, speed, and reliability of our software. Not only do we look out for common issues, such as malformed packets and various server response codes, but in line with our use cases, we also delve into more complex situations, like dealing with TCP resets (RST) and evaluating TCP packets. Ensuring we cover all these bases significantly enhances the overall user experience as it enables us to identify and clear potential problems before they affect end users and hamper throughput.
TCP resets, or TCP RST, may occur for a variety of reasons. For instance, if a mobile proxy believes you've been connected for too long, it might issue an RST, accusing you of idling and unnecessarily consuming bandwidth. Alternatively, if something unfortunate happens within the Linux operating system, a TCP reset might be dispatched from the server. For example, suppose the web server process exhausts its memory while serving a page. In that case, its last action might be a TCP reset, informing the client that the process has come to an unexpected halt. It's the kind of situation that throws up the dreaded "The Connection has been Restarted" browser error page when trying to test the environment.
Testing for headers and malformed JSON packets is a relatively straightforward task, which can be executed from the command line, since most of these responses can be simulated using high-level languages and frameworks, such as Ruby on Rails or Python. You can follow the code snippet below, hosted on GitHub, to emulate these scenarios in a Unix-based operating system:
Why Induce a TCP Reset?
Inducing a TCP reset might seem like an unnecessary action. However, this is a crucial part of our TCP/IP protocol specification tests. We need to understand how your system reacts to such events. Essentially, dealing with a TCP RST can be tricky because TCP/IP is not a trivial protocol and handling the intricacies of setting flags, ACK/SEQ sequence numbers, etc., can be quite complex. The debug process isn't straightforward largely to prevent malicious use of TCP RSTs, which are a tool in the Denial of Service (DoS) attack toolkit.
How to Induce a TCP Reset
Inducing a TCP reset requires a relatively simple three-step process using our open source tools:
Setup a port forward from the remote server to your localhost.
Configure your clients to connect to your local IP address.
Use IPFW to reject connections and issue a RST.
For this process, you'll need tools such as SSH (Secure Shell) and IPFW (IP Firewall) available in your macOS or any Linux-based operating system. These tools are already installed by default if you're using a Mac. If not, click the links to learn how to install them on your system. Please note that these tools are an integral part of our unit testing setup.
Also, be sure your sshd_config has the setting: GatewayPorts client-specified
This will enable your port forwards to be accessible by remote hosts. Then port forward to your remote server infrastructure:
sudo ssh gcohen@localhost -L 0.0.0.0:80:remoteServer.example.com:80
Replace as appropriate the login, and remote host info. The above command line says:
Logging in as myself to my local machine, I am forwarding the remote port 80 of remoteServer.example.com to my local machine’s port 80. The 0.0.0.0 dictates for SSH to bind on all local IPs (my physical, and local interfaces). sudo is needed because port 80 (locally) has special requirements, which require admin access. If you were already running say, a web server on port 80 locally, then you would get an error like “Port already in use” or “bind: Address already in use”.
You can test to see if its working now, by telnetting to port 80 on your local host. If it works, then the tunnel is working. If not, make it work before proceeding.
Next comes the magic… we’ll use IP Firewall to send a reset to anyone currently connecting to example.com via your machine.
Run the following command:
Caution: Doing this on a remote system where you have no physical access will most likely deny you [pseudo-]permanent access to that machine. Use at your own risk.
sudo ipfw add 130 reset tcp from any to any 80; sleep 3; ipfw -q flush
This command says:
Refuse and send a TCP RST to anyone trying to connect from anywhere to anywhere on port 80
Wait 3 seconds
Throw out that TCP RST rule we just created, and restore the system to the default (allow anyone to connect)
You can make this command more precise, by giving the exact IPs of the source and destination traffic:
sudo ipfw add 130 reset tcp from s.s.s.s/32 to d.d.d.d/32 80; sleep 3; ipfw -q flush
You could also replace 80 with 443 (to test SSL for example).
Just be warned, using this the wrong way can lock the Internet out from your machine, so be sure you have the ability to run the flush command:
sudo ipfw -q flush
If anything goes wrong, it should restore your config to where it was before. I took note of my default config:
root@:/Users/gcohen$ ipfw list
65535 allow ip from any to any
Just in case I did something stupid. Never hurts to have a backup plan.
How can PubNub help you?
At PubNub, we've built our platform as an API to support developers in their quest to build, deliver, and manage real-time interactivity for web apps, mobile apps, and IoT devices. The foundation of our platform is the industry's largest and most scalable real-time edge messaging network, boasting over 15 points of presence worldwide, supporting 800 million monthly active users, and maintaining a 99.999% reliability. We strive to conform to both TCP and UDP protocol specifications to ensure optimal performance.
Get Setup
When you sign up for a PubNub account, you get immediate access to PubNub keys for free.
Our PubNub docs are also there to guide you through your journey with us. They are made to be clear and easy to follow, irrespective of your use case or SDK. They include helpful details about our services, including how to configure your account, how to manage your firewall settings, and how to perform successful TCP connection tests.