Someone called and asked me to look into a problem, slow downloads happening from remote clients, using the VPN connection.
I asked to join in the technical conversation, to discuss the issue. The complaint, several people when downloading files, spreadsheets, have very slow speed. In further conversation I found out that it was only downloads where the problem was occurring.
I was quickly supplied with a packet trace on the inside of the VPN.
In the packet trace two problems became clear, Microsoft broadcast packets were in a routing loop, and second, lots of retransmissions.
Routing loops can be found pretty easily. When you see packets that look the same, repeated several in a row, you then need to look for two conditions, IP identification ID and the Time-To-Live decrementing downward. Packets usually start with a higher number, let's say 128 and then it decreases down for each router the packet goes through. Seeing them on the same segment or VLAN, indicates the same packet being sent back and forth between two routers, and will continue until the Time-To-Live decreases to one. The packet is then thrown away by the router to protect this from a continuous loop.
Figure: 1. Wireshark Packet Trace Routing Loop (Time-To-Live with IP identification ID)
Figure: 2. Wireshark IO Graph Broadcast Traffic (packets per second)
The second issue was retransmissions, all coming from lost packets, upstream from where the packet trace was taken pointing to the VPN appliance and on inbound traffic. With further investigation it was found that the MSS size was to small. So adjusting it to a smaller MSS caused the packets not be segmented.
Figure: 3. Wireshark Packet Trace Retransmissions
Figure: 4. Wireshark IO Graph (Packet Loss & Throughput)
Things to Consider
TCP retransmissions-dropped packets, it’s a good place to start looking for what is causing this problem, it can take you to the Root-Cause.
Finding Routing Loop-In the trace file, do packet look the same, such as a group 10 or more. To find a routing loop, look for the IP identification field show here in this trace (0x24e1 this is hex value and also shows numeric 9441), is it the same value in each packet? --Yes!-- That's not a loop yet, you need to see if Time to Live (TTL) is the same, or secondly, is the TTL is decrementing. Easy way to see this is to select the Time to Live in the IP header, select-click-apply as a column.