Inbound Fragments Dropped over 100Mbps

[mattcorallo]

Huh. This one also appears to show some kind of shaping down to about 40/50 Mbps, which the Sonic lab didn't.

Just as another point of reference, the iperf test works fine for me:

Code: Select all

$ iperf3 -4 -c iperf.he.net -R -u -l 1800 -t 60 -b 200M
warning: UDP block size 1800 exceeds TCP MSS 1448, may result in fragmentation / drops
Connecting to host iperf.he.net, port 5201
Reverse mode, remote host iperf.he.net is sending
[  6] local 192.168.1.45 port 55922 connected to 216.218.207.42 port 5201
[ ID] Interval           Transfer     Bitrate         Jitter    Lost/Total Datagrams
[  6]   0.00-1.00   sec  4.61 MBytes  38.7 Mbits/sec  2.092 ms  9964/12649 (79%)
[  6]   1.00-2.00   sec  5.12 MBytes  42.9 Mbits/sec  1.739 ms  10917/13898 (79%)
[  6]   2.00-3.00   sec  5.06 MBytes  42.5 Mbits/sec  2.197 ms  10935/13885 (79%)
[  6]   3.00-4.00   sec  5.10 MBytes  42.8 Mbits/sec  2.013 ms  10922/13891 (79%)
[  6]   4.00-5.00   sec  5.11 MBytes  42.9 Mbits/sec  2.068 ms  10904/13882 (79%)
[  6]   5.00-6.00   sec  5.53 MBytes  46.4 Mbits/sec  2.478 ms  10723/13945 (77%)
[etc.]

[mattcorallo]

OK, I gave up and shipped one of the same boxes to a friend's rack at HE Fremont (same NICs, same firmware, same kernel, etc) - turns out Linux's fragment reassembly code keeps older fragments and throws away new ones even with ipfrag_max_dist, even with NIC acceleration features off (and those unassembled fragments don't show up in tcpdump, or at least they don't with acceleration on). Setting ipfrag_time lower helped (I mean when do you ever see fragments on the internet separated by a full second?), but still leaves fragments dropped over a relatively low bandwidth. Doesn't answer why my box at HE sees relatively low packet loss but Sonic tops out fragmented UDP at 40/50Mbps, but I suppose that may just be GPON packet loss.

Code: Select all

echo 1 > /proc/sys/net/ipv4/ipfrag_time && echo 1 > /proc/sys/net/ipv6/ip6frag_time