Gigabit Fiber Ludicrous Speed (990mbps+)
Posted: Tue Feb 05, 2019 7:00 pm
I'm extremely happy with my Gigabit Fiber. I consistently get 940mbps symmetric on speed tests when I'm the only one using the network. Using iperf3, I'm able to see about the same speed between wired computers on my network which indicates to me that 940mbps is about the highest data transmission rate I'm able to achieve over ethernet. This makes reasonable sense because of the packet overhead for TCP networking limits gigabit ethernet under normal conditions limits data transmission to around that speed (see also: a good article with the math). Everything is functioning as it should at its max speed.
All of this is great, and I am very happy with 940mbps, but I have been curious about something for a while. What follows should be viewed as an academic exercise to understand how things work.
When the Sonic technician (Don Pierson -- he was great!) came for the first time, I looked over his shoulder as he ran a SpeedTest.net test against the FastMetrics Inc. server from his Dell Windows laptop. He was able to achieve 994.32mbps down! I've been reading about MTU and jumbo frames, and it seems like the only way to transmit data that quickly over a gigabit ethernet connection is by enabling 'jumbo frames' (MTU: 9000). This means every packet includes up to 9000 bytes of data per packet (instead of the default 1500), thus minimizing the packet framing overhead. Experimentally on my local network, when I enable jumbo frames, I'm able to get iperf3 scores of 990mbps between computers on my network.
This all suggests to me that somehow, the Sonic technician was receiving jumbo frames from the fiber modem (optical network terminal - ONT). Or possibly somehow he was using ipv6 (though in my understanding that's not yet enabled for fiber customers). I've tried enabling jumbo frames on my computer and plugging straight into the ONT. I've also tried then restarting the ONT in hopes that maybe it does some kind of path MTU discovery at startup to determine the MTU. However my speed tests still top out at 940mbps and running `ping -D -s 8972 GATEWAYIP` using the IP of the gateway I'm getting from DHCP shows that I'm not able to send packets to the gateway larger than MTU~2000. I'm a bit stumped (and very curious) as to how that technician was able to achieve 994mbps down! if anyone had any information or explanation I'd be very excited to hear it!
All of this is great, and I am very happy with 940mbps, but I have been curious about something for a while. What follows should be viewed as an academic exercise to understand how things work.
When the Sonic technician (Don Pierson -- he was great!) came for the first time, I looked over his shoulder as he ran a SpeedTest.net test against the FastMetrics Inc. server from his Dell Windows laptop. He was able to achieve 994.32mbps down! I've been reading about MTU and jumbo frames, and it seems like the only way to transmit data that quickly over a gigabit ethernet connection is by enabling 'jumbo frames' (MTU: 9000). This means every packet includes up to 9000 bytes of data per packet (instead of the default 1500), thus minimizing the packet framing overhead. Experimentally on my local network, when I enable jumbo frames, I'm able to get iperf3 scores of 990mbps between computers on my network.
This all suggests to me that somehow, the Sonic technician was receiving jumbo frames from the fiber modem (optical network terminal - ONT). Or possibly somehow he was using ipv6 (though in my understanding that's not yet enabled for fiber customers). I've tried enabling jumbo frames on my computer and plugging straight into the ONT. I've also tried then restarting the ONT in hopes that maybe it does some kind of path MTU discovery at startup to determine the MTU. However my speed tests still top out at 940mbps and running `ping -D -s 8972 GATEWAYIP` using the IP of the gateway I'm getting from DHCP shows that I'm not able to send packets to the gateway larger than MTU~2000. I'm a bit stumped (and very curious) as to how that technician was able to achieve 994mbps down! if anyone had any information or explanation I'd be very excited to hear it!