Buffer Bloat

[geogriffin]

I think that buffer bloat is a red herring here, and suspect that QoS is the real challenge. This is something we are actively working with Pace on, the 4111N doesn't currently support upstream ACK prioritization, and that can impair performance during times of upstream saturation with some applications.

agreed.. maybe it's not really 'bufferbloat' if it's on the CPE. but since there is an upload buffer presumably for performance reasons with DSL, then yes, control packets should be brought to the front of the queue. does that sound right? (this is purely curiosity)

FYI, i just got an AT&T-provisioned Pace 5268AC for Sonic FTTN and it is suffering the same latency issues during upstream saturation as the Pace 4111N did with Fusion.

[dherr]

Maybe I am misunderstanding the point but note that "buffer bloat" can be added on either side of the CPE or from the CPE itself. The NIC could come with built in buffering, the OS could be buffering, etc. I don't have any idea how likely either is for reasonably current hardware & software tho. The only point that I am making is that "Buffer Bloat" as a problem can be due to either inbound or outbound saturation and it can come from very close or very far. Not saying that I am pushing for this as an answer to this individual issue, just trying to add some clarity on the possibilities.

[geogriffin]

Maybe I am misunderstanding the point but note that "buffer bloat" can be added on either side of the CPE or from the CPE itself. The NIC could come with built in buffering, the OS could be buffering, etc. I don't have any idea how likely either is for reasonably current hardware & software tho. The only point that I am making is that "Buffer Bloat" as a problem can be due to either inbound or outbound saturation and it can come from very close or very far. Not saying that I am pushing for this as an answer to this individual issue, just trying to add some clarity on the possibilities.

yeah but the 'bloat' part of 'bufferbloat' refers to ISP routers in the middle inflating their buffers over time as a bandaid for congested links, to drop less packets and make the congestion appear to go away while actually sacrificing latency. and 'bufferbloat' may refer to the cumulative effect of all inflating buffers all over the whole system.

in our CPE case, I think the buffers are probably a justified means of improving performance over DSL, provided that some basic QoS is performed as Dane mentioned.

[dherr]

Ah, got it, thanks much. That does make the usage of that term much more understandable.

[dtaht]

I do not think bufferbloat is a "red herring" here. The vast majority of modern dsl interfaces are dramatically overbuffered, and your 1+second results in line with rather large datasets. www.dslreports.com/speedtest/results/bufferbloat?up=1

Best results would come from using a modem with absolute minimal firmware buffering using bytes rather than packets,"BQL", essentially, then combined with a latency sensitive aqm/fq system like fq_codel on top of that. Older DSL modems did this, with hardware flow control. The only DSL device I know of getting it right this way, today, is free.fr's revolution V6 modems.

Newer ones overbuffer and connect to switches that cannot do hardware flow control.

Second best results are using a good QoS system that also does fq and aqm underneath, running at slightly less than line rate - and many QoS systems do do that, nowadays. But I was reluctant to call it QoS because that implies that packet prioritization is useful when there are seconds of uncontrolled buffers underneath. We ended up inventing a new term - "smart queue management" that described things better. See openwrt's sqm-scripts for details.

I am not a fan of ack prioritization - what's an ack? In IPv6? In QUIC? In other protocols? - but of a combined fair queuing and aqm approach as per the above.

[geogriffin]

I am not a fan of ack prioritization - what's an ack? In IPv6? In QUIC? In other protocols? - but of a combined fair queuing and aqm approach as per the above.

good point, though I think packet prioritization implies that the system know something about the transport protocols being used. what do you mean by your comment about ipv6, since that's a lower layer protocol?

I didn't realize some DSL modems utilized hardware flow control. i get the feeling that large buffers and packet prioritization is a bandaid.

[filbo]

FYI, i just got an AT&T-provisioned Pace 5268AC for Sonic FTTN and it is suffering the same latency issues during upstream saturation as the Pace 4111N did with Fusion.

Nuts. I have install appointment next week for FTTNx2 upgrade from Fusion(x1), was hoping to see an improvement from the current Pace "SuperUltraBuffetbloat(tm)" 4111N. This is coming to you from my cell phone (over 1-bar barely visible tower) since my youngest daughter is syncing her Chromebook up to Google Drive before wiping it to install dev mode -> Crouton -> Xenial -> Minecraft. Which means my ping times are hovering around 2500ms with excursions to 4500ms.

[filbo]

4111N can't DNS www.sonic.net (5 tries)

sonic1.png (97.81 KiB) Viewed 8202 times

half hour mtr

sonic2.png (105.87 KiB) Viewed 8202 times

short mtr to www.sonic.net

sonic3.png (72.57 KiB) Viewed 8202 times

[ehill]

I'm seeing huge numbers tracing through ae0.cr2.lsatca11.sonic.net, averaging 1350ms over the past 12 hours or so.