Sonic.net

So this is driving me bats. The ping time from my firewall to the downstream hop (Sonic fiber) is 1-2ms .. except every 2 seconds it consistently jumps 100-200ms for about 0.3 seconds, before returning to the more usual 1-2ms. I have no idea why. I've tried different firewall hardware (both pfSense mind you) but the problem persists. The internal network is fine (and this ping-ing is happening from the firewall anyway).

So, just me? The command i'm using to see this is 'ping -i 0.1 <downstream ip address>'.

Our nodes do not prioritize responding to pings so that may be what you are seeing. Are you having issues with responses/latency outside of our network? If ping time is consistently low to a destination beyond that hop then this would be expected behavior. Try running a traceroute to another destination to get a full picture.

I see the same thing with 'ping -i 0.1 1.1.1.1'. It's the consistency of this problem I find so weird. If this was some traffic priority thing I'd expect it to vary in some way, but it doesn't.

The first line of traceroute (100ms delay, 20 queries) to 1.1.1.1 looks like this:
Again you see these moments where the time bumps up from the usual ~1ms to ~500ms and, again, it's the consistency of these bumps that's so odd to me.

I've not given up on it being some failing hardware on my end, so I'll replacing more tomorrow and see if it changes anything.

Your traceroute results are exactly what we would expect. You would get similar results across most of our services using different devices. I don't believe swapping hardware would change much but you are welcome to try.

Could you explain a little about why this behavior pattern is expected and so consistent?

Well I'm going to have to open a ticket with Sonic. The problem persists and about once a day get's so bad that I get packet loss and the network goes down completely. In desperation I just plugged a laptop into the ONT but I still see the same issues (so I guess it's not my hardware at least). Tried new cables - still the same. My RTT on pings (as measure by pfSense - a rolling average) used to always be sub 5ms, but these days it's 120ms to 200ms.

If you are having issues when connected straight into the ONT, try power cycling the ONT and make sure the Ethernet cable is securely plugged in. If problems persist we may need to setup a dispatch.

So, after much messing and testing, I've concluded this problem is related to outgoing traffic - the more of it there is, and the more consistent it is, the more I see this problem. I'm not doing anything terribly exciting - incoming vpns for family and a minecraft server for the kids and their friends - but once I increase the load on those (consistently 20+ Mb/s outgoing say) the problems becomes really obvious. You can still see the issue at lower traffic levels too, but there the ping times are only going up to 10ms which is hardly problematic.

Now as far as I know Sonic doesn't pull any traffic shaping shenanigans, so I'm at a loss as to what's going on (and yes, power cycled the ONT, got a new ONT last week, new cables, different new cables .. still the same). I had another Sonic customer run this test for me (they have the same 2 down/1 up configuration) and they don't see this issue at all.