DMZ mode on FTTN Pace Modem stopped working

[myname]

In second week of December 2019, AT&T did something to their PACE modem and my DMZ mode has stopped working without any apparent indications. I blamed my router, replaced it without any success. Then I found following post.

https://forums.att.com/conversations/at ... a13b#M7940

I tried the solution and confirmed that PACE modem is the culprit, but forwarding all the ports is not working 100% for me. My VPN and VOIP are failing intermittently.

I use my own router/firewall, VOIP and VPN server. It will not work with double NAT. I do not trust AT&T router to provide sufficient protection and prefer to use my own firewall.

Sonic support says "Unfortunately this is unsupported territory". Why is this unsupported? I thought sonic was all about giving best service with knowledgeable staff. DMZ is well accepted industry practice for people who know what they are doing. And, it was working fine till AT&T screwed up.

If Sonic staff is unable to understand DMZ mode, How do I get Arris BGW210-700 as suggested by someone on that forum? Do I need to involve Mr. Dane Jasper for such a trivial matter?

[chirano]

You can try to upgrade or downgrade the Pace firmware to a version where DMZ Plus works. AT&T may push the defective firmware again to your gateway, but some users have found that not to happen in their case. There's a thread in the AT&T forum on DSL Reports about the Pace firmware where you can learn which versions work, which ones don't, and where to get them.

[gtwrek]

Firmware downgrade no longer works for me on my Pace. That was my solution previously when DMZ+ stopped working. Now ATT's pushed another update, and I can no longer downgrade my firmware.

viewtopic.php?f=10&t=8449

ATT has really botched this firmware update. I'm exploring various more drastic options now, like just using the Pace for 802.11 authentication only, and then completely turning off the PACE after that occurs.

It's sad that ATT is forcing this problem on users who wish/need to use their own router.

[myname]

Thanks for the replies. So, I am not alone. How do we escalate this to Sonic Support?

[dct]

Sonic support says "Unfortunately this is unsupported territory". Why is this unsupported?

This is unsupported territory because the network and the equipment is provided and maintained by AT&T. When they push a firmware update across their network and devices, there isn't anything we can do to intervene. We definitely understand how frustrating that is - it's frustrating on our end too, we want your service the work the way it should!

Looking at the thread you linked from AT&T's support portal, it looks like the most recent version of the firmware, 11.4, resolves the DMZplus issues, but may introduce problems with wifi calling according to at least one post.

If that proves unsuccessful, we can request AT&T replace your equipment, but we've had limited success in the past dictating which equipment should be used, or even that the device is replaced, as it is at the discretion of the technician. Please reach out to me or call our support team to get this coordinated.

[gtwrek]

Dan,

11.4 does NOT solve the DMZ+ issues. SSH connections are pretty much dead across the link. I don't understand the underlying failure mode. I don't know if this failure mode is related to the wifi calling failure.

DMZ+ is listed at ATT as supported (https://www.att.com/esupport/article.ht ... gsi=1n7s6l)... But doesn't work with 11.x firmware. ATT seems disinterested in fixing.

I understand Sonic is caught between a rock and a hard place here, but here's where we are...

Edit : FWIW, posts seem to indicate that the Arris Router works fine. However, getting ATT to actually deliver one of those seems to be impossible without a truck roll from ATT, with the tech physically delivering the correct router.

Regards,
Mark

[gtwrek]

FWIW I've installed wireshark on my router, and captured some bad SSH sessions. I'm a wireshark wannabe - I can setup captures just fine, but analyzing the data isn't something I've really done.

My setup:
On, my local router, setup wireshark capture:

Code: Select all

tshark -i eth1 -f 'net 208.201.242.22' -w capture-output3.pcap

The ip address is "sh.sonic.net" (I've got a sonic shell account)
I then run:

Code: Select all

scp gtwrek@sh.sonic.net:foo ./runtest1

File is dummy random data file length=2M

I typically get results like:

Code: Select all

foo  100% 2048KB  19.1KB/s   01:47

Note the performance keeps going down on larger file xfers. I'm trying to keep the pcap files to a minimum for the moment...

Wireshark (from my naive interpretations) seems to be showing many DUP TCP Acks, and TCP Out-Of-Order errors.
Some sort of retransmission timeout problem?

--Mark

[myname]

Hi Dan,

My version is 11.4.1.532484-att and I can confirm that DMZ does not work.

[gtwrek]

FYI, I followed the instructions for completely bypassing the Pace from here:
https://www.dslreports.com/forum/r32491 ... ce-gateway

TL/DR one uses the Pace to do the 802.11 authentication. Then, one powers down the Pace, and runs your router (from the same switch), with the cloned Pace MAC address. Your router can then connect to upstream WAN just fine. The Pace remains powered down and unused from then on.

With this workaround, I'm now seeing my previous Performance:

Code: Select all

scp gtwrek@sh.sonic.net:foo ./runtest
foo 100%   16MB  58.9MB/s   00:00

When using the borked firmware Pace this xfer to 20 minutes and I was getting around 10 KB/sec. (On my 1 Gb/s Fiber link).
I think this pretty much definitively concludes that the 11.4 ATT Pace firmware is trash.

I've got wireshark PCAPs for both the good and bad xfers in case I feel motivated in debugging more (if anyone else is interested I'd be happy to share these as well)

This setup doesn't have a high Spouse Acceptance Factor - in case of power out, one needs to physically swap cables on the various routers to re-init a link. I've added battery backup to all, so there's that mitigation. I may be motivated in trying some more automated re-initialization methods involving a net (or usb) controlled power switch on the Pace, and some scripting. This all depends on how reliable the link is...

Regards,

Mark