Is a stable IPv6 prefix delegation possible?

[gadams]

I need a stable IPv6 prefix delegation, but I'm having trouble getting one.

I've recently had 10Gb fiber service turned up in Bernal Heights. I'm pleased that IPv6 is natively supported, and I brought it up with no effort. DHCPv6-PD gives me a nice /56 to distribute to my DMZ, trusted hosts, IOT, guest, and management subnets, etc. Excellent!

However, the delegated prefix changes every time I renew my lease. This makes it prohibitively difficult to use, since the hosts on my local network (my file server, DNS server, etc.) keep being renumbered, so they can't be found via IPv6 even within the same network. It also means I can't find my hosts to connect to them when I'm outside of the house.

This is exactly why BCOP690 says that unstable prefixes are harmful. (Read sections 5.2 "Why non-persistent assignments are considered harmful" and 5.3 "Why persistent prefix assignments are recommended.") The Best Current Operational Practices document also stresses that the need to rotate addresses between different clients as was done in IPv4 is entirely absent in IPv6.

I know that the Sonic fiber FAQ says that static addressing is not supported on the fiber service, yet. The wording sounds like it's probably talking about the IPv4 side. I don't really care about IPv4, except insofar as it's needed to reach quite a few web sites and other services (including this forum!). I'm fine with NATting everything behind a rotating IPv4 address. I really just need stable IPv6 addresses.

I've tried a number of workarounds, but they all result in outages of one form or another, or have no effect at all. Among them:

• I've set a stable client DUID, which ideally the DHCPv6 server would use to provide a stable prefix. It did not.
• I've given my subnets ULAs together with subnets of the Sonic-delegated prefix. But ULAs have lower precedence than IPv4 addresses, so this effectively disables IPv6 on the network. (See ULA is Broken in Dual Stack Networks for more on that.)
• Dynamic DNS isn't feasible for all the relevant hosts, and makes the network hard to manage. (I have no intention of making managing my home network a full-time job.) It's particularly difficult for most hosts that implement RFC 7217 semantically opaque interface identifiers that change with each prefix change. It's impossible for the DNS server, which needs a stable IPv6 address to be useable by any of the other hosts in the first place.

And of course, none of these workarounds should be required. There's no shortage of IPv6 prefixes, and delegating stable ones should be the default.

So, my question is whether there is a way to solve this problem in the short term, and how long it will be until this problem is solved in general on Sonic fiber.

I called support to ask about the options, but they told me they aren't trained in IPv6, and suggested I would get more expert assistance here. Does anyone have any suggestions? Or do I need to cancel my service?

Thanks!

[jerrielm]

Hello!

Thanks for the time and effort you put into your post. As you have stated, Sonic does not supply Static IP addresses on its Fiber services. While we are looking into this, it is not currently a priority for the OPS team.

From my understanding of Prefix delegation, it seems that this would allow our customers to create networks on their Sonic networks for other users. This is not something we allow as it could allow for customers to resell our services.

https://help.sonic.com/hc/en-us/article ... t-Policies
"Resale of Sonic services, directly or indirectly, without express written consent from Sonic. This includes sale of Wi-Fi access or any off-premises access to services."

I am sorry if this becomes a deal breaker for you. Let me know if you have any other questions.

Best Wishes!

[gadams]

Thanks for your reply.

I think your impression that prefix delegation is for reselling services is incorrect. The moment any customer, even the most unsophisticated of users, places a router of any kind between the Sonic ONT and their home network, a delegated prefix is required in order to hand out IPv6 addresses to any client devices (laptops, mobile homes, streaming TVs, etc.) on the local network. (The /128 that the customer router receives via SLAAC on the Sonic side clearly can't be handed out to any local devices! You'd have a network with the router and nothing else on it!)

It wouldn't normally make any sense to try to resell a delegated prefix, anyway, since it would be too small to carve up further. This is really a non-issue.

In any event, Sonic does, fortunately support prefix delegation. But the delegated prefix changes at the drop of a hat, in clear conflict with Best Current Operational Practice 690. I strongly encourage the Sonic network engineering team to read that document.

I must stress: It's really important that the delegated IPv6 prefix not change. This is called "Flash Renumbering," and it seriously breaks connectivity for the local hosts on the network. There are a lot of documents on the web describing the problem, and there is really no good solution for it. It just breaks things. For instance, see Is your ISP constantly changing the delegated IPv6 prefix on your CPE/router? (My first Google search hit just now.) Their suggested customer-side mitigation of reducing the preferred lifetime in RAs to 45 minutes in not tenable, however, since most devices will ignore RAs with lifetimes under two hours (and it violates RFC4862). And even if it did work, you've only reduced the outage to 45 minutes.

Do you represent Sonic? I would like this message to reach the attention of the network engineering team, so I can find out when this problem can be addressed, and if there are any mitigations available in the short term.

If there is no immediate mitigation, yes, this is definitely a dealbreaker. I need my connectivity to be stable. My wife frequently works from home, and relies on the network to stay up. I'm sure a lot of Sonic's other customers also expect stable service.

[kgc]

We agree that your delegation should not be changing when you do a lease renewal and further, it should not behave that way now. What are you using as CPE and is it possible to run a packet capture on it? We'd like to see the packet for the renewal and discovery process.

[gadams]

It's great to hear that this is unexpected!

My CPE is a VyOS router. It is running dhcp6c under the hood to request the delegated prefix. Here's the config:

Code: Select all

# man https://www.unix.com/man-page/debian/5/dhcp6c.conf/
interface eth9 {
    send client-id 00:03:00:01:20:7c:14:f5:90:51;
    request domain-name-servers;
    request domain-name;
    send ia-na 0; # non-temporary address
    send ia-pd 1; # prefix delegation #1
    script "/etc/wide-dhcpv6/dhcp6c.eth9.script";
};

id-assoc na 0 {
    # Identity association for non temporary address
};

id-assoc pd 1 {
    prefix ::/56 infinity;
    prefix-interface eth6.10 {
        sla-len 8;
        sla-id 10;
        ifid 1;
    };
    prefix-interface eth6.20 {
        sla-len 8;
        sla-id 11;
        ifid 1;
    };
};

Here is a packet capture from a manual lease renewal (which includes a lease release, naturally). Note that before the lease renewal, I had been delegated 2001:5a8:42a2:2b00::/56. After the renewal, I was delegated 2001:5a8:42a1:6300::/56. Note that there is a valid client DUID in the request.

When I return home I can capture the packets from an event like unplugging and re-connecting the ONT<->router ethernet. That will likely not include a lease release, but I expect the result to be the same: yet another new prefix.

Thank you for taking a look!

[kgc]

Lease release/renew is not expected to behave the same way as simple renewal and would very likely result in new v4/v6 addresses being delegated.

[gadams]

I'll upload a packet capture from a disconnect/reconnect event this evening.

But consider that any time the CPE router reboots, the release/request cycle will happen, and then receiving a different delegation will disrupt all the local hosts. Surely this can't be expected.

Interestingly, I've been given the same IPv4 address for the last three or four release/request cycles. That would be hardly any disruption at all, because my router NATs all the IPv4 client traffic anyway, but your DHCP server is somehow able to offer the same address again, at least sometimes.

Again, it's really important for the IPv6 delegation not to change after things as simple as a CPE router reboot. Is there no expectation that Sonic will observe BCOP 690?

[gadams]

I have tested unplugging the ethernet cable between the ONT and my router and reconnecting it, and I have also tested power-cycling the ONT. In these cases, my router requests a specific address for the upstream link and the delegated prefix, and the D?HcPv6 server leases them to my router, presumably because my router already had an active lease for them:

Code: Select all

22:48:21.850907 IP6 (flowlabel 0x4c860, hlim 1, next-header UDP (17) payload length: 114) fe80::227c:14ff:fef5:9059.dhcpv6-client > ff02::1:2.dhcpv6-server: [bad udp cksum 0xc4d4 -> 0x9cf2!] dhcp6 renew (xid=9f37b5 (client-ID hwaddr type 1 207c14f59051) (server-ID vid 0000058362343a66) (IA_NA IAID:0 T1:10800 T2:17280 (IA_ADDR 2001:5a8:601:30::9:39c pltime:21600 vltime:21600)) (elapsed-time 0) (option-request DNS-server DNS-search-list))
22:48:21.851258 IP6 (flowlabel 0x4c860, hlim 1, next-header UDP (17) payload length: 115) fe80::227c:14ff:fef5:9059.dhcpv6-client > ff02::1:2.dhcpv6-server: [bad udp cksum 0xc4d5 -> 0xe6ad!] dhcp6 renew (xid=1790a0 (client-ID hwaddr type 1 207c14f59051) (server-ID vid 0000058362343a66) (elapsed-time 0) (option-request DNS-server DNS-search-list) (IA_PD IAID:1 T1:10800 T2:17280 (IA_PD-prefix 2001:5a8:42ab:c800::/56 pltime:21600 vltime:21600)))
22:48:21.951017 IP6 (class 0xc0, hlim 64, next-header UDP (17) payload length: 136) fe80::b6f9:5dff:fe35:2e3c.dhcpv6-server > fe80::227c:14ff:fef5:9059.dhcpv6-client: [udp sum ok] dhcp6 reply (xid=9f37b5 (client-ID hwaddr type 1 207c14f59051) (server-ID vid 0000058362343a66) (IA_NA IAID:0 T1:10800 T2:17280 (IA_ADDR 2001:5a8:601:30::9:39c pltime:21600 vltime:21600)) (DNS-server ns1.sonic.net ns2.sonic.net))
22:48:32.654488 IP6 (flowlabel 0x4c860, hlim 1, next-header UDP (17) payload length: 115) fe80::227c:14ff:fef5:9059.dhcpv6-client > ff02::1:2.dhcpv6-server: [bad udp cksum 0xc4d5 -> 0xe275!] dhcp6 renew (xid=1790a0 (client-ID hwaddr type 1 207c14f59051) (server-ID vid 0000058362343a66) (elapsed-time 1080) (option-request DNS-server DNS-search-list) (IA_PD IAID:1 T1:10800 T2:17280 (IA_PD-prefix 2001:5a8:42ab:c800::/56 pltime:21600 vltime:21600)))
22:48:32.763020 IP6 (class 0xc0, hlim 64, next-header UDP (17) payload length: 137) fe80::b6f9:5dff:fe35:2e3c.dhcpv6-server > fe80::227c:14ff:fef5:9059.dhcpv6-client: [udp sum ok] dhcp6 reply (xid=1790a0 (client-ID hwaddr type 1 207c14f59051) (server-ID vid 0000058362343a66) (IA_PD IAID:1 T1:10800 T2:17280 (IA_PD-prefix 2001:5a8:42ab:c800::/56 pltime:21600 vltime:21600)) (DNS-server ns1.sonic.net ns2.sonic.net))

So, that's good.

But normally, when a router reboots, it'll release the lease first, or, if it unexpectedly loses power, it'll start from scratch not requesting a particular prefix. (FWIW, the IPv4 side seems to do somewhat better here. It more often than not seems to hand back the same address even after a release.)

So, I suspect based on what you said above, that this is how you expect the Sonic DHCPv6 server to behave. I'll configure my router not to release the leases on configuration changes or reboots, and see how long I can go without outages. This may at least be a substantial improvement.

I'll point out that this is definitely what BCOP 690 warns against, and I think Sonic can do better here. For example, AT&T, on their fiber product, gives back the same prefix to the same customer even when the customer's router releases the lease. I'm somewhat shocked to say it, but that appears to be a service differentiation in their favor! I have never had this sort of outage there, and I can even populate my local DNS with sensible values, and my DNS and file servers can be reachable.

Thanks for your attention.

[matt.kinkele]

At this time this is the expected behavior. Sending release removes your lease from the local dhcp server and subsequent requests start from a blank slate. This implementation was designed exclusively for dynamic leases at a time with different best practices.

In practice we don't typically see people rolling their PD leases simply from rebooting. Presumably most RGs don't send release when gracefully reloaded, or alternately these devices aren't ever truly reloaded gracefully - just yanked from power, leaving their lease alive and waiting for them when they come back online.

Enabling greater persistence is unfortunately not a switch we can flip in the current design, and the best way to do so may involve adopting a vastly different dhcp server architecture, which is something we're evaluating for many reasons reasons, this included.

In short, I hear your complaints and agree with them, and this feature is one of many we're considering as we evaluate future iteration of our network. Sane v6 ranks rather highly for me personally, and I'd like to thank you for speaking about it as feedback really does help us pick our priorities.

[justin.miller]

I'm just chiming in to say thanks Matt for advocating for sane v6 and to add my name to the list of people who want this. Also thanks gadams for posting the thread as I'm also trying to get this set up currently.