A little more info:
Last night there was a blackout here the entire evening. PG&E got the lights restored around 1 am. While that was going on, I was able to plug my old phone in and get the dial tone.
Just now, I plugged the line back into my cordless phone, but the phone seems to not be able to sense anything coming from the line. I replugged the corded phone back in, but no dial tone comes out now.
Obviously I'm still connected to the internet, so cool. I brought my corded phone over to my router, unplugged the cable from the back of the router and stuck it in the corded phone. I got a dial tone, but I can't actually dial anything from that phone. Sigh.