OpenVPN Service with the EdgeRouter X / EdgeRouter Lite

Advanced feature discussion, beta programs and unsupported "Labs" features.
10 posts Page 1 of 1
by forest » Thu Mar 24, 2016 3:45 pm
Following up on my post in the OpenVPN Service topic:

I now have FTTN, and have done some tests with OpenVPN running on both the EdgeRouter X and the EdgeRouter Lite. Both these wired-only routers are made by Ubiquiti, cost under $100, and include an OpenVPN client that can be configured through the command line.

I would recommend these routers only for people who are either good with command line network configuration, or who have someone else to deal with such things. Although Ubiquiti's web UI does have setup wizards that can make these devices work like consumer-grade routers, the wizards don't cover OpenVPN or the firewall rules that would prevent VPN leaks. On the plus side, the command line tools and configuration structure are quite sensible to someone who knows networking, the support community is pretty good, and common operations like port forwarding and checking connectivity can be done from the web UI. After getting mine configured the way I wanted it, I haven't really had to touch it.

Device specs:
https://wikidevi.com/wiki/Ubiquiti_Netw ... geRouter_X
https://wikidevi.com/wiki/Ubiquiti_Netw ... outer_LITE

The EdgeRouter Lite has 3 ethernet ports, a 500 MHz dual-core CPU, and can offload some routing functions to hardware, making it capable of gigabit speeds in certain configurations. Unfortunately, OpenVPN makes the hardare offloading features mostly useless.

The EdgeRouter X is about half the size and price of the Lite, has 5 ethernet ports, and an 880 MHz dual-core dual-threaded CPU. Although the processor has some hardware offloading capabilities, they are not used by EdgeOS 1.8 (the current version). This makes it slower than an offload-enabled EdgeRouter Lite for basic routing, but faster for things like OpenVPN and QoS.

I tested both routers using Sonic's production OpenVPN server through my 20 Mbps FTTN line, against both https://speedof.me/ and ftp://ftp.sonic.net/pub/testfile.compressed.100meg (with curl). I had a few basic firewall, NAT, and port forwarding rules enabled, all typical of consumer internet setups.

EdgeRouter Lite results:
speedof.me: 9.24Mbps
ftp.sonic.net: 11.66Mbps
openvpn CPU load: 85-96%

EdgeRouter X results:
speedof.me: 18.19Mbps
ftp.sonic.net: 18.61Mbps
openvpn CPU load: 71-83%

The EdgeRouter Lite handled OpenVPN up to about half the download rate of Sonic's FTTN x1 service. In other words, too slow. A better approach for this device would be to make use of its IPSEC VPN acceleration, as Dane mentioned here:
viewtopic.php?p=20083#p20083

The EdgeRouter X handled OpenVPN at full FTTN x1 speed. With a $50 price tag, an outstanding feature set, and very modest power consumption, that makes it a winner for me.

Judging by the CPU usage I saw, I'm guessing that today's EdgeRouter X would be unable to run OpenVPN at full FTTN x2 speed. However, this might change with future OS updates. The OpenVPN client is currently a single-threaded application, which leaves most of the device's CPU cores sitting idle even while the process hits a CPU limit. A multi-threaded implementation is already on the project roadmap, and depending on the design, could raise OpenVPN's effective bandwidth limit on this device. Also, the MediaTek MT7621AT SoC used in the EdgeRouter X reportedly has hardware acceleration features, including the AES ciphers used in OpenVPN, that simply aren't exposed by the OS yet. If Ubiquiti were to get the right drivers and use them in an EdgeOS update (much like they did to enable hardware offload on the EdgeRouter Lite) this little box would become even more capable.

https://community.openvpn.net/openvpn/w ... #Threading
https://forum.mqmaker.com/t/is-crypto-e ... are/241/16
https://community.ubnt.com/t5/EdgeMAX-F ... -p/1469737

Most of the time I spent setting up my device was about learning EdgeOS configuration options and testing for firewall/VPN leaks. The .ovpn file generated by Sonic's server was usable almost as-is; I just had to copy it to the router, tell EdgeOS where to find it, and set the auth-user-pass line to the path of a username/password file I created. I also ended up making a simple cron job to restart the openvpn process if it ever dies, such as after a power failure or when an FTTN outage lasts more than a few minutes.

It's worth noting that the EdgeRouter X has been noticed by the OpenWRT community, and some very early porting work has already begun. That could eventually turn into a good OS alternative.
by clamothe » Wed May 04, 2016 3:45 pm
I just spoke with tech support and they said IPSec should be available, to look at the wiki for info. That said, they might be uninformed.

This page specifies IPsec VPN configuration.

I also have an EdgeRouter Lite. With FTTNx2 service, the speeds you were seeing for this router and OpenVPN are insufficient. This router has hardware acceleration for IPsec though, so I would like to try it.

Has anyone attempted this?
by forest » Wed May 04, 2016 3:57 pm
If you're referring to Sonic's old Cisco VPN server, good luck. I tried it with an EdgeRouter Lite, but didn't find settings in the EdgeOS web interface that would get it to work. Perhaps it's possible using the command line and config files?

To be fair, I didn't try very hard, since several Sonic users (including myself) have experienced a high rate of unexplained session failures with the Cisco VPN server. Even if I got an EdgeRouter to talk to it, a VPN service that drops connections all the time wouldn't be useful to me.
by clamothe » Wed May 04, 2016 5:45 pm
Good to know. Given that, I will stick with OpenVPN on my machine until my router has hardware acceleration for OpenVPN. I read somewhere that the EdgeRouter Lite technically has the hardware to accelerate this but the OS / drivers don't utilize it fully.
by faisal » Sun May 15, 2016 11:22 pm
Forest -- would you be willing to share your configuration, or describe what you did to get things set up?
by forest » Mon May 16, 2016 1:30 pm
faisal wrote:Forest -- would you be willing to share your configuration, or describe what you did to get things set up?
My EdgeRouter configuration is about 500 lines, mostly VPN rules and stuff that's specific to my environment. It's easy enough to share the VPN pieces, though.

WARNING:
This will cause your EdgeRouter to tunnel through your internet gateway and acquire an IP address that is exposed directly to the internet, bypassing any protection provided by your gateway's firewall and NAT services. If you plan to do this, you better learn how to set up a firewall (or at least NAT with selective port forwarding) on the EdgeRouter, lest you become yet another victim of the many automated attacks that are constantly running on the internet. I will not teach you how, since it would take more time than I have to spare, but Ubiquiti's forum and support site are good resources, as is the Vyatta 6.3 documentation. The Ubiquiti enthusiasts on reddit can also be helpful. In addition to blocking unwanted incoming traffic, I recommend you configure your firewall to block non-VPN outboud traffic on the WAN interface, for privacy.

Installing your OpenVPN configuration:
  • Have Sonic's VPN server generate a client.ovpn file
  • Save that file someplace sensible on the EdgeRouter; for example: /config/vpn/vpn-client.ovpn
  • Edit the file, appending a password file path (which you will create shortly) to the auth-user-pass line. For example: auth-user-pass /config/vpn/vpn-client.pass
  • Create the password file. The first line should contain your main Sonic username. The second line should contain your main Sonic password. Set permissions on this file appropriately.
Telling EdgeOS to use your OpenVPN configuration:

Code: Select all

configure
set system static-host-mapping host-name ovpn.sonic.net inet 209.148.113.36
set interfaces openvpn vtun0 config-file /config/vpn/vpn-client.ovpn
set interfaces openvpn vtun0 description 'Sonic.net VPN'
commit
save
I think the EdgeRouter started the openvpn client as soon as I ran those commands, but I don't remember for sure. You can watch what it's doing with the "show log tail" command, and keep an eye on the network interfaces with the web UI.
by forest » Mon May 16, 2016 2:42 pm
Here's a set of EdgeOS configuration-mode commands that should bring an EdgeRouter X from its stock setup to a home internet router that tunnels all traffic through Sonic's OpenVPN service. These are excerpts from a config that I used when evaluating hardware a few months ago; I have not tested them as-is.

Note:
I post this reluctantly, because I really don't have the free time for troubleshooting or tutoring, but I still want to encourage people to learn and and benefit from the work I've already done. With that in mind, please educate yourself and use Ubiquiti's support channels if there's something you don't understand. The Vyatta 6.3 documentation covers most of these commands, and can easily be found with a web search. As much as I like to be helpful, I will not teach you networking or show you how to customize this for your own needs. I will answer educated questions about the approach I used, though, and correct errors if you find any.

This design is intended to fail closed, meaning that the LAN will lose internet access if the VPN goes down, which is better than leaking data. Ethernet ports 1-4 will act as a switch for your LAN devices. Port 0 will become the WAN port, for connecting to one of your internet gateway's "LAN" ports. This means that if you're logged into the EdgeRouter via port 0, you will lose your login session when you commit this configuration. If that happens, you'll have to log back in using another port in order to save the configuration and do any tidying up. If something else goes wrong, you might end up firewalled out of your EdgeRouter on all ports, so be sure you know how to do a factory reset before you try any of this. Also, it's possible that none of this will work until you put your internet gateway into bridge mode, DMZ-Plus mode, or something of that kind.

In the commands below, you'll want to replace /config/vpn/client.ovpn with whatever path you use for your OpenVPN config file, and replace 192.168.1.1 with whatever IP address your internet gateway uses on its LAN port. (For example, some AT&T/Pace gateways use 192.168.1.254.)

Code: Select all

set firewall all-ping enable
set firewall broadcast-ping disable
set firewall group address-group VPN-SERVERS address 209.148.113.36
set firewall ipv6-receive-redirects disable
set firewall ipv6-src-route disable
set firewall ip-src-route disable
set firewall log-martians enable
set firewall name LAN-TO-ROUTER default-action accept
set firewall name LAN-TO-VPN default-action accept
set firewall name LAN-TO-WAN default-action reject
set firewall name ROUTER-TO-LAN default-action accept
set firewall name ROUTER-TO-VPN default-action accept
set firewall name ROUTER-TO-WAN default-action reject
set firewall name ROUTER-TO-WAN rule 10 action accept
set firewall name ROUTER-TO-WAN rule 10 description 'VPN tunnel from router'
set firewall name ROUTER-TO-WAN rule 10 destination group address-group VPN-SERVERS
set firewall name ROUTER-TO-WAN rule 10 protocol tcp_udp
set firewall name ROUTER-TO-WAN rule 20 action accept
set firewall name ROUTER-TO-WAN rule 20 description 'Unicast DHCP renew to WAN Gateway'
set firewall name ROUTER-TO-WAN rule 20 destination address 192.168.1.1
set firewall name ROUTER-TO-WAN rule 20 destination port bootps
set firewall name ROUTER-TO-WAN rule 20 protocol udp
set firewall name ROUTER-TO-WAN rule 20 source port bootpc
set firewall name VPN-TO-LAN default-action drop
set firewall name VPN-TO-LAN rule 10 action accept
set firewall name VPN-TO-LAN rule 10 description 'Established/related traffic'
set firewall name VPN-TO-LAN rule 10 state established enable
set firewall name VPN-TO-LAN rule 10 state related enable
set firewall name VPN-TO-ROUTER default-action drop
set firewall name VPN-TO-ROUTER rule 10 action accept
set firewall name VPN-TO-ROUTER rule 10 description 'Established/related traffic'
set firewall name VPN-TO-ROUTER rule 10 state established enable
set firewall name VPN-TO-ROUTER rule 10 state related enable
set firewall name VPN-TO-ROUTER rule 20 action drop
set firewall name VPN-TO-ROUTER rule 20 description 'Invalid state (explicit; skip remaining rules)'
set firewall name VPN-TO-ROUTER rule 20 state invalid enable
set firewall name VPN-TO-WAN default-action drop
set firewall name WAN-TO-LAN default-action drop
set firewall name WAN-TO-LAN rule 10 action accept
set firewall name WAN-TO-LAN rule 10 description 'Established/related traffic'
set firewall name WAN-TO-LAN rule 10 state established enable
set firewall name WAN-TO-LAN rule 10 state related enable
set firewall name WAN-TO-ROUTER default-action drop
set firewall name WAN-TO-ROUTER rule 10 action accept
set firewall name WAN-TO-ROUTER rule 10 description 'Established/related traffic'
set firewall name WAN-TO-ROUTER rule 10 state established enable
set firewall name WAN-TO-ROUTER rule 10 state related enable
set firewall name WAN-TO-ROUTER rule 20 action drop
set firewall name WAN-TO-ROUTER rule 20 description 'Invalid state (explicit; skip remaining rules)'
set firewall name WAN-TO-ROUTER rule 20 state invalid enable
set firewall name WAN-TO-VPN default-action drop
set firewall receive-redirects disable
set firewall send-redirects enable
set firewall source-validation loose
set firewall syn-cookies enable
set interfaces ethernet eth0 address dhcp
set interfaces ethernet eth0 description Internet
set interfaces ethernet eth0 dhcp-options default-route update
set interfaces ethernet eth0 dhcp-options default-route-distance 210
set interfaces ethernet eth0 dhcp-options name-server no-update
set interfaces ethernet eth0 duplex auto
set interfaces ethernet eth0 speed auto
set interfaces ethernet eth1 description Local
set interfaces ethernet eth1 duplex auto
set interfaces ethernet eth1 speed auto
set interfaces ethernet eth2 description Local
set interfaces ethernet eth2 duplex auto
set interfaces ethernet eth2 speed auto
set interfaces ethernet eth3 description Local
set interfaces ethernet eth3 duplex auto
set interfaces ethernet eth3 speed auto
set interfaces ethernet eth4 description Local
set interfaces ethernet eth4 duplex auto
set interfaces ethernet eth4 speed auto
set interfaces loopback lo
set interfaces openvpn vtun0 config-file /config/vpn/client.ovpn
set interfaces openvpn vtun0 description 'Sonic.net VPN'
set interfaces switch switch0 address 10.0.0.1/24
set interfaces switch switch0 description Local
set interfaces switch switch0 mtu 1500
set interfaces switch switch0 switch-port interface eth1
set interfaces switch switch0 switch-port interface eth2
set interfaces switch switch0 switch-port interface eth3
set interfaces switch switch0 switch-port interface eth4
set port-forward auto-firewall enable
set port-forward hairpin-nat enable
set port-forward lan-interface switch0
set port-forward wan-interface vtun0
set protocols static interface-route 192.168.1.1/32 next-hop-interface eth0 description 'WAN Gateway'
set protocols static interface-route 192.168.1.1/32 next-hop-interface eth0 distance 1
comment protocols static interface-route 192.168.1.1/32 'Route to WAN gateway private address (for DHCP renew)'
set service dhcp-server disabled false
set service dhcp-server hostfile-update enable
set service dhcp-server shared-network-name LAN-DHCP-SERVICE authoritative enable
set service dhcp-server shared-network-name LAN-DHCP-SERVICE subnet 10.0.0.0/24 default-router 10.0.0.1
set service dhcp-server shared-network-name LAN-DHCP-SERVICE subnet 10.0.0.0/24 dns-server 10.0.0.1
set service dhcp-server shared-network-name LAN-DHCP-SERVICE subnet 10.0.0.0/24 lease 86400
set service dhcp-server shared-network-name LAN-DHCP-SERVICE subnet 10.0.0.0/24 start 10.0.0.100 stop 10.0.0.199
set service dns forwarding cache-size 150
set service dns forwarding listen-on switch0
set service dns forwarding name-server 208.201.224.11
set service dns forwarding name-server 208.201.224.33
set service gui https-port 443
set service nat rule 5010 description 'Masquerade behind VPN address'
set service nat rule 5010 outbound-interface vtun0
set service nat rule 5010 type masquerade
set service ssh port 22
set service ssh protocol-version v2
set system conntrack tcp loose disable
set system domain-name local
comment system domain-name 'Override any resolv.conf domain assigned by the WAN DHCP server'
set system host-name edgerouter
set system name-server 127.0.0.1
set system ntp server 0.ubnt.pool.ntp.org
set system ntp server 1.ubnt.pool.ntp.org
set system ntp server 2.ubnt.pool.ntp.org
set system ntp server 3.ubnt.pool.ntp.org
set system static-host-mapping host-name ovpn.sonic.net inet 209.148.113.36
set system time-zone America/Los_Angeles
set zone-policy zone LAN default-action drop
set zone-policy zone LAN from ROUTER firewall name ROUTER-TO-LAN
set zone-policy zone LAN from VPN firewall name VPN-TO-LAN
set zone-policy zone LAN from WAN firewall name WAN-TO-LAN
set zone-policy zone LAN interface eth1
set zone-policy zone LAN interface eth2
set zone-policy zone LAN interface eth3
set zone-policy zone LAN interface eth4
set zone-policy zone LAN interface switch0
set zone-policy zone ROUTER default-action drop
set zone-policy zone ROUTER description 'Router-specific (not forwarded) traffic'
set zone-policy zone ROUTER from LAN firewall name LAN-TO-ROUTER
set zone-policy zone ROUTER from VPN firewall name VPN-TO-ROUTER
set zone-policy zone ROUTER from WAN firewall name WAN-TO-ROUTER
set zone-policy zone ROUTER local-zone
set zone-policy zone VPN default-action drop
set zone-policy zone VPN from LAN firewall name LAN-TO-VPN
set zone-policy zone VPN from ROUTER firewall name ROUTER-TO-VPN
set zone-policy zone VPN from WAN firewall name WAN-TO-VPN
set zone-policy zone VPN interface vtun0
set zone-policy zone WAN default-action drop
set zone-policy zone WAN from LAN firewall name LAN-TO-WAN
set zone-policy zone WAN from ROUTER firewall name ROUTER-TO-WAN
set zone-policy zone WAN from VPN firewall name VPN-TO-WAN
set zone-policy zone WAN interface eth0
by faisal » Sun Oct 02, 2016 8:29 am
Here's a set of EdgeOS configuration-mode commands that should bring an EdgeRouter X from its stock setup to a home internet router that tunnels all traffic through Sonic's OpenVPN service.
Thank you for posting this.

With mild modification, this gets an ER Lite up and running, downloading at about 8-10 mbps with a pegged CPU.

With similar modification, an ER Pro downloads at around 22-24 mbps with a CPU that's barely moving. The CPU utilization makes me wonder if it's hitting limits on Sonic's side.

Has anyone seen faster download rates with OpenVPN over Fusion FTTN? I'd be interested in your setup.
by Guest » Sun Oct 02, 2016 12:46 pm
faisal wrote:With mild modification, this gets an ER Lite up and running, downloading at about 8-10 mbps with a pegged CPU.

With similar modification, an ER Pro downloads at around 22-24 mbps with a CPU that's barely moving. The CPU utilization makes me wonder if it's hitting limits on Sonic's side.
This is just telling you there's hardware acceleration on the Pro and none on the Lite. Nothing wrong with the Sonic side. You're on FTTN X1, right? I thought all EdgeRouters don't have acceleration on OpenVPN but guess not.
by faisal » Sun Oct 02, 2016 2:32 pm
Guest wrote:This is just telling you there's hardware acceleration on the Pro and none on the Lite. Nothing wrong with the Sonic side. You're on FTTN X1, right? I thought all EdgeRouters don't have acceleration on OpenVPN but guess not.
Without the VPN, I'm regularly getting 45-50 Mbps downloads, so I believe something about the VPN tunnel is slowing things down. I don't expect either is offloading, and the Pro just has a beefier CPU. How would I find out for sure?
10 posts Page 1 of 1

Who is online

In total there are 28 users online :: 0 registered, 0 hidden and 28 guests (based on users active over the past 5 minutes)
Most users ever online was 999 on Mon May 10, 2021 1:02 am

Users browsing this forum: No registered users and 28 guests