VyOS is a free Linux-based distribution that can turn any regular computer or virtual machine into a fully-featured router. By combining existing open source software into a single image, and tying it together with a single configuration file, upgrade mechanism, and an automation API, it makes the job of network engineers and sysadmins easy.
It’s been a few years since I last used it, after determining it was not good enough for me, but the progress they’ve made in this time frame is unbelievable, and it’s practically a different thing by now. I plan to blog about VyOS more soon, and this transformation that happened, but for now I’d like to go over a setup that I did.
We’re going to be using a VyOS router that will connect in an ISP on eth0, it
will receive an IPv4 and an IPv6 address using PPPoE, it will request a /56
IPv6 prefix using DHCPv6-PD, and it will then create two VLANs on eth1, one
of which will be IPv6-First, while the second will be IPv6-only.
This can be a home, office, or event / conference network. As long as we have clients that want to reach the entire Internet, this should all work. This setup can also occur when you have static addresses, and you don’t need PPPoE or DHCPv6-PD.
Because this is 2024, we want to make this an IPv6-First installation, and not a dual-stack one.
The first thing we need to do is to configure PPPoE to receive our IPv6 and IPv4 addresses, and upstream connectivity:
set interfaces pppoe pppoe0 authentication username admin
set interfaces pppoe pppoe0 authentication password admin
set interfaces pppoe pppoe0 source-interface eth0
Of course, change the username and password, and review other options under
pppoe0 such as description, no-peer-dns, ip adjust-mss clamp-mss-to-pmtu, ipv6 adjust-mss clamp-mss-to-pmtu, etc. In some cases you
may also need ipv6 address autoconf.
We’re now going to create our two VLANs in eth1, which will be 10 and 20.
We will add the IPv4 to one of them, which will be IPv6-mostly, which will come
from the RFC1918 IP space:
set interfaces ethernet eth1 vif 10 description "IPv6-First"
set interfaces ethernet eth1 vif 10 address 192.168.1.1/24
set interfaces ethernet eth1 vif 20 description "IPv6-Only"
We now need to request some IPv6 LANs (/64 networks) from the ISP. This is
done using DHCPv6-PD. Most ISPs will give you a /56, while some will even
hand out a /48 network to you. These contain 256 LANs and 65,536 LANs
respectively. Unfortunately, there are some networks that may give you less,
e.g. a /58 or /60. Figure out how much your ISP is handing out because it
will be needed. In almost any case, if you request more, you’ll get the
maximum, but it’s better to be exact here.
Request a network from the pppoe0 interface like this, and then assign two
LANs to the two VLANs created above:
set interfaces pppoe pppoe0 dhcpv6-options pd 1 length 56
set interfaces pppoe pppoe0 dhcpv6-options pd 1 interface eth1.10 address 1
set interfaces pppoe pppoe0 dhcpv6-options pd 1 interface eth1.10 sla-id 0
set interfaces pppoe pppoe0 dhcpv6-options pd 1 interface eth1.20 address 1
set interfaces pppoe pppoe0 dhcpv6-options pd 1 interface eth1.20 sla-id 1
In the first line, we are requesting a /56 network. If your ISP supports a
different number, adjust this accordingly.
In the next set of lines, we configure VLAN 10, where we ask the router to
receive address ::1 of this /64, and use the 0th network for this.
If DHCPv6-PD gives us 2a0d:3dc0:1337::/56, then our VLANs will look like this:
eth1.10 - 2a0d:3dc0:1337:0::1/64
eth1.20 - 2a0d:3dc0:1337:1::1/64
The address setting can be 1 to 2^64-1, but usually a router has ::1. I
don’t know why, but VyOS does not support 0, which would be the real first
address of the LAN. Because some devices do not support 0, it’s a bad idea to
use that anyways though.
The sla-id therefore depends on how many LANs (or /64s) you got. If you get
a /56 then it’s 0-255, and so on.
Because our IPv4 address is dynamic, over PPPoE (or DHCP), we can’t configure a static IPv4 for NAT64. We can work around this problem by assigning a loopback private IP:
set interfaces loopback lo address 192.168.192.168/32
Now we can configure NAT64:
set nat64 source rule 10 source prefix 64:ff9b::/96
set nat64 source rule 10 translation pool 1 address 192.168.192.168
set nat64 source rule 10 translation pool 1 port 1500-65000
We are telling VyOS to use the standard NAT64 prefix, and then map all
connections to IPv4 hosts to address 192.168.192.168 using ports 1500 -
65500. If you are using an IPv4 VyOS uses to communicate to the Internet,
such as for example a public PPPoE IPv4 that is static, you need to leave some
ports for that. To figure out which ports VyOS may use, run:
cat /proc/sys/net/ipv4/ip_local_port_range
If you are using a loopback IPv4 address that is private, NAT44 will take
care of everything for you automatically.
If you have a very large number of hosts that are very active, you may need more than one IPv4 address. In that case, in the second command, you can also use a prefix:
set nat64 source rule 10 translation pool 1 address 193.5.16.0/29
This, however, requires you to somehow have a few IPv4 addresses on your router (such as when you receive a /29 from your ISP).
We now need to enable Router Advertisements (RAs), so all the clients within the two VLANs know that IPv6 is available, and they can configure it.
We’re now going to set up a DNS server locally, on the router. VyOS ships with PowerDNS and it can either resolve all domains locally, on the machine, or it can forward the queries to a third-party resolver, such as Google DNS or Cloudflare.
Before we do that though, since all IPv6 addresses we receive are dynamic, we’ll need a static address to use as our DNS IP address.
IPv6 has a huge range that’s equivalent to RFC1918, called
ULA. Practically all
addresses that start with fdXX:: are available, so we can assign one of them
to our loopback interface. Since it’s common to have two DNS addresses, we
can add another one, just for fun:
set interfaces loopback lo address fd53::53/128
set interfaces loopback lo address fd64::64/128
We can now enable the local DNS resolver:
set service dns forwarding dnssec validate
set service dns forwarding ignore-hosts-file
set service dns forwarding dns64-prefix 64:ff9b::/96
set service dns forwarding listen-address fd53::53
set service dns forwarding listen-address fd64::64
set service dns forwarding listen-address 192.168.1.1
You can optionally adjust the cache-size if you’d like VyOS to cache more
than the current default of 10,000 DNS records locally.
If you want to forward DNS requests to a remote DNS resolver, instead of doing full recursion, you also need to configure that:
set service dns forwarding name-server 2001:4860:4860::8888
set service dns forwarding name-server 2606:4700:4700::1111
set service dns forwarding name-server 8.8.8.8
set service dns forwarding name-server 1.1.1.1
TIP: If, for some reason, you don’t want to have to operate a DNS server,
Google offers DNS64 resolvers: 2001:4860:4860::64 and 2001:4860:4860::6464.
Now that we have NAT64 and DNS64 working, it’s time to add IPv6 to our clients. We will be using SLAAC, which is the preferred method for assigning IPv6 addresses to clients:
set service router-advert interface eth1.10 name-server fd53::53
set service router-advert interface eth1.10 name-server fd64::64
set service router-advert interface eth1.10 prefix ::/64
set service router-advert interface eth1.10 nat64prefix 64:ff9b::/96
set service router-advert interface eth1.20 name-server fd53::53
set service router-advert interface eth1.20 name-server fd64::64
set service router-advert interface eth1.20 prefix ::/64
set service router-advert interface eth1.20 nat64prefix 64:ff9b::/96
Here we are enabling Router Advertisements for the two VLANs. We are handing
out the two local DNS servers we set up earlier, we use ::/64 for the prefix
as it’s dynamically assigned, and we deploy PREF64 using the nat64prefix
option.
With the RA configuration above, all IPv6 clients should turn on their CLATs if they have them. In case they don’t, DNS64 will take care of the rest.
Here’s a couple of options that may be interesting or useful:
set service router-advert interface eth1.10 link-mtu 1492
set service router-advert interface eth1.10 default-lifetime 300
The first one will ask all IPv6 hosts to set their MTU to 1492, since we are
using PPPoE, and this is what pppoe0 has. Even if we don’t do this, it’s
fine, as we’ve configured MSS clamping on that interface.
The second option makes the lifetime of these announcements last 5 minutes. In
that case, if DHCPv6-PD ever gives us a different prefix, and we have to
renumber everything, some devices just may be down for up to 5 minutes.
In practice, DHCPv6-PD changing the prefix is not a common occurrence, so this
is probably fine too for most places. You can even set that to 3600 and keep
it if you don’t experience any issues.
It’s now time to add the dreaded NAT44. To do this, we’re going to masquerade
anything going out of pppoe0:
set nat source rule 10 outbound-interface name pppoe0
set nat source rule 10 translation address masquerade
Finally, for VLAN 10, our IPv6-first LAN, we will add the DHCPv4 server,
handing out IPv4 addresses, but also informing clients it is okay to be
IPv6-only:
set service dhcp-server shared-network-name LocalNet authoritative
set service dhcp-server shared-network-name LocalNet option default-router 192.168.1.1
set service dhcp-server shared-network-name LocalNet subnet 192.168.1.0/24 lease 3600
set service dhcp-server shared-network-name LocalNet subnet 192.168.1.0/24 option name-server 192.168.1.1
set service dhcp-server shared-network-name LocalNet subnet 192.168.1.0/24 range all start 192.168.1.5
set service dhcp-server shared-network-name LocalNet subnet 192.168.1.0/24 range all stop 192.168.1.200
set service dhcp-server shared-network-name LocalNet subnet 192.168.1.0/24 subnet-id 10
set service dhcp-server shared-network-name LocalNet subnet 192.168.1.0/24 option ipv6-only-preferred 86400
It is this last line that turns on “Option 108” or “IPv6-Only Preferred”. With that, our DHCP server will not assign an IPv4 if the device claims to be IPv6-only capable. All Apple devices, latest Androids, and Windows 11, most notably, will now completely turn off their IPv4 network stack!
Since with IPv6 all addresses used are public, you may want to add a stateful firewall that will not allow incoming connections to your devices, unless they ask for them. There’s a traditional way to do it with VyOS, but I’ll be showing off a feature I am excited about, which is global (non-interface-tied) firewall rules:
set firewall group interface-group stateful-fw interface eth1.10
set firewall group interface-group stateful-fw interface eth1.20
set firewall ipv6 forward filter rule 10 action accept
set firewall ipv6 forward filter rule 10 outbound-interface group stateful-fw
set firewall ipv6 forward filter rule 10 state established
set firewall ipv6 forward filter rule 10 state related
set firewall ipv6 forward filter rule 20 action accept
set firewall ipv6 forward filter rule 20 outbound-interface group stateful-fw
set firewall ipv6 forward filter rule 20 protocol ipv6-icmp
set firewall ipv6 forward filter rule 99 action reject
set firewall ipv6 forward filter rule 99 outbound-interface group stateful-fw
We first create a group of interfaces we will apply these rules towards. This
includes the two VLANs we have on eth1.
We then add 3 rules, one allowing only established and related connections
(incoming flow only if outgoing communication was initiated), ICMP (e.g. ping),
and finally a rule to block all other traffic.
At this point, you may ask: well, what’s the point of having public addresses if we block all incoming traffic anyways? This breaks the end-to-end principle!
Not really: punching a hole through a stateful firewall is much easier, faster, and safer than punching a hole through NAT. It’s so easy it’s almost guaranteed to work, even in the real world. You don’t need external services to tell you which port you’re using, you’re not going to lose that port after some time or under mysterious circumstances, it’s just a couple of packets and you’re accepting peer to peer traffic!
Similarly, one can create the IPv4 firewall that VLAN 10 needs. Since VLAN
20 is IPv6-only, we don’t need duplicate work here :)
We now have a VyOS router that establishes a PPPoE connection on one port, and then serves two LANs on another port. We run a local DNS resolver that optionally forwards requests to a third party, and it assists with IPv6 usage. Both LANs have IPv6, and it’s either required or encouraged to use that instead of IPv4. Connectivity works end-to-end from IPv6 to both IPv6 as well as IPv4 hosts.
If you have any feedback, suggestions, or issues you ran into, feel free to contact me, as I’d like to improve this guide! If you want to understand what’s happening better, make sure to read my previous blog post on what’s happening here behind the scenes.