Homelab · Networking

pfSense Site-to-Site WireGuard VPN: Connecting Multi-Site Enterprise Labs with Routed Subnets

A comprehensive architectural guide to configuring a high-performance WireGuard site-to-site tunnel between pfSense firewalls, establishing routed transit networks, cryptokey routing, and firewall filtering for multi-site enterprise labs.

By K Shankar R Karanth Homelab Concept Guide Tested on: pfSense Plus 24.03 / CE 2.7.2, WireGuard kmod 0.1.6, Proxmox VE 8.2
Quick idea (BLUF): WireGuard site-to-site tunnels on pfSense replace bloated IPsec and OpenVPN daemons with high-throughput in-kernel cryptokey routing over UDP 51820. Establishing seamless LAN-to-LAN routing between multi-site enterprise labs requires four tightly coupled elements: matching tunnel transfer addresses (10.99.0.0/30), explicit peer Allowed IPs covering remote subnets, an assigned interface tab for stateful reply-to filtering, and an interface MTU clamped to 1420 to prevent packet fragmentation.
Cryptokey Routing

Allowed IPs handles both ingress access filtering and egress kernel route table lookups.

Kernel-Level Noise

Zero user-space context switches; runs in FreeBSD kernel with ChaCha20-Poly1305 and Curve25519.

Interface Binding

Assigning tun_wg0 enables native pfSense gateway health monitoring, state tracking, and reply-to firewall filtering.

The Multi-Site Enterprise Lab Dilemma

Standing up an authentic enterprise testbed rarely stops at a single hypervisor host. Once you introduce offsite disaster recovery, remote Proxmox replication targets, branch office simulations, or hybrid cloud endpoints, you face a familiar infrastructure hurdle: connecting geographically separated subnets without introducing massive operational friction.

For decades, enterprise networking defaulted to IPsec IKEv2 or OpenVPN tunnels. While both protocols remain reliable, they extract a noticeable toll in homelabs and branch environments. IPsec demands complex Phase 1 and Phase 2 proposal negotiations, brittle dead-peer detection timers, and multi-layered crypto suites. OpenVPN, running primarily as a user-space daemon on FreeBSD, introduces substantial CPU overhead and context-switching penalties that throttle throughput on compact, low-power firewall appliances.

WireGuard fundamentally changes this operational calculation. Implemented as an in-kernel module in FreeBSD (via wireguard-kmod on pfSense Plus and Community Edition), WireGuard operates with a minimalist codebase of under 4,000 lines. It strips away cipher negotiation in favour of state-of-the-art primitives: Curve25519 for key exchange, ChaCha20-Poly1305 for authenticated encryption, and BLAKE2s for hashing. The result is instant connection establishment, silent idle states, and near-line-rate forwarding.

Homelab context: Running a multi-site tunnel between a primary lab on Proxmox VE and a remote disaster recovery site allows services like Active Directory Domain Services (AD DS) multi-master replication, Proxmox Backup Server sync jobs, and Ceph external monitors to communicate as if they sat on adjacent rack switches.

The Pneumatic Tube Analogy

To master WireGuard routing, you must discard the mental models formed by traditional VPN tunnels.

Think of traditional IPsec or OpenVPN as a formal international border crossing. Every vehicle must stop at the checkpoint. Border guards inspect documentation, verify security certificates, negotiate which stamp to apply, issue temporary transit permits, and conduct periodic identity checks. If a packet arrives with slightly mismatched phase parameters, the entire vehicle is impounded while renegotiation takes place.

WireGuard operates like an encrypted pneumatic dispatch tube bolted directly between two secure vaults.

There are no guards, no visa applications, and no handshake negotiation ceremonies. Each vault door has an unalterable cryptographic seal (the public key) and a strictly verified routing manifest (Allowed IPs). When a packet enters the tube at Site A, the sending mechanism encrypts it using the destination vault’s badge and fires it across the wire. If the receiving vault at Site B validates the seal and verifies that the originating IP address is on the approved manifest, the canister opens instantly. If the badge or IP does not match, the packet is discarded silently into the void without generating an error response.

Key principle: WireGuard is fundamentally stateless from a connection perspective. It does not maintain a persistent session state machine. If an upstream ISP connection drops for five minutes, the tunnel does not “fail over” or require renegotiation; the moment packets resume flowing, encryption resumes instantaneously.

Architectural Topology & Addressing Plan

To configure routed site-to-site tunnels without encountering address conflicts or asymmetric routing loops, you must establish a clear IP addressing schema. Our lab environment connects two physical sites hosting enterprise workloads:

+-----------------------------------+             +-----------------------------------+
|          PRIMARY LAB (SITE A)     |             |         DR / BRANCH (SITE B)      |
|                                   |             |                                   |
| LAN: 192.168.10.0/24              |             | LAN: 192.168.20.0/24              |
| DC01.corp.karanth.ovh (10.10)     |             | DC02.corp.karanth.ovh (20.11)     |
| Proxmox Cluster Node 1            |             | Proxmox DR Node                   |
|                                   |             |                                   |
| pfSense Gateway: 192.168.10.1     |             | pfSense Gateway: 192.168.20.1     |
| Public WAN: 203.0.113.10          |             | Public WAN: 198.51.100.20         |
| Tunnel Interface: 10.99.0.1/30    |             | Tunnel Interface: 10.99.0.2/30    |
+-----------------+-----------------+             +-----------------+-----------------+
                  |                                                 |
                  |         WireGuard Tunnel (UDP 51820)            |
                  +=================================================+
                               Transfer Net: 10.99.0.0/30

The transit network connecting the two pfSense firewalls uses a dedicated /30 point-to-point subnet: 10.99.0.0/30. Site A assumes 10.99.0.1, and Site B assumes 10.99.0.2. This point-to-point network exists solely to provide gateway endpoints for static routing.

Segment Subnet Gateway IP Purpose & Workloads
Site A LAN 192.168.10.0/24 192.168.10.1 Primary lab hosting DC01 (192.168.10.10) and Proxmox VE compute cluster.
Site B LAN 192.168.20.0/24 192.168.20.1 Secondary DR lab hosting DC02 (192.168.20.11) and offsite storage.
Tunnel Transfer 10.99.0.0/30 Point-to-Point Dedicated transit interconnect across the public internet between firewalls.

Cryptokey Routing: How Allowed IPs Actually Works

The most common point of failure for engineers deploying WireGuard is misunderstanding the Allowed IPs parameter. In traditional routing, route tables dictate where packets go, and firewall rules dictate whether they are allowed. In WireGuard, Allowed IPs performs both roles simultaneously via a mechanism called Cryptokey Routing.

When you define Allowed IPs on a peer entry, the WireGuard engine uses that list for two separate operational decisions:

1. Ingress Source Verification: When pfSense receives an encrypted packet over UDP 51820, it decrypts the packet using the peer’s public key. It then inspects the inner IPv4 header. If the source IP address does not match an entry in that peer’s Allowed IPs list, the packet is dropped immediately inside the kernel. It never reaches the pfSense packet filter (pf).

2. Egress Destination Routing: When pfSense decides to send a packet out of the WireGuard interface (tun_wg0), WireGuard inspects the destination IP address against the Allowed IPs table of all configured peers on that interface. It selects the peer whose Allowed IPs prefix matches the destination, encrypts the payload with that peer’s public key, and transmits it to that peer’s public endpoint.

Crucial configuration rule: Because Allowed IPs handles both ingress validation and egress routing, Site A’s peer definition for Site B must include both Site B’s tunnel address (10.99.0.2/32) and Site B’s routed local LAN subnet (192.168.20.0/24). If you only specify the tunnel address, Site A will silently drop every packet originating from Site B’s LAN hosts.

Required Ports, Protocols & Firewall Rules

Securing a site-to-site enterprise lab requires separating the public tunnel handshake from the internal routed traffic. The public WAN interface only needs a single UDP port open, while the virtual tunnel interface must permit the required enterprise protocols between subnets.

Port / Protocol Direction Interface Purpose & Service
UDP 51820 Inbound / Outbound WAN WireGuard tunnel transport, peer key handshakes, and encapsulated traffic.
TCP / UDP 53 Bi-directional WG_S2S (Assigned) Active Directory DNS resolution and cross-site conditional forwarding.
TCP / UDP 88 Bi-directional WG_S2S (Assigned) Kerberos ticket-granting service (KDC) authentication across sites.
TCP / UDP 389 Bi-directional WG_S2S (Assigned) LDAP directory access, DC locator pings, and schema queries.
TCP 636 Bi-directional WG_S2S (Assigned) Secure LDAP (LDAPS) over TLS for encrypted directory management.
TCP 445 Bi-directional WG_S2S (Assigned) SMB SYSVOL replication (DFSR) and inter-site administrative file shares.
TCP 135 Bi-directional WG_S2S (Assigned) RPC Endpoint Mapper for Active Directory inter-DC replication.
TCP 49152–65535 Bi-directional WG_S2S (Assigned) RPC Dynamic High Ports allocated for Active Directory directory replication.
ICMP (Type 3, 8) Bi-directional WG_S2S (Assigned) Echo request/reply and Destination Unreachable (Path MTU Discovery).

To test connectivity from a Windows management workstation or Domain Controller across the tunnel once rules are staged, use the one-line PowerShell firewall verification command:

# Test LDAP reachability to the remote Domain Controller across the WireGuard tunnel
Test-NetConnection -ComputerName 192.168.20.11 -Port 389

Step-by-Step Configuration: Building the Tunnel

Configuring a fully routed site-to-site WireGuard connection in pfSense involves five coordinated phases across both firewalls.

Phase 1: Package Installation & Key Generation

On both pfSense systems, open System > Package Manager > Available Packages. Search for wireguard and install the official package. This registers the kernel module and exposes the WireGuard configuration menu under VPN > WireGuard.

Before creating tunnels, generate a cryptographically strong Pre-shared Key (PSK). While WireGuard’s default asymmetric key exchange is exceptionally secure, adding a 256-bit symmetric pre-shared key provides quantum-resistant defence-in-depth. You can generate a PSK directly from the pfSense shell or command prompt:

# Generate a 256-bit base64 pre-shared key on pfSense
wg genpsk

Save this string securely; the identical pre-shared key will be entered into the peer configuration on both firewalls.

Phase 2: Configuring WireGuard Tunnels

Navigate to VPN > WireGuard > Tunnels on Site A and click Add Tunnel:

1. Description: S2S-SiteB-Primary
2. Listen Port: 51820
3. Interface Keys: Click Generate to produce the Private and Public keypair for Site A. Copy the Public Key to your notepad; Site B will require it.
4. Interface Addresses: Add 10.99.0.1 / 30.

Click Save Tunnel.

Repeat this procedure on Site B under VPN > WireGuard > Tunnels:

1. Description: S2S-SiteA-DR
2. Listen Port: 51820
3. Interface Keys: Click Generate. Copy Site B’s Public Key to your notepad.
4. Interface Addresses: Add 10.99.0.2 / 30.

Click Save Tunnel.

Phase 3: Defining Peers & Cryptokey Routes

Navigate to VPN > WireGuard > Peers on Site A and click Add Peer:

1. Tunnel: Select tun_wg0 (S2S-SiteB-Primary).
2. Description: Peer-SiteB-DR.
3. Dynamic: Uncheck. Enter Site B’s public WAN IP (198.51.100.20) or FQDN in Endpoint and port 51820.
4. Public Key: Paste Site B’s Public Key.
5. Pre-shared Key: Paste the shared PSK generated earlier.
6. Allowed IPs: Add two separate entries:
    • 10.99.0.2 / 32 (Site B tunnel interface)
    • 192.168.20.0 / 24 (Site B local LAN subnet)
7. Keepalive: If either site resides behind an upstream ISP router or Carrier-Grade NAT (CGNAT), set Keep Alive to 25 seconds to keep firewall state tables open.

On Site B, navigate to VPN > WireGuard > Peers and add the peer for Site A:

1. Tunnel: Select tun_wg0 (S2S-SiteA-DR).
2. Description: Peer-SiteA-Primary.
3. Dynamic: Uncheck. Enter Site A’s public WAN IP (203.0.113.10) and port 51820.
4. Public Key: Paste Site A’s Public Key.
5. Pre-shared Key: Paste the identical PSK.
6. Allowed IPs:
    • 10.99.0.1 / 32 (Site A tunnel interface)
    • 192.168.10.0 / 24 (Site A local LAN subnet)
7. Keepalive: Set to 25.

Click Save and Apply Changes on both firewalls.

Phase 4: Interface Assignment & Static Routing

By default, pfSense groups all WireGuard traffic under a generic “WireGuard” firewall tab. While convenient for road-warrior VPNs, this generic tab lacks reply-to tags and prevents native gateway health tracking. To establish enterprise routing, we must assign the tunnel to an explicit interface.

1. Navigate to Interfaces > Assignments on Site A.
2. In the Available network ports dropdown, select tun_wg0 and click Add.
3. Click the newly assigned interface (typically named OPT1).
4. Check Enable Interface.
5. Set Description to WG_S2S.
6. Leave IPv4 Configuration Type set to None (the tunnel configuration already binds 10.99.0.1 to tun_wg0).
7. Set MTU to 1420 (explained below).
8. Click Save and Apply Changes. Repeat this exact interface assignment on Site B.

Next, create the gateway. On Site A, navigate to System > Routing > Gateways and click Add:

• Interface: WG_S2S
• Gateway Name: WG_S2S_GW_SITEB
• Gateway IP: 10.99.0.2
• Monitor IP: 10.99.0.2

On Site B, create the gateway pointing back to Site A:

• Interface: WG_S2S
• Gateway Name: WG_S2S_GW_SITEA
• Gateway IP: 10.99.0.1
• Monitor IP: 10.99.0.1

Now create the static route. On Site A, navigate to System > Routing > Static Routes:

• Destination Network: 192.168.20.0 / 24
• Gateway: WG_S2S_GW_SITEB - 10.99.0.2
• Description: Route to Site B Lab Subnet

On Site B, add the reverse static route:

• Destination Network: 192.168.10.0 / 24
• Gateway: WG_S2S_GW_SITEA - 10.99.0.1
• Description: Route to Site A Lab Subnet

Phase 5: Firewall Rules & The reply-to Mechanism

Firewall rules must be created on two distinct interfaces on both firewalls:

1. WAN Interface: Under Firewall > Rules > WAN, add an inbound pass rule:
• Action: Pass
• Interface: WAN
• Address Family: IPv4
• Protocol: UDP
• Source: Remote Firewall WAN IP (198.51.100.20 on Site A; 203.0.113.10 on Site B)
• Destination: WAN address
• Destination Port Range: 51820

2. Assigned WG_S2S Interface: Under Firewall > Rules > WG_S2S, add rules permitting inter-lab traffic. For full lab integration, create a rule permitting all IPv4 traffic from the remote lab subnet:
• Action: Pass
• Interface: WG_S2S
• Address Family: IPv4
• Protocol: Any
• Source: 192.168.20.0/24 (on Site A) or 192.168.10.0/24 (on Site B)
• Destination: LAN net (or Any)

Technical scar (Why assigning the interface matters): In pfSense, rules placed on the generic “WireGuard” group tab do not apply FreeBSD Packet Filter reply-to tags. On multi-WAN systems or firewalls with non-default routing paths, return packets exiting across the tunnel may be misrouted out of the default internet WAN gateway rather than back through the tunnel, causing asymmetric routing drops. Assigning the interface generates dedicated per-interface rules that inject proper reply-to state definitions.

The MTU Black Hole: Clamping MSS and Avoiding Dropped Frames

You configure the tunnel, verify handshakes, and run an ICMP ping from Site A to Site B. The ping replies with 1 ms latency and 0% loss. You open a web interface or start an Active Directory replication sync, and the connection stalls indefinitely. The browser spins forever; SMB transfers freeze after copying a few kilobytes; Active Directory logs Event ID 1722 (RPC Server Unavailable).

You have fallen into the classic MTU Black Hole.

Standard Ethernet frames support a Maximum Transmission Unit (MTU) of 1500 bytes. When a host transmits a packet, WireGuard encapsulates that Layer 3 payload inside a standard UDP datagram. Encapsulation introduces protocol overhead:

• Outer IPv4 Header: 20 bytes
• UDP Transport Header: 8 bytes
• WireGuard Type & Authentication Tag: 32 bytes
• Total WireGuard IPv4 Overhead: 60 bytes (80 bytes if using IPv6)

If an internal server sends a maximum-sized 1500-byte TCP frame with the DF (Don’t Fragment) bit set, WireGuard adds 60 bytes of encapsulation, resulting in a 1560-byte UDP packet. The physical WAN interface cannot transmit a 1560-byte frame over standard internet connections. If intermediate routers drop the oversized packet and drop the ICMP Type 3 Code 4 (“Fragmentation Needed”) response, Path MTU Discovery (PMTUD) fails silently. ICMP ping packets succeed because they are only 64 or 84 bytes, while full TCP segments disappear.

The resolution: In pfSense, configure the assigned WG_S2S interface MTU to 1420 (1500 − 80 byte buffer for dual-stack IPv4/IPv6). If your WAN connection uses PPPoE (1492 MTU), clamp the interface MTU to 1412. Setting the MTU on the assigned interface forces FreeBSD’s TCP stack to clamp the Maximum Segment Size (MSS) during TCP SYN handshakes to exactly MTU − 40 (1380 bytes), eliminating fragmentation across the tunnel.

Enterprise Service Integration: Active Directory & DNS Across Sites

Once the routed WireGuard tunnel is established, we integrate it into the authentic enterprise lab topology: domain corp.karanth.ovh, featuring DC01.corp.karanth.ovh (192.168.10.10) at Site A and DC02.corp.karanth.ovh (192.168.20.11) at Site B.

Active Directory Sites and Subnets

Windows clients and servers must understand the physical WAN boundary between Site A and Site B. Without proper site topology, a workstation at Site B might authenticate against DC01 across the WireGuard tunnel instead of querying local DC02.

1. Open Active Directory Sites and Services (dssite.msc) on DC01.
2. Create two sites: SiteA-Primary and SiteB-DR.
3. Under Subnets, create two subnet objects and associate them with their respective sites:
    • 192.168.10.0/24 → SiteA-Primary
    • 192.168.20.0/24 → SiteB-DR
4. Move DC01 into SiteA-Primary and DC02 into SiteB-DR.
5. Under Inter-Site Transports > IP, configure a site link named SITELINK-WG containing both sites. Configure a replication cost of 50 and a replication schedule of 15 minutes.

DNS Resolution & Domain Overrides

For non-domain hosts (such as Proxmox hypervisor nodes, Linux storage appliances, and administrative workstations) to resolve resources across sites, configure Domain Overrides in the pfSense DNS Resolver (Unbound):

On Site A’s pfSense (Services > DNS Resolver > General Settings):
• Under Domain Overrides, add corp.karanth.ovh pointing to 192.168.10.10 and 192.168.20.11.
• Add reverse lookup zones for 20.168.192.in-addr.arpa pointing to 192.168.20.11.

On Site B’s pfSense, configure identical Domain Overrides pointing primary queries to 192.168.20.11 with fallback to 192.168.10.10.

Scripts & Diagnostic Commands

Use the following commands on the pfSense shell (FreeBSD) and Windows management endpoints to inspect tunnel state, test handshakes, trace routing tables, and verify Active Directory replication health.

# --- pfSense FreeBSD Console Diagnostic Commands ---

# Display active WireGuard tunnel state, latest handshakes, and transfer metrics
wg show

# Inspect the active running configuration for tun_wg0
wg showconf tun_wg0

# Check the kernel IPv4 routing table for static routes
netstat -rn -f inet | grep -E '192.168|10.99'

# Verify point-to-point ping across the tunnel transfer network
ping -S 10.99.0.1 -c 4 10.99.0.2

# Test Path MTU with Don't Fragment (-D) flag set to verify 1420 MTU limit (1392 + 28 bytes header = 1420)
ping -D -s 1392 10.99.0.2

# Trace the routed path from Site A LAN interface to Site B gateway
traceroute -s 192.168.10.1 192.168.20.1

# Inspect active Packet Filter (pf) rules applied to WireGuard interfaces
pfctl -sr | grep -i wg

# Monitor live WireGuard packets on the tunnel interface
tcpdump -ni tun_wg0 -vv

# --- Windows Server / Active Directory Verification Commands ---

# Verify TCP port reachability to remote Domain Controller over WireGuard
Test-NetConnection -ComputerName 192.168.20.11 -Port 389

# Verify Kerberos KDC reachability across the tunnel
Test-NetConnection -ComputerName 192.168.20.11 -Port 88

# Inspect Active Directory replication partner metadata from DC01
Get-ADReplicationPartnerMetadata -Target "DC01.corp.karanth.ovh" -Scope Server

# Force immediate replication across all directory partitions
repadmin /syncall /AdeP

# Verify replication health and directory consistency across all DCs
dcdiag /test:Replication /v
Healthy output: In wg show, look for latest handshake: X seconds ago (under 180 seconds) and positive values for transfer: X KiB received, Y KiB sent. If latest handshake shows “never” or exceeds 3 minutes, the tunnel is down.

WireGuard CLI & Handshake Status Cheat Sheet

The table below details key diagnostic fields returned by wg show and how to interpret their operational status.

Output Field Expected Value Diagnostic Meaning Status
latest handshake < 180 seconds ago Cryptographic key exchange completed successfully. Keys re-negotiate every 120 seconds under traffic. Healthy
latest handshake (none) / “never” No handshake received. Remote endpoint unreachable, UDP 51820 blocked, or public key mismatch. Failure
transfer: sent Incrementing bytes Local firewall is encrypting and routing packets toward the peer. Normal
transfer: received 0 B (with sent > 0) Oneway traffic. Remote peer is not replying, remote firewall is dropping packets, or Allowed IPs is wrong. Investigate
endpoint IP:Port (Resolved) Public socket of the remote peer. Dynamically updates if peer endpoint shifts behind NAT. Normal
allowed ips Tunnel IP + LAN Subnet Cryptokey routing prefixes accepted from and routed to this peer. Healthy

Troubleshooting Cheat Sheet

When connectivity across the site-to-site tunnel fails, use this structured symptom-to-resolution matrix to isolate the issue.

Symptom Likely Cause Fix
Handshake shows “never” or times out UDP 51820 blocked on WAN, public key mismatch, or wrong endpoint IP. Verify WAN firewall rules pass UDP 51820 to WAN Address. Re-copy and verify public keys on both sides.
Handshake succeeds, but cannot ping remote LAN Remote LAN subnet missing from peer’s Allowed IPs list on either firewall. Add the remote subnet (e.g. 192.168.20.0/24) to Allowed IPs under VPN > WireGuard > Peers.
Ping works, but SMB and AD replication hang MTU Black Hole. Large packets exceed tunnel MTU and PMTUD is blocked. Set MTU to 1420 on the assigned WG_S2S interface under Interfaces > WG_S2S.
Packets pass initially, then freeze after 30 seconds of idle State table timeout on upstream NAT router or CGNAT firewall. Configure PersistentKeepalive = 25 on the peer definition behind NAT.
Traffic drops intermittently on multi-WAN firewalls Missing reply-to tag causing asymmetric routing out default gateway. Assign tun_wg0 to an interface under Interfaces > Assignments and apply rules on the assigned tab.
Static routes fail to resolve next-hop Gateway configured on wrong interface or gateway IP outside tunnel subnet. Verify gateway IP is 10.99.0.2 (for Site A) bound strictly to the WG_S2S interface.
Domain Controllers fail to locate each other Missing DNS conditional forwarding or Domain Overrides. Configure Unbound Domain Overrides for corp.karanth.ovh pointing to remote DC IP addresses.

Final Thoughts

WireGuard represents a significant leap forward in site-to-site connectivity for enterprise homelabs and branch infrastructures. By replacing legacy cryptographic negotiations with clean in-kernel cryptokey routing, it delivers sub-millisecond connection recovery, minimal CPU overhead, and exceptional throughput.

The key to a successful deployment lies in adhering to core architectural principles: isolate your tunnel transit network to a dedicated /30 subnet, configure Allowed IPs to include both the tunnel peer and the routed LANs, assign the tunnel interface to preserve stateful reply-to packet filtering, and clamp interface MTU to 1420 to avoid the MTU black hole.

Key takeaway: If wg show displays an active handshake within the last 180 seconds and Test-NetConnection -ComputerName 192.168.20.11 -Port 389 confirms TCP port reachability, your routed site-to-site WireGuard tunnel is properly established and ready for enterprise workloads.
Next in this series

Next in our enterprise architecture series, we explore TrueNAS SCALE storage design: configuring ZFS storage pools, dataset record sizes, and NFS/iSCSI target architectures for multi-hypervisor Proxmox clusters.


Discover more from Karanth.ovh

Subscribe to get the latest posts sent to your email.