Dynamic Multipoint VPNs…
- June 12th, 2010
- Write comment
Recently, the company I work for had a need to provide site to site VPN functionality. A traditional GRE IPsec or LAN to LAN direct encapsulation IPsec tunnel (like the one I recently wrote about) worked to support one of our remote sites for a while. However, the need to support several sites, and more in the future, became the new challenge. Of course, LAN to LAN VPNs will work but it is an administrative burden from a scalability perspective. So, how do you design an infrastructure that will grow easily without building all those unique tunnels? In short: DMVPN.
Dynamic Multipoint VPNs are the next step in the evolution of site-to-site VPNs. It is not a protocol per se, but more of a solution. It leverages IPsec, multipoint GRE (mGRE), and Next Hop Resolution Protocol (NHRP) to deliver a highly scalable and secure VPN solution. One of my favorite features of DMVPN is the ability for remote site routers to create on demand IPsec tunnels if a host needs to communicate with another remote site even if all remote sites have dynamically assigned public addresses. So, let’s take a look at how to set it up. For this example, I’m using Cisco 3725 routers running IOS 12.4T code. Although I’m using all RFC 1918 addressing, anything in 10.0.0.0/8 is considered public and anything in 172.16.0.0/12 is considered non-routable. Here is the network topology:
DMVPN Topology
So, we have two remote sites (A and B) and a hub router that could be located at the corporate office. Both remote routers are dynamically assigned public addresses by the ISP and the hub router has a statically assigned public address. Have a look at the basic configuration of Site A before we start the DMVPN configuration:
SiteA>en SiteA#sh run int f0/0 Building configuration... Current configuration : 100 bytes ! interface FastEthernet0/0 description Internet WAN ip address dhcp speed 100 full-duplex end SiteA#
Nothing special. The only thing to note is the “ip address dhcp” command. You should also note that no static route has been configured since the router will automatically get the gateway of last resort from the DHCP server:
SiteA#sh ip route Codes: C - connected, S - static, R - RIP, M - mobile, B - BGP D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2 E1 - OSPF external type 1, E2 - OSPF external type 2 i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2 ia - IS-IS inter area, * - candidate default, U - per-user static route o - ODR, P - periodic downloaded static route Gateway of last resort is 10.0.0.1 to network 0.0.0.0 10.0.0.0/24 is subnetted, 1 subnets C 10.0.0.0 is directly connected, FastEthernet0/0 S* 0.0.0.0/0 [254/0] via 10.0.0.1 SiteA#
Now we begin the DMVPN configuration. Since DMVPN does utilize IPsec, I like to configure all the cryptography first. It will look similar to the “crypto maps” I talked about in my last L2L VPN post, but slightly different. So lets take a look at the configuration:
SiteA#sh run | b crypto isakmp crypto isakmp policy 10 encr 3des authentication pre-share group 2 crypto isakmp key ISAKMP-KEY address 0.0.0.0 0.0.0.0 ! ! crypto ipsec transform-set DMVPN esp-3des esp-sha-hmac mode transport ! crypto ipsec profile DMVPN set transform-set DMVPN set pfs group2 !
Just like a static VPN tunnel, we setup an ISAKMP policy and define the attributes. The attributes used for this configuration can be modified. For example, if you wanted to use AES encryption and group 5 attributes you can do so. Also note that we set the ISAKMP key to be used for any address. This will allow for the setup of dynamic IPsec tunnels because spoke routers will not know the address of the peer. We also set up our IPsec transform in similar fashion to a L2L VPN tunnel but what is different is the use of a IPsec profile instead of a crypto map. Once our mGRE with NHRP tunnel is configured, we’ll apply this profile to the tunnel interface. So let’s setup the mGRE/NHRP tunnel:
SiteA#sh run int tun 0 Building configuration... Current configuration : 409 bytes ! interface Tunnel0 bandwidth 1000 ip address 172.16.0.2 255.255.255.0 no ip redirects ip nhrp authentication NHRP-KEY ip nhrp map 172.16.0.1 10.0.1.2 ip nhrp map multicast 10.0.1.2 ip nhrp network-id 100 ip nhrp nhs 172.16.0.1 delay 1000 shutdown tunnel source FastEthernet0/0 tunnel mode gre multipoint tunnel key 1 end SiteA#
Ok, lots to look at here. I’ve created interface Tunnel 0 and gave it an IP address. Notice it’s a /24 and not a /30. I’ve also configured the tunnel source interface, but in this case, no destination. Remember this is a mGRE tunnel and not point to point. The “tunnel mode gre multipoint” command makes it such. There are a lot of “ip nhrp” commands on this interface and each one is important. The first line adds a bit of security with the key “NHRP-KEY.” The second line “maps” the public (in this example) to the private tunnel interface of the hub router at the corporate office. Multicast packets, such as the ones kicked out by EIGRP, would automatically be sent across a point to point GRE tunnel. Since that is no longer the case, multicast packets must be sent to a specific destination address. The “ip map multicast” servers this purpose. The last two lines define the NHRP network and the IP address of the next hop server(router). The next hop server holds the mappings of public-to-private address in the NHRP network. At this point we only need to add our IPsec profile to the tunnel and “no shut” the interface and verify the tunnel came up:
SiteA#config t
Enter configuration commands, one per line. End with CNTL/Z.
SiteA(config)#int tun 0
SiteA(config-if)#tunnel protection ipsec profile DMVPN
SiteA(config-if)#no shut
SiteA(config-if)#end
SiteA#sh ip nhrp
172.16.0.1/32 via 172.16.0.1, Tunnel0 created 00:22:52, never expire
Type: static, Flags: used
NBMA address: 10.0.1.2
SiteA#sh dmvpn
Legend: Attrb --> S - Static, D - Dynamic, I - Incompletea
N - NATed, L - Local, X - No Socket
# Ent --> Number of NHRP entries with same NBMA peer
Tunnel0, Type:Spoke, NHRP Peers:1,
# Ent Peer NBMA Addr Peer Tunnel Add State UpDn Tm Attrb
----- --------------- --------------- ----- -------- -----
1 10.0.1.2 172.16.0.1 UP 00:13:59 S
SiteA#sh tunnel endpoints
Tunnel0 running in multi-GRE/IP mode
Endpoint transport 10.0.1.2 Refcount 2 Base 0x67326DA4
overlay 172.16.0.1 Refcount 2 Parent 0x67326DA4
SiteA#sh ip eigrp neighbors
IP-EIGRP neighbors for process 1
H Address Interface Hold Uptime SRTT RTO Q Seq
(sec) (ms) Cnt Num
0 172.16.0.1 Tu0 10 00:16:19 1318 5000 0 115
SiteA#sh ip route
Codes: C - connected, S - static, R - RIP, M - mobile, B - BGP
D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2
i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
ia - IS-IS inter area, * - candidate default, U - per-user static route
o - ODR, P - periodic downloaded static route
Gateway of last resort is 10.0.0.1 to network 0.0.0.0
172.16.0.0/16 is variably subnetted, 3 subnets, 2 masks
D 172.16.1.1/32 [90/2944000] via 172.16.0.1, 00:18:23, Tunnel0
C 172.16.0.0/24 is directly connected, Tunnel0
C 172.16.2.1/32 is directly connected, Loopback0
10.0.0.0/24 is subnetted, 1 subnets
C 10.0.0.0 is directly connected, FastEthernet0/0
S* 0.0.0.0/0 [254/0] via 10.0.0.1
SiteA#sh crypto isakmp sa
IPv4 Crypto ISAKMP SA
dst src state conn-id slot status
10.0.1.2 10.0.0.4 QM_IDLE 1003 0 ACTIVE
IPv6 Crypto ISAKMP SA
SiteA#sh crypto ipsec sa
interface: Tunnel0
Crypto map tag: Tunnel0-head-0, local addr 10.0.0.4
protected vrf: (none)
local ident (addr/mask/prot/port): (10.0.0.4/255.255.255.255/47/0)
remote ident (addr/mask/prot/port): (10.0.1.2/255.255.255.255/47/0)
current_peer 10.0.1.2 port 500
PERMIT, flags={origin_is_acl,}
#pkts encaps: 277, #pkts encrypt: 277, #pkts digest: 277
#pkts decaps: 277, #pkts decrypt: 277, #pkts verify: 277
#pkts compressed: 0, #pkts decompressed: 0
#pkts not compressed: 0, #pkts compr. failed: 0
#pkts not decompressed: 0, #pkts decompress failed: 0
#send errors 9, #recv errors 0
local crypto endpt.: 10.0.0.4, remote crypto endpt.: 10.0.1.2
path mtu 1500, ip mtu 1500, ip mtu idb FastEthernet0/0
current outbound spi: 0xF54A9D77(4115307895)
inbound esp sas:
spi: 0xD7FCE35F(3623674719)
transform: esp-3des esp-sha-hmac ,
in use settings ={Transport, }
conn id: 5, flow_id: SW:5, crypto map: Tunnel0-head-0
sa timing: remaining key lifetime (k/sec): (4551688/2361)
IV size: 8 bytes
replay detection support: Y
Status: ACTIVE
inbound ah sas:
inbound pcp sas:
outbound esp sas:
spi: 0xF54A9D77(4115307895)
transform: esp-3des esp-sha-hmac ,
in use settings ={Transport, }
conn id: 6, flow_id: SW:6, crypto map: Tunnel0-head-0
sa timing: remaining key lifetime (k/sec): (4551688/2361)
IV size: 8 bytes
replay detection support: Y
Status: ACTIVE
outbound ah sas:
outbound pcp sas:
SiteA#
Given the above output, we can tell the configuration worked. In short they were “sh ip nhrp,” “sh dmvpn,” “sh tunnel endpoints,” sh ip eigrp neighbors,” sh ip route,” “sh crypto isakmp sa,” and “sh crypto ipsec sa.” These verify we have an active DMVPN configuration. What I’ll show now is the hub router configuration and the configuration of EIGRP:
Hub#sh run Building configuration... Current configuration : 1602 bytes ! version 12.4 service timestamps debug datetime msec service timestamps log datetime msec no service password-encryption ! hostname Hub ! ! Omitted for brevity ! crypto isakmp policy 10 encr 3des authentication pre-share group 2 crypto isakmp key ISAKMP-KEY address 0.0.0.0 0.0.0.0 ! ! crypto ipsec transform-set DMVPN esp-3des esp-sha-hmac mode transport ! crypto ipsec profile DMVPN set transform-set DMVPN set pfs group2 ! ! ! ! ! ! ! ! interface Loopback0 ip address 172.16.1.1 255.255.255.255 ! interface Tunnel0 bandwidth 1000 ip address 172.16.0.1 255.255.255.0 no ip redirects no ip next-hop-self eigrp 1 ip nhrp authentication NHRP-KEY ip nhrp map multicast dynamic ip nhrp network-id 100 no ip split-horizon eigrp 1 delay 1000 tunnel source FastEthernet0/0 tunnel mode gre multipoint tunnel key 1 tunnel protection ipsec profile DMVPN ! interface FastEthernet0/0 ip address 10.0.1.2 255.255.255.252 speed 100 full-duplex ! ! Omitted for brevity ! router eigrp 1 passive-interface default no passive-interface Tunnel0 network 172.16.0.0 0.0.0.255 network 172.16.1.1 0.0.0.0 no auto-summary ! ip forward-protocol nd ip route 0.0.0.0 0.0.0.0 10.0.1.1 ! !
There are a couple things to note about the hub router configuration. First is the “no ip split-horizon eigrp 1″ command on the Tunnel 0 interface. This allows EIGRP to advertise routes learned from one spoke router and share it with other spoke routers. This configuration is only needed on the hub router. Another is the “ip nhrp map multicast dynamic” command. This command allows the hub router to automatically add spoke routers to NHRP when they form the mGRE IPsec tunnels. This command is also only required on the hub router. The last important configuration command is “no ip next-hop-self eigrp 1″ command. This instructs EIGRP to use the orginal next hop router learned and not the hub router. With out this command, dynamic spoke to spoke tunnels will not form because they will route traffic through the hub router. The last set of configurations I want to share is the addition of another spoke and the demonstration of a dynamic spoke to spoke tunnel:
Hub#sh dmvpn Legend: Attrb --> S - Static, D - Dynamic, I - Incompletea N - NATed, L - Local, X - No Socket # Ent --> Number of NHRP entries with same NBMA peer Tunnel0, Type:Hub, NHRP Peers:2, # Ent Peer NBMA Addr Peer Tunnel Add State UpDn Tm Attrb ----- --------------- --------------- ----- -------- ----- 1 10.0.0.4 172.16.0.2 UP never D 1 10.0.0.6 172.16.0.3 UP never D Hub# SiteA#sh dmvpn Legend: Attrb --> S - Static, D - Dynamic, I - Incompletea N - NATed, L - Local, X - No Socket # Ent --> Number of NHRP entries with same NBMA peer Tunnel0, Type:Spoke, NHRP Peers:1, # Ent Peer NBMA Addr Peer Tunnel Add State UpDn Tm Attrb ----- --------------- --------------- ----- -------- ----- 1 10.0.1.2 172.16.0.1 UP 00:54:51 S SiteA# SiteB#sh dm SiteB#sh dmvpn Legend: Attrb --> S - Static, D - Dynamic, I - Incompletea N - NATed, L - Local, X - No Socket # Ent --> Number of NHRP entries with same NBMA peer Tunnel0, Type:Spoke, NHRP Peers:1, # Ent Peer NBMA Addr Peer Tunnel Add State UpDn Tm Attrb ----- --------------- --------------- ----- -------- ----- 1 10.0.1.2 172.16.0.1 UP 00:00:52 S SiteB#
As you can see from the “sh dmvpn” commands on all routers above, there are two static (indicated by a “S” on the spoke routers) to the hub router. The hub router “sh dmvpn” command shows them as two dynamic tunnels (indicated by a “D”). Also notice that there is no tunnel between spokes. This is now going to change:
SiteA#ping 172.16.3.1 Type escape sequence to abort. Sending 5, 100-byte ICMP Echos to 172.16.3.1, timeout is 2 seconds: !.!!! Success rate is 80 percent (4/5), round-trip min/avg/max = 228/284/344 ms SiteA#sh dmvpn Legend: Attrb --> S - Static, D - Dynamic, I - Incompletea N - NATed, L - Local, X - No Socket # Ent --> Number of NHRP entries with same NBMA peer Tunnel0, Type:Spoke, NHRP Peers:1, # Ent Peer NBMA Addr Peer Tunnel Add State UpDn Tm Attrb ----- --------------- --------------- ----- -------- ----- 2 10.0.1.2 172.16.0.1 UP 00:57:44 S SiteA# SiteB#sh dmvpn Legend: Attrb --> S - Static, D - Dynamic, I - Incompletea N - NATed, L - Local, X - No Socket # Ent --> Number of NHRP entries with same NBMA peer Tunnel0, Type:Spoke, NHRP Peers:2, # Ent Peer NBMA Addr Peer Tunnel Add State UpDn Tm Attrb ----- --------------- --------------- ----- -------- ----- 1 10.0.1.2 172.16.0.1 UP 00:03:36 S 1 10.0.0.4 172.16.0.2 UP never D SiteB#
Looking at the “sh dmvpn” output on the Site B router after a ping was initiated from the Site A router, we can now see the a dynamic tunnel was formed between site A and B. Pretty cool, huh? Also note that the dynamic tunnel is only shown on the router that did not initiate the VPN tunnel. So now, if we wanted to deploy an army of routers, all we would have to do is change the mGRE tunnel ip address to something note used int the 172.16.0.0/24 address space and copy and paste the configuration from another spoke router. For those interested, here is the full configuration of the spoke router (the configuration for the hub is above):
SiteA#sh run Building configuration... Current configuration : 1576 bytes ! version 12.4 ! hostname SiteA ! ! ! Omitted for brevity ! ! crypto isakmp policy 10 encr 3des authentication pre-share group 2 crypto isakmp key ISAKMP-KEY address 0.0.0.0 0.0.0.0 ! ! crypto ipsec transform-set DMVPN esp-3des esp-sha-hmac mode transport ! crypto ipsec profile DMVPN set transform-set DMVPN set pfs group2 ! ! ! interface Loopback0 ip address 172.16.2.1 255.255.255.255 ! interface Tunnel0 bandwidth 1000 ip address 172.16.0.2 255.255.255.0 no ip redirects ip nhrp authentication NHRP-KEY ip nhrp map 172.16.0.1 10.0.1.2 ip nhrp map multicast 10.0.1.2 ip nhrp network-id 100 ip nhrp nhs 172.16.0.1 delay 1000 tunnel source FastEthernet0/0 tunnel mode gre multipoint tunnel key 1 tunnel protection ipsec profile DMVPN ! interface FastEthernet0/0 description Internet WAN ip address dhcp speed 100 full-duplex ! router eigrp 1 passive-interface default no passive-interface Tunnel0 network 172.16.0.0 0.0.0.255 network 172.16.2.1 0.0.0.0 no auto-summary ! ! Omitted for brevity ! end
That’s all there is to it. If you have GNS3 and a legal copy of 12.4T for Cisco’s 3725 router, you can try it out yourself. Of course you could always buy four 3725s (yes, another 3725 acted as the “Internet” in this example) but, it may be expensive! If you have any questions or comments, leave a comment below. Enjoy!
-Tim
