Pages

Friday, May 30, 2014

Checkpoint Cluster Member Down because interfaces show partially up

Symptoms:

One of cluster member shows problem. It always happened on standby member. If goes into deep, you will find some of cluster member interfaces are showing down or partially up, although physically interface is up and connected properly.

Log into command line on primary member:
[Expert@CP1]# cphaprob stat

Cluster Mode:   New High Availability (Active Up)
 with IGMP Membership

Number     Unique Address  Assigned Load   State    

1 (local)  1.1.1.1         100%            Active    
2          1.1.1.2         0%              Down      


On standby checkpoint member :
[Expert@CP2]# cphaprob stat

Cluster Mode:   New High Availability (Active Up)
 with IGMP Membership

Number     Unique Address  Assigned Load   State  

1          1.1.1.1         100%            Active  
2 (local)  1.1.1.2         0%              Down    

[Expert@CP2]# cphaprob -i list
Built-in Devices:

Device Name: Interface Active Check
Current state: problem

Device Name: HA Initialization
Current state: OK

Registered Devices:

Device Name: Synchronization
Registration number: 0
Timeout: none
Current state: OK
Time since last report: 93466.5 sec

Device Name: Filter
Registration number: 1
Timeout: none
Current state: OK
Time since last report: 93439.2 sec

Device Name: cphad
Registration number: 2
Timeout: 2 sec
Current state: OK
Time since last report: 0.2 sec

Device Name: fwd
Registration number: 3
Timeout: 2 sec
Current state: OK
Time since last report: 0.2 sec

[Expert@CP2]# cphaprob -a if
Required interfaces: 4
Required secured interfaces: 1

DMZ        UP                    non sync(non secured), multicast
Internal   Inbound: DOWN (10.9 secs)    Outbound: DOWN (88822.4 secs) non sync(non secured), multicast
Lan1       UP                    sync(secured), multicast
External   Inbound: DOWN (88822.4 secs) Outbound: DOWN (89001.8 secs) non sync(non secured), multicast

Virtual cluster interfaces: 3

DMZ             100.9.2.30    
Internal        100.9.40.1    
External        100.9.38.20

Solution: 

Change Cluster mode from Multicast mode to Broadcast mode. From command line, it is "cphaconf set_ccp broadcast". This change does not require system reboot or cpstop/cpstart. Also it can survive reboot.


[Expert@CP1]# cphaconf set_ccp broadcast
[Expert@CP1]# cphaprob -a if

Required interfaces: 4
Required secured interfaces: 1

DMZ        UP                    non sync(non secured), broadcast
Internal   UP                    non sync(non secured), broadcast
Lan1       UP                    sync(secured), broadcast
External   UP                    non sync(non secured), broadcast

Virtual cluster interfaces: 3

DMZ             10.99.2.30      
Internal        10.99.140.1      
External        10.99.138.20      

[Expert@CP1]# cphaprob stat
Cluster Mode:   New High Availability (Active Up)
 with IGMP Membership

Number     Unique Address  Assigned Load   State    

1 (local)  1.1.1.1         100%            Active    
2          1.1.1.2         0%              Standby  


[Expert@CP2]# cphaconf set_ccp broadcast
[Expert@CP2]# cphaprob stat
Cluster Mode:   New High Availability (Active Up)
 with IGMP Membership

Number     Unique Address  Assigned Load   State    

1          1.1.1.1         100%            Active    
2 (local)  1.1.1.2         0%              Standby  

[Expert@CP2]# cphaprob -a if
Required interfaces: 4
Required secured interfaces: 1

DMZ        UP                    non sync(non secured), broadcast
Internal   UP                    non sync(non secured), broadcast
Lan1       UP                    sync(secured), broadcast
External   UP                    non sync(non secured), broadcast

Virtual cluster interfaces: 3

DMZ             100.9.2.30      
Internal        100.9.40.1      
External        100.9.38.20      


Note: To softly switch cluster statue between cluster member, use this command "clusterXL_admin <up|down> [-p]"
[Expert@CP]# clusterXL_admin up
Setting member to normal operation ...
Member current state is Standby

Saturday, May 24, 2014

Free Dynamic DNS Service Site - http://duckdns.org/



Duck DNS is a free dynamic DNS hosted on Amazon EC2.



To get it working is quite easy. Add a sub domain and update your current ip address to it.

In above screenshot, I created 51sec.duckdns.org as a DNS to point to my home network ISP ip. 

On one of my home internal machines, just installed a client to update the ip on 51sec.duckdns.org this dns name dynamically. The DuckDNS settings is shown in below screenshot. Put into your sub domain name and token. After all done, just click ok button, then it will automatically update it every 5 minutes.


Wednesday, May 21, 2014

Cisco VPN LAB 1: Simple Easy VPN Example between Routers

VPN Lab Series:
Cisco VPN LAB 1 : Simple Easy VPN Example between Routers and Comparison with DMVPN
Cisco VPN LAB 2 : IPSec VPN Example Between Two ASA 8.4.2
Cisco VPN LAB 3 : EZ VPN Between ASA 8.4.2, IOS Router and EZVPN Client Software

Working on Easy VPN and DMVPN, I completed this first lab for EZVPN lab and also list some using resource at the bottom of this post.

1. Topology


It is still with my favorite IOU rack v3. The physical connection with IP addresses have been shown on the diagram.

R1 will be easy VPN server and R2 will be the client. This lab will show how to configure a basic easy vpn client / server set up. R3 and R6 will be only used to do test ping with only default route and interface ip configured on them.

2. Configuration: (Redundancy configuration have been omitted)




@EZVPN Server Configuration
R1#show run
Building configuration...

Current configuration : 2631 bytes
!
version 12.4
service timestamps debug datetime msec
service timestamps log datetime msec
no service password-encryption
!
hostname R1
!
boot-start-marker
boot-end-marker
!
security passwords min-length 1

!--- Enable Authentication, Authorizing and Accounting (AAA)
!--- for user authentication and group authorization.
aaa new-model
!--- Enable the AAA commands in order 
!--- to enable Xauth for user authentication.
aaa authorization network hw-client-groupname local 
!--- Enable the AAA commands
!--- in order to enable group authorization.
aaa authorization network groupauthor local 
!
!
!         
!
aaa session-id common
clock timezone CST 8
mmi polling-interval 60
no mmi auto-configure
no mmi pvc
mmi snmp-timeout 180
ip source-route
!
ip cef
no ip domain lookup
no ipv6 traffic interface-statistics
no ipv6 cef
!
multilink bundle-name authenticated

!--- Define the username and password to use for Xauth.
username cisco password 0 cisco123
!
redundancy

!--- Create an Internet Security Association and
!--- Key Management Protocol (ISAKMP) policy for Phase 1 negotiations.
crypto isakmp policy 1
 authentication pre-share
 group 2
crypto isakmp client configuration address-pool local dynpool
!
!--- Create a group with the pre-shared key for IKE authentication.
crypto isakmp client configuration group hw-client-groupname
 key hw-client-password

!--- Create the Phase 2 policy for actual data encryption.
crypto ipsec transform-set transform-1 esp-des esp-sha-hmac 
!
!--- Create a dynamic map and
!--- apply the transform set that was created earlier.
crypto dynamic-map dynmap 1
 set transform-set transform-1 
 reverse-route
!
!--- Create the actual crypto map,
!--- and apply the AAA lists that were created earlier.
!--- These commands associate the AAA commands to the crypto map.
crypto map dynmap isakmp authorization list hw-client-groupname
crypto map dynmap client configuration address respond
crypto map dynmap 1 ipsec-isakmp dynamic dynmap 
!
!--- Apply the crypto map on the interface where
!--- traffic leaves the router.
interface Ethernet0/0
 description connected to Internet
 ip address 20.20.20.2 255.255.255.0
 crypto map dynmap
!
interface Ethernet0/1
 no ip address
 shutdown 
!
interface Ethernet0/2
 ip address 30.30.30.1 255.255.255.0
!
!
ip forward-protocol nd
!
!         
no ip http server
no ip http secure-server
ip route 0.0.0.0 0.0.0.0 Ethernet0/0
!
!
control-plane
!
!
line con 0
 logging synchronous
line aux 0
line vty 0 4
!
exception data-corruption buffer truncate

end

@EZVPN Client Configuration
R2#sh run
Building configuration...

Current configuration : 2420 bytes
!
version 12.4
service timestamps debug datetime msec
service timestamps log datetime msec
no service password-encryption
!
hostname R2
!
boot-start-marker
boot-end-marker
!
security passwords min-length 1
!
aaa new-model
!
!
aaa authentication login userauthen local
aaa authorization network groupauthor local 
!
aaa session-id common
clock timezone CST 8
mmi polling-interval 60
no mmi auto-configure
no mmi pvc
mmi snmp-timeout 180
ip source-route
!
!
ip cef
no ip domain lookup
ip domain name cisco.com
no ipv6 traffic interface-statistics
no ipv6 cef
!
multilink bundle-name authenticated
!
username cisco password 0 cisco123
!
redundancy
!--- Set the parameters to connect to the 
!--- appropriate Easy VPN group on the Easy VPN server.
crypto ipsec client ezvpn hw-client
 connect auto
 group hw-client-groupname key hw-client-password
 mode client
 peer 20.20.20.2
 xauth userid mode interactive
!
!--- Use the crypto ipsec client ezvpn <name> command on the
!--- interface that connects to the Easy VPN server
!--- in order to complete the Easy VPN.
interface Ethernet0/0
 description INTERNET
 ip address 20.20.20.1 255.255.255.0
 crypto ipsec client ezvpn hw-client


!--- Define the inside interfaces that will access 
!--- and can be accessed via Easy VPN.
interface Ethernet0/1
 description LAN
 ip address 10.10.10.1 255.255.255.0
 crypto ipsec client ezvpn hw-client inside
!
interface Ethernet0/2
 no ip address
 shutdown
!
ip forward-protocol nd
!
!
no ip http server
no ip http secure-server
ip route 0.0.0.0 0.0.0.0 Ethernet0/0
!
control-plane
!
!
line con 0
 logging synchronous
line aux 0
line vty 0 4
!
exception data-corruption buffer truncate
end

3. Verify

R2#show crypto ipsec sa

interface: Ethernet0/0
    Crypto map tag: Ethernet0/0-head-0, local addr 20.20.20.1

   protected vrf: (none)
   local  ident (addr/mask/prot/port): (30.30.30.20/255.255.255.255/0/0)
   remote ident (addr/mask/prot/port): (0.0.0.0/0.0.0.0/0/0)
   current_peer 20.20.20.2 port 500
     PERMIT, flags={origin_is_acl,}
    #pkts encaps: 0, #pkts encrypt: 0, #pkts digest: 0
    #pkts decaps: 0, #pkts decrypt: 0, #pkts verify: 0
    #pkts compressed: 0, #pkts decompressed: 0
    #pkts not compressed: 0, #pkts compr. failed: 0
    #pkts not decompressed: 0, #pkts decompress failed: 0
    #send errors 0, #recv errors 0

     local crypto endpt.: 20.20.20.1, remote crypto endpt.: 20.20.20.2
     path mtu 1500, ip mtu 1500, ip mtu idb Ethernet0/0
     current outbound spi: 0x2B595786(727275398)
     PFS (Y/N): N, DH group: none

     inbound esp sas:
      spi: 0xD56D693B(3580717371)

R2#show crypto ipsec client ezvpn 
Easy VPN Remote Phase: 8

Tunnel name : hw-client
Inside interface list: Ethernet0/1
Outside interface: Ethernet0/0
Current State: IPSEC_ACTIVE
Last Event: MTU_CHANGED
Address: 30.30.30.20 (applied on Loopback10000)
Mask: 255.255.255.255
DNS Primary: 30.30.30.10
DNS Secondary: 30.30.30.11
NBMS/WINS Primary: 30.30.30.12
NBMS/WINS Secondary: 30.30.30.13
Default Domain: cisco.com
Save Password: Disallowed
Current EzVPN Peer: 20.20.20.2

When ping from R2 (10.10.10.1) to R6 (30.30.30.30) and show crypto ipsec sa, we see encrypted packet is 5 but decrypted packet is 4. That is because first ping packet failed to reach the destination as we can see it from ping following result. After that, all ping packets will be encrypted and decrypted properly.

R2#ping 30.30.30.30 source 10.10.10.1

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 30.30.30.30, timeout is 2 seconds:
Packet sent with a source address of 10.10.10.1
.!!!!
Success rate is 80 percent (4/5), round-trip min/avg/max = 4/5/8 ms

R2#show crypto ipsec sa            

interface: Ethernet0/0
    Crypto map tag: Ethernet0/0-head-0, local addr 20.20.20.1

   protected vrf: (none)
   local  ident (addr/mask/prot/port): (30.30.30.20/255.255.255.255/0/0)
   remote ident (addr/mask/prot/port): (0.0.0.0/0.0.0.0/0/0)
   current_peer 20.20.20.2 port 500
     PERMIT, flags={origin_is_acl,}
    #pkts encaps: 5, #pkts encrypt: 5, #pkts digest: 5
    #pkts decaps: 4, #pkts decrypt: 4, #pkts verify: 4
    #pkts compressed: 0, #pkts decompressed: 0
    #pkts not compressed: 0, #pkts compr. failed: 0
    #pkts not decompressed: 0, #pkts decompress failed: 0
    #send errors 0, #recv errors 0

******Some Other Resources. 

From Cisco page:

  • Cisco Enhanced Easy VPN and DMVPN Comparison


Service/Feature Name

Enhanced Easy VPN

DMVPN

Scalability per Hub

Large number of spokes can be supported per hub

Depends on routing protocol chosen

Identical Configuration for All Spokes

Yes

No

Cross-Platform Support

Yes

No

Support for Software/Hardware Client

Yes

No software client support

Stateful Failover

No; but available with legacy Easy VPN

Depends on routing protocol for recovery

Always up Tunnel to Hub

Not required

Yes

Support for Multicast Traffic

Yes

Yes

Spoke-to-Spoke Direct Communication

No

Yes

Support for QoS

Yes

Yes

Support for Routing Protocols

No

Yes

Support for Certificates

Yes

Yes


  • Cisco Site-to-Site VPN Solution Comparison

Cisco GET-VPN

Cisco DMVPN

Cisco GRE-Based VPN

Cisco Easy VPN

Standard IPsec VPN

Tunnel-less VPN

Tunnel-based VPN

Customer Benefits

• Simplifies encryption integration on IP and Multiprotocol Label Switching (MPLS) WANs

• Simplifies encryption management through use of "group laying" instead of point-to-point key pairs

• Enables scalable and manageable any-to-any connectivitiy between sites

• Supports quality of services (QoS) multicase and routing

• Simplifies encryption of configuration and management for point-to-point GRE tunnels

• Supports QoS, multicast, and routing

• Enable transport of multicast and routing traffic across an IPsec VPN

• Support non-IP protocols

• Supports QoS

• Simplifies IPsec and remote-site device management through dynamic configuration policy-push

• Supports QoS

• Provides encryption between sites

• Supports QoS

When to Use

• Add encryption to MPLS or IP WANs while preserving any-to-any connectivity and networking features

• Other scalable, full-time meshing for IPsec VPNS

• Enables participation of smaller routers in meshed networks

• Simplifies encryption key management while supporting routing, QoS, and multicast

• Simplifies configuration for hub-and-spoke VPNs while supporting routing, QoS, and multicast

• Provides low-scale, on-demand meshing

• Use when routing must be supported across the VPN

• Use for same functions as hub-and-spoke DMVPN, but it requires more detailed configuration

• Use when simplifying overall VPN

• Configuration and management is the primary goal but only limited networking features are required

• Use to provide simple, unified configuration framework for mix of Cisco VPN products

• Use when multivendor interoperability is required

Product Interoperability

Cisco routers only

Cisco routers only

Cisco routers only

Cisco ASA 5500 Series, Cisco VPN 3000 Series, and Cisco PIX® Firewall

Mutlivendor

Scale

Thousands

Thousands hub and spoke; hundreds partially meshed spoke-to-spoke connections

Thousands

Thousands

Thousands

Provisioning and Management

CLI Cisco Security Manager

Cisco Security Manager and Cisco Router and Security Device Manager

Cisco Security Manager and Cisco Router and Security Device Manager

Configuration automatically pushed to remote sites from headend; headend policies defined in Cisco Security Manager or Cisco Router and Security Device Manager

Cisco Security Manager and Cisco Router and Security Device Manager

Topology

Hub and spoke; any-to-any

Hub and spoke on-demand spoke-to-spoke partial mesh; spoke-to-spoke connections automatically terminated when no traffic present

Hub and spoke; small-scale meshing as manageability allows

Hub and spoke

Hub and spoke; small-scale meshing as manageability allows

Routing

Supported; Cisco GET-VPN any-to-any connectivity capability can also be used to provide secure routing across any entire router backbone

Supported

Supported

Not Supported

Not Supported

QoS

Supported

Supported

Supported

Supported but QoS policy is not dynamically pushed to the remote sites

Supported

Multicast

Natively supported across MPLS and private IP networks, tunneled across Internet-based WANs

Tunneled

Tunneled

Not Supported

Not Supported

Non-IP Protocols

Not Supported

Not Supported

Supported

Not Supported

Not Supported

Private IP Addressing

Requires use of GRE or DMVPN with Cisco GET-VPN to support private addresses across public Internet backbones

Supported

Supported

Supported

Supported

High Availability

Routing

Routing

Routing

Stateless failover

Stateless failover


Sunday, May 18, 2014

Enabling SNMP v3 - Part 1: Cisco IOS Devices - Disable SNMP v1 and SNMP v2c

1. Enable SNMPv3

It is time to retire SNMPv2 on our network environment. Here is sample configuration for all of our Cisco devices. Some of old devices do not support AES , then DES will be the choice.

ip access-list standard snmp-Allow
permit 192.168.1.0 0.0.0.255

snmp-server view ReadAccess iso included

snmp-server view ReadAccess 1.3.6.1.6.3.18 excluded
snmp-server view ReadAccess 1.3.6.1.6.3.16 excluded
snmp-server view ReadAccess 1.3.6.1.6.3.15 excluded
snmp-server view ReadAccess 1.3.6.1.2.1.4.21 excluded
snmp-server view ReadAccess 1.3.6.1.2.1.4.22 excluded

snmp-server view ReadAccess iso included
snmp-server view ReadAccess internet included
snmp-server view ReadAccess system included
snmp-server view ReadAccess interfaces included
snmp-server view ReadAccess chassis included
snmp-server view WriteAccess iso included
snmp-server view WriteAccess internet included
snmp-server view WriteAccess system included
snmp-server view WriteAccess interfaces included
snmp-server view WriteAccess chassis included

snmp-server view WriteAccess iso included

snmp-server view WriteAccess 1.3.6.1.6.3.18 excluded
snmp-server view WriteAccess 1.3.6.1.6.3.16 excluded
snmp-server view WriteAccess 1.3.6.1.6.3.15 excluded
snmp-server view WriteAccess 1.3.6.1.2.1.4.21 excluded
snmp-server view WriteAccess 1.3.6.1.2.1.4.22 excluded

snmp-server group AccessRW v3 priv read ReadAccess write WriteAccess  access snmp-Allow

snmp-server group AccessRO v3 priv read ReadAccess access snmp-Allow

snmp-server user NetServices-RW  AccessRW v3 auth sha cisco priv aes 128 cisco

snmp-server user NetServices-RO AccessRO v3 auth sha cisco priv aes 128 cisco


snmp-server host 192.168.1.40 trap version 3 priv NetService-RO


snmp-server enable traps

2. Disable SNMP v1 and SNMP v2C

CiscoTest#show snmp group 
groupname: ILMI                             security model:v1 
contextname: <no context specified>         storage-type: permanent
readview : *ilmi                            writeview: *ilmi                           
notifyview: <no notifyview specified>       
row status: active

groupname: ILMI                             security model:v2c 
contextname: <no context specified>         storage-type: permanent
readview : *ilmi                            writeview: *ilmi                           
notifyview: <no notifyview specified>       
row status: active

groupname: SNMPv3-RO                        security model:v3 priv 
contextname: <no context specified>         storage-type: nonvolatile
readview : ReadView-All                     writeview: <no writeview specified>        
notifyview: <no notifyview specified>       
row status: active      access-list: snmp-Allow

groupname: SNMPv3-RW                        security model:v3 priv 
contextname: <no context specified>         storage-type: nonvolatile
readview : ReadView-All                     writeview: WriteView-All                   
notifyview: <no notifyview specified>       
row status: active      access-list: snmp-Allow

groupname: NetService-RO                    security model:v3 priv 
contextname: <no context specified>         storage-type: nonvolatile
readview : <no readview specified>          writeview: <no writeview specified>        
notifyview: *tv.FFFFFFFF.FFFFFFFF.FFFFFFFF.F
row status: active

CiscoTest(config)#no snmp-server group ILMI v1
CiscoTest(config)#no snmp-server group ILMI v2c
CiscoTest(config)#do sh snmp group
groupname: SNMPv3-RO                        security model:v3 priv 
contextname: <no context specified>         storage-type: nonvolatile
readview : ReadView-All                     writeview: <no writeview specified>        
notifyview: <no notifyview specified>       
row status: active      access-list: snmp-Allow

groupname: SNMPv3-RW                        security model:v3 priv 
contextname: <no context specified>         storage-type: nonvolatile
readview : ReadView-All                     writeview: WriteView-All                   
notifyview: <no notifyview specified>       
row status: active      access-list: snmp-Allow

groupname: NetService-RO                    security model:v3 priv 
contextname: <no context specified>         storage-type: nonvolatile
readview : <no readview specified>          writeview: <no writeview specified>        
notifyview: *tv.FFFFFFFF.FFFFFFFF.FFFFFFFF.F
row status: active

Unfortunately those groups will come back after system rebooted. Best way is to remove them from system view by following commands:

snmp-server view *ilmi system excluded
snmp-server view *ilmi atmForumUni excluded
snmp-server view v1default iso excluded

R-Test-Lab#show snmp view
*ilmi system - excluded nonvolatile active
*ilmi atmForumUni - excluded nonvolatile active
cac_view pimMIB - included read-only active
cac_view msdpMIB - included read-only active
cac_view interfaces - included read-only active
cac_view ip - included read-only active
cac_view ospf - included read-only active
cac_view bgp - included read-only active
cac_view dot1dBridge - included read-only active
cac_view ifMIB - included read-only active
cac_view nhrpMIB - included read-only active
cac_view ipMRouteStdMIB - included read-only active
cac_view igmpStdMIB - included read-only active
cac_view ipForward - included read-only active
cac_view ipTrafficStats - included read-only active
cac_view ospfTrap - included read-only active
cac_view sysUpTime.0 - included read-only active
cac_view ciscoPingMIB - included read-only active
cac_view ciscoIpSecFlowMonitorMIB - included read-only active
cac_view ciscoIpSecPolMapMIB - included read-only active
cac_view ciscoPimMIB - included read-only active
cac_view ciscoMgmt.187 - included read-only active
cac_view ciscoIfExtensionMIB - included read-only active
cac_view ciscoEigrpMIB - included read-only active
cac_view ciscoCefMIB - included read-only active
cac_view ciscoNhrpExtMIB - included read-only active
cac_view ciscoIpMRouteMIB - included read-only active
cac_view ciscoIPsecMIB - included read-only active
cac_view cospf - included read-only active
cac_view ciscoExperiment.101 - included read-only active
cac_view ciscoIetfIsisMIB - included read-only active
cac_view ciscoIetfBfdMIB - included read-only active
cac_view ifIndex - included read-only active
cac_view ifDescr - included read-only active
cac_view ifType - included read-only active
cac_view ifAdminStatus - included read-only active
cac_view ifOperStatus - included read-only active
cac_view snmpTraps.3 - included read-only active
cac_view snmpTraps.4 - included read-only active
cac_view snmpTrapOID.0 - included read-only active
cac_view internet.6.3.1.1.4.3.0 - included read-only active
cac_view lifEntry.20 - included read-only active
cac_view cciDescriptionEntry.1 - included read-only active
v1default iso - excluded nonvolatile active
v1default internet.6.3.15 - excluded permanent active
v1default internet.6.3.16 - excluded permanent active
v1default internet.6.3.18 - excluded permanent active
v1default ciscoMgmt.394 - excluded permanent active
v1default ciscoMgmt.395 - excluded permanent active
v1default ciscoMgmt.399 - excluded permanent active
v1default ciscoMgmt.400 - excluded permanent active