Pushing Policy Failed because Checkpoint Firewall "Load on module failed - no memory"

One day when pushing firewall policy from Checkpoint management server to UTM 272 cluster gateways, it failed and I got error message "Load on module failed - no memory" on one of cluster members.

"Network Security Policy 'Montreal_DMZ' was prepared on Tue Sep 16 10:00:58 2014.

The following errors and warnings exist:

Installation failed. Reason: Load on Module failed - no memory. ( message from member fw_Montreal)"

Searched online and found a couple of sk and posts regarding this error. But all of them are not same as my scenario. Eventually with an open case, Checkpoint support send me those commands fixed my issue. This post records this for the reference

On both cluster members perform under expert mode:

1. tellpm process:monitord
2. ps aux | grep cpd
3. kill -15 <PID_of_CPD> 
4. tellpm process:monitord t
5. cpwd_admin start -name CPD -path "$CPDIR/bin/cpd" -command "cpd"

Here is the output from my problem devices:
[Expert@CP-DMZ:0]# tellpm process:monitord
[Expert@CP-DMZ:0]#
Message from syslogd@ at Tue Sep 16 10:27:42 2014 ...
CP-M-DMZ monitord[4129]: monitord got killed
[Expert@CP-DMZ:0]#
[Expert@CP-DMZ:0]# ps aux | grep cpd
admin     4461  0.0  0.3 212412  3196 ?        Dsl  Aug06  42:47 cpd
admin     6905  0.0  0.0   1816   492 pts/2    S+   10:27   0:00 grep cpd

[Expert@CP-DMZ:0]# kill -15 4461

[Expert@CP-DMZ:0]#
[Expert@CP-DMZ:0]# tellpm process:monitord t

[Expert@CP-DMZ:0]#
[Expert@CP-DMZ:0]# cpwd_admin start -name CPD -path "$CPDIR/bin/cpd" -command "cpd"
cpwd_admin:
Process CPD started successfully (pid=7030)
[Expert@CP-DMZ:0]#

After that, pushed policy again and this time both cluster member got policy without any error.

VPN Lab between Cisco Router and Checkpoint Firewall

Topology:

1. Checkpoint configuration

a. Set up Checkpoint Gateway IPv4 address as the peer address showing in the cisco router.

b. Create a new Interoperable Device for Cisco Router

c. ISAKMP Settings and IPSec Settings

d. Pre-shared key

 e. Advanced ISAKMP Configurations and IPSec Configurations

f. Set up access-list

2. Cisco Router Configuration

crypto isakmp policy 10
 encr aes 256
 authentication pre-share
 group 2
crypto isakmp key cisco1234 address 192.168.2.37
!
crypto ipsec transform-set VPN1 esp-aes esp-sha-hmac
!
crypto map VPN-MAP 10 ipsec-isakmp
 set peer 192.168.2.37
 set transform-set VPN1
 match address ACL-VPN
!
interface GigabitEthernet0/0
 ip address 192.168.2.38 255.255.255.0
 crypto map VPN-MAP

ip route 10.94.200.0 255.255.255.0 192.168.2.37
!
ip access-list extended ACL-VPN
 permit ip 192.168.1.0 0.0.0.255 10.94.200.0 0.0.0.255
!

Resize Checkpoint Firewall's Disk/Partition Space (Gaia and Splat Platform)

Checkpoint Firewall Appliances are using Logical Volume Manager (LVM)

Sometimes, the logic volume do not have enough size to perform your operations. Checkpoint Expert Mode have some commands to modify the volume size while keeping your data on the disk.

1. Check volume size in the volume group vg_splat

[Expert@gw-a9533e:0]# lvs
  /dev/hdc: open failed: No medium found
  LV         VG       Attr   LSize Origin Snap%  Move Log Copy%
  lv_current vg_splat -wi-ao 5.00G                             
  lv_log     vg_splat -wi-ao 3.00G

From dh -h command it shows 97% usage on the system volume vg_splat-lv_current. R77.10 will not allow you to operation the firewall if it found insufficient disk space with the warning "there is insufficient disk space to complete the operation'

[Expert@gw-a9533e:0]# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/vg_splat-lv_current
                      4.9G  4.8G  52M  97% /
/dev/sda1             289M   24M  251M   9% /boot
tmpfs                 980M     0  980M   0% /dev/shm
/dev/mapper/vg_splat-lv_log
                      3.0G  175M  2.6G   7% /var/log

2. Check free space on the disk

[Expert@gw-a9533e:0]# pvs
  /dev/hdc: open failed: No medium found
  PV         VG       Fmt  Attr PSize  PFree
  /dev/sda3  vg_splat lvm2 a-   14.69G 6.69G

3. Check how much space assigned to the logical volume

[Expert@gw-a9533e:0]# lvs
  /dev/hdc: open failed: No medium found
  LV         VG       Attr   LSize Origin Snap%  Move Log Copy%
  lv_current vg_splat -wi-ao 5.00G                             
  lv_log     vg_splat -wi-ao 3.00G      

4. Resize the logic volume from 5G to 11G

[Expert@gw-a9533e:0]# lvresize -L 11G vg_splat/lv_current
  /dev/hdc: open failed: No medium found
  Extending logical volume lv_current to 11.00 GB
  Logical volume lv_current successfully resized


 Check the space again with pvs command, there is only 704M free space left
[Expert@gw-a9533e:0]# pvs
  /dev/hdc: open failed: No medium found
  PV         VG       Fmt  Attr PSize  PFree 
  /dev/sda3  vg_splat lvm2 a-   14.69G 704.00M


[Expert@gw-a9533e:0]# lvs
  /dev/hdc: open failed: No medium found
  LV         VG       Attr   LSize  Origin Snap%  Move Log Copy%
  lv_current vg_splat -wi-ao 11.00G                             
  lv_log     vg_splat -wi-ao  3.00G  

5. Resize the file system                    

[Expert@gw-a9533e:0]# resize2fs /dev/vg_splat-lv_current
resize2fs 1.39 (29-May-2006)
Filesystem at /dev/vg_splat/lv_current is mounted on /; on-line resizing required
Performing an on-line resize of /dev/vg_splat/lv_current to 2883584 (4k) blocks.
The filesystem on /dev/vg_splat/lv_current is now 2883584 blocks long.

6. Verify with df command

[Expert@gw-a9533e:0]# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/vg_splat-lv_current
                       11G  3.8G  6.4G  38% /
/dev/sda1             289M   24M  251M   9% /boot
tmpfs                 980M     0  980M   0% /dev/shm
/dev/mapper/vg_splat-lv_log
                      3.0G  176M  2.6G   7% /var/log

References:

• http://blog.lachmann.org/wp-content/uploads/2010/09/2010-CPUG-CON-Tobias-Lachmann-Best-practices-for-the-Check-Point-UTM-1-appliances.pdf
• http://www.networkoc.net/check-point-gaia-extending-your-disk/

Workaround for can not Edit Virtual Machine Settings in VMware vSphere ESXi 5.5

After upgraded ESXi to 5.5, some VMs got following screenshot error when tried to edit the configuration.

"You cannot use the vSphere Client to edit the settings of virtual machines of version 10 or higher.

Use the vSphere Web Client to edit the settings of this virtual machine
"

 

The solution is to use vSphere Web client. The workaround for me is in following two methos:

1. Change virtualHW.version to 9 or lower

• In ESXi, Remote VM from Inventory
• SSH into ESXi
• cd /vmfs/volumes/datastore1/testvm
• VI the VM's .VMX file
• Find the line that says 'virtualHW.version'. Example : virtualHW.version = "10"
• Change the value to 8. Example : virtualHW.version = "9"
• Register this VM back into Inventory with command "vim-cmd solo/registervm /vmfs/volumes/datastore1/testvm/testvm.vmx"

2. Use VMware Workstation 

In Workstation, in there VM's manage menu, you can change VM's Hardware Compatbility to Workstation 9.0 or lower, then upload it to ESXi as shown in this post step 7.

OpenWRT in Vmware as a light weight router and virtual host

OpenWrt is a Linux distribution for embedded devices and provides a fully writable filesystem with package management.

Also for me, it is perfect for running OpenWRT as a small router or virtual host in my virtual rack. I was using BSD Router (BSDRP) for this purpose quite a while. Now it seems better one coming.

To make a mark on it, I list all steps regarding how to load it in the vmware workstation.

1. Download the package from

https://downloads.openwrt.org/backfire/10.03.1/x86_generic/openwrt-x86-generic-combined-ext2.vmdk
MD5Sums：  a258b7a5787f6bd8c8169391941813f4  

2. Create a vm with following configurations with almost all default settings except choosing Other Linux 2.6.x kernel guest operating system

Memory = 32M
Hard Disk = 52M

3. Most important part is to choose IDE as your hard disk type and using existing downloaded image. Else if the default SCSI type will make your vm stop at "Waiting for root device /dev/sda2..."

4. Booting Console Windows Outputs

Please be patient, while OpenWrt loads ...
- preinit -
Press the [f] key and hit [enter] to enter failsafe mode
- regular preinit -
- init -

Please press Enter to activate this console. natsemi dp8381x driver, version 2.1, Sept 11, 2006
  originally by Donald Becker <becker@scyld.com>
  2.4.x kernel port by Jeff Garzik, Tjeerd Mulder
PPP generic driver version 2.4.2
ip_tables: (C) 2000-2006 Netfilter Core Team
NET: Registered protocol family 24
nf_conntrack version 0.5.0 (449 buckets, 1796 max)
CONFIG_NF_CT_ACCT is deprecated and will be removed soon. Please use
nf_conntrack.acct=1 kernel parameter, acct=1 nf_conntrack module option or
sysctl net.netfilter.nf_conntrack_acct=1 to enable it.
8139too Fast Ethernet driver 0.9.28
e100: Intel(R) PRO/100 Network Driver, 3.5.24-k2-NAPI
e100: Copyright(c) 1999-2006 Intel Corporation
Intel(R) PRO/1000 Network Driver - version 7.3.21-k5-NAPI
Copyright (c) 1999-2006 Intel Corporation.
ne2k-pci.c:v1.03 9/22/2003 D. Becker/P. Gortmaker
pcnet32.c:v1.35 21.Apr.2008 tsbogend@alpha.franken.de
pcnet32 0000:02:00.0: PCI INT A -> GSI 18 (level, low) -> IRQ 18
pcnet32: PCnet/PCI II 79C970A at 0x2000, 00:0c:29:cb:1b:48 assigned IRQ 18.
eth0: registered as PCnet/PCI II 79C970A
pcnet32 0000:02:01.0: PCI INT A -> GSI 19 (level, low) -> IRQ 19
pcnet32: PCnet/PCI II 79C970A at 0x2080, 00:0c:29:cb:1b:52 assigned IRQ 19.
eth1: registered as PCnet/PCI II 79C970A
pcnet32: 2 cards_found.
eth0: link up
sis900.c: v1.08.10 Apr. 2 2006
device eth0 entered promiscuous mode
br-lan: port 1(eth0) entering forwarding state
via-rhine.c:v1.10-LK1.4.3 2007-03-06 Written by Donald Becker



BusyBox v1.15.3 (2011-11-24 18:38:13 CET) built-in shell (ash)
Enter 'help' for a list of built-in commands.

  _______                     ________        __
 |       |.-----.-----.-----.|  |  |  |.----.|  |_
 |   -   ||  _  |  -__|     ||  |  |  ||   _||   _|
 |_______||   __|_____|__|__||________||__|  |____|
          |__| W I R E L E S S   F R E E D O M
 Backfire (10.03.1, r29592) ------------------------
  * 1/3 shot Kahlua    In a shot glass, layer Kahlua
  * 1/3 shot Bailey's  on the bottom, then Bailey's,
  * 1/3 shot Vodka     then Vodka.
 ---------------------------------------------------
root@OpenWrt:/#

5. Basic Configuration with UCI Show command

root@OpenWrt:/# uci
Usage: uci [<options>] <command> [<arguments>]

Commands:
        batch
        export     [<config>]
        import     [<config>]
        changes    [<config>]
        commit     [<config>]
        add        <config> <section-type>
        add_list   <config>.<section>.<option>=<string>
        show       [<config>[.<section>[.<option>]]]
        get        <config>.<section>[.<option>]
        set        <config>.<section>[.<option>]=<value>
        delete     <config>[.<section[.<option>]]
        rename     <config>.<section>[.<option>]=<name>
        revert     <config>[.<section>[.<option>]]
        reorder    <config>.<section>=<position>

Options:
        -c <path>  set the search path for config files (default: /etc/config)
        -d <str>   set the delimiter for list values in uci show
        -f <file>  use <file> as input instead of stdin
        -L         do not load any plugins
        -m         when importing, merge data into an existing package
        -n         name unnamed sections on export (default)
        -N         don't name unnamed sections
        -p <path>  add a search path for config change files
        -P <path>  add a search path for config change files and use as default
        -q         quiet mode (don't print error messages)
        -s         force strict mode (stop on parser errors, default)
        -S         disable strict mode
        -X         do not use extended syntax on 'show'

root@OpenWrt:/# uci show
dhcp.@dnsmasq[0]=dnsmasq
dhcp.@dnsmasq[0].domainneeded=1
dhcp.@dnsmasq[0].boguspriv=1
dhcp.@dnsmasq[0].filterwin2k=0
dhcp.@dnsmasq[0].localise_queries=1
dhcp.@dnsmasq[0].rebind_protection=1
dhcp.@dnsmasq[0].rebind_localhost=1
dhcp.@dnsmasq[0].local=/lan/
dhcp.@dnsmasq[0].domain=lan
dhcp.@dnsmasq[0].expandhosts=1
dhcp.@dnsmasq[0].nonegcache=0
dhcp.@dnsmasq[0].authoritative=1
dhcp.@dnsmasq[0].readethers=1
dhcp.@dnsmasq[0].leasefile=/tmp/dhcp.leases
dhcp.@dnsmasq[0].resolvfile=/tmp/resolv.conf.auto
dhcp.lan=dhcp
dhcp.lan.interface=lan
dhcp.lan.start=100
dhcp.lan.limit=150
dhcp.lan.leasetime=12h
dhcp.wan=dhcp
dhcp.wan.interface=wan
dhcp.wan.ignore=1
dropbear.@dropbear[0]=dropbear
dropbear.@dropbear[0].PasswordAuth=on
dropbear.@dropbear[0].RootPasswordAuth=on
dropbear.@dropbear[0].Port=22
firewall.@defaults[0]=defaults
firewall.@defaults[0].syn_flood=1
firewall.@defaults[0].input=ACCEPT
firewall.@defaults[0].output=ACCEPT
firewall.@defaults[0].forward=REJECT
firewall.@zone[0]=zone
firewall.@zone[0].name=lan
firewall.@zone[0].network=lan
firewall.@zone[0].input=ACCEPT
firewall.@zone[0].output=ACCEPT
firewall.@zone[0].forward=REJECT
firewall.@zone[1]=zone
firewall.@zone[1].name=wan
firewall.@zone[1].network=wan
firewall.@zone[1].input=REJECT
firewall.@zone[1].output=ACCEPT
firewall.@zone[1].forward=REJECT
firewall.@zone[1].masq=1
firewall.@zone[1].mtu_fix=1
firewall.@forwarding[0]=forwarding
firewall.@forwarding[0].src=lan
firewall.@forwarding[0].dest=wan
firewall.@rule[0]=rule
firewall.@rule[0].name=Allow-DHCP-Renew
firewall.@rule[0].src=wan
firewall.@rule[0].proto=udp
firewall.@rule[0].dest_port=68
firewall.@rule[0].target=ACCEPT
firewall.@rule[0].family=ipv4
firewall.@rule[1]=rule
firewall.@rule[1].name=Allow-Ping
firewall.@rule[1].src=wan
firewall.@rule[1].proto=icmp
firewall.@rule[1].icmp_type=echo-request
firewall.@rule[1].family=ipv4
firewall.@rule[1].target=ACCEPT
firewall.@rule[2]=rule
firewall.@rule[2].name=Allow-DHCPv6
firewall.@rule[2].src=wan
firewall.@rule[2].proto=udp
firewall.@rule[2].src_ip=fe80::/10
firewall.@rule[2].src_port=547
firewall.@rule[2].dest_ip=fe80::/10
firewall.@rule[2].dest_port=546
firewall.@rule[2].family=ipv6
firewall.@rule[2].target=ACCEPT
firewall.@rule[3]=rule
firewall.@rule[3].name=Allow-ICMPv6-Input
firewall.@rule[3].src=wan
firewall.@rule[3].proto=icmp
firewall.@rule[3].icmp_type=echo-request destination-unreachable packet-too-big time-exceeded bad-header unknown-header-type router-solicitation neighbour-solicitation
firewall.@rule[3].limit=1000/sec
firewall.@rule[3].family=ipv6
firewall.@rule[3].target=ACCEPT
firewall.@rule[4]=rule
firewall.@rule[4].name=Allow-ICMPv6-Forward
firewall.@rule[4].src=wan
firewall.@rule[4].dest=*
firewall.@rule[4].proto=icmp
firewall.@rule[4].icmp_type=echo-request destination-unreachable packet-too-big time-exceeded bad-header unknown-header-type
firewall.@rule[4].limit=1000/sec
firewall.@rule[4].family=ipv6
firewall.@rule[4].target=ACCEPT
firewall.@include[0]=include
firewall.@include[0].path=/etc/firewall.user
luci.main=core
luci.main.lang=auto
luci.main.mediaurlbase=/luci-static/openwrt.org
luci.main.resourcebase=/luci-static/resources
luci.flash_keep=extern
luci.flash_keep.uci=/etc/config/
luci.flash_keep.dropbear=/etc/dropbear/
luci.flash_keep.openvpn=/etc/openvpn/
luci.flash_keep.passwd=/etc/passwd
luci.flash_keep.opkg=/etc/opkg.conf
luci.flash_keep.firewall=/etc/firewall.user
luci.flash_keep.uploads=/lib/uci/upload/
luci.languages=internal
luci.languages.en=English
luci.sauth=internal
luci.sauth.sessionpath=/tmp/luci-sessions
luci.sauth.sessiontime=3600
luci.ccache=internal
luci.ccache.enable=1
luci.themes=internal
luci.themes.OpenWrt=/luci-static/openwrt.org
network.loopback=interface
network.loopback.ifname=lo
network.loopback.proto=static
network.loopback.ipaddr=127.0.0.1
network.loopback.netmask=255.0.0.0
network.lan=interface
network.lan.ifname=eth0
network.lan.type=bridge
network.lan.proto=static
network.lan.ipaddr=192.168.1.1
network.lan.netmask=255.255.255.0
system.@system[0]=system
system.@system[0].hostname=OpenWrt
system.@system[0].timezone=UTC
system.ntp=timeserver
system.ntp.server=0.openwrt.pool.ntp.org 1.openwrt.pool.ntp.org 2.openwrt.pool.ntp.org 3.openwrt.pool.ntp.org
ucitrack.@network[0]=network
ucitrack.@network[0].init=network
ucitrack.@network[0].affects=dhcp radvd
ucitrack.@wireless[0]=wireless
ucitrack.@wireless[0].affects=network
ucitrack.@firewall[0]=firewall
ucitrack.@firewall[0].init=firewall
ucitrack.@firewall[0].affects=luci-splash qos miniupnpd
ucitrack.@olsr[0]=olsr
ucitrack.@olsr[0].init=olsrd
ucitrack.@dhcp[0]=dhcp
ucitrack.@dhcp[0].init=dnsmasq
ucitrack.@dropbear[0]=dropbear
ucitrack.@dropbear[0].init=dropbear
ucitrack.@httpd[0]=httpd
ucitrack.@httpd[0].init=httpd
ucitrack.@fstab[0]=fstab
ucitrack.@fstab[0].init=fstab
ucitrack.@qos[0]=qos
ucitrack.@qos[0].init=qos
ucitrack.@system[0]=system
ucitrack.@system[0].init=led
ucitrack.@system[0].affects=luci_statistics
ucitrack.@luci_splash[0]=luci_splash
ucitrack.@luci_splash[0].init=luci_splash
ucitrack.@upnpd[0]=upnpd
ucitrack.@upnpd[0].init=miniupnpd
ucitrack.@ntpclient[0]=ntpclient
ucitrack.@ntpclient[0].init=ntpclient
ucitrack.@samba[0]=samba
ucitrack.@samba[0].init=samba
ucitrack.@tinyproxy[0]=tinyproxy
ucitrack.@tinyproxy[0].init=tinyproxy
uhttpd.main=uhttpd
uhttpd.main.listen_http=0.0.0.0:80
uhttpd.main.listen_https=0.0.0.0:443
uhttpd.main.home=/www
uhttpd.main.rfc1918_filter=1
uhttpd.main.cert=/etc/uhttpd.crt
uhttpd.main.key=/etc/uhttpd.key
uhttpd.main.cgi_prefix=/cgi-bin
uhttpd.main.script_timeout=60
uhttpd.main.network_timeout=30
uhttpd.main.tcp_keepalive=1
uhttpd.px5g=cert
uhttpd.px5g.days=730
uhttpd.px5g.bits=1024
uhttpd.px5g.country=DE
uhttpd.px5g.state=Berlin
uhttpd.px5g.location=Berlin
uhttpd.px5g.commonname=OpenWrt

6. Change Interface IP Address

• VI /etc/config/network
• or use UCI command

root@OpenWrt:/# ifconfig
br-lan    Link encap:Ethernet  HWaddr 00:0C:29:CB:1B:48 
          inet addr:192.168.1.1  Bcast:192.168.1.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:109 errors:0 dropped:0 overruns:0 frame:0
          TX packets:2 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:25588 (24.9 KiB)  TX bytes:812 (812.0 B)

eth0      Link encap:Ethernet  HWaddr 00:0C:29:CB:1B:48 
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:109 errors:0 dropped:0 overruns:0 frame:0
          TX packets:2 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:27114 (26.4 KiB)  TX bytes:812 (812.0 B)
          Interrupt:18 Base address:0x2000

lo        Link encap:Local Loopback 
          inet addr:127.0.0.1  Mask:255.0.0.0
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:864 errors:0 dropped:0 overruns:0 frame:0
          TX packets:864 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:60480 (59.0 KiB)  TX bytes:60480 (59.0 KiB)

root@OpenWrt:/# uci set network.lan.proto=static
root@OpenWrt:/# uci set network.lan.ipaddr=192.168.1.130
root@OpenWrt:/# uci set network.lan.netmask=255.255.255.0
root@OpenWrt:/# uci set network.lan.gateway=192.168.1.1
root@OpenWrt:/# uci set network.lan.dns=8.8.8.8
 
root@OpenWrt:/# /etc/init.d/network restart
br-lan: port 1(eth0) entering disabled state
device eth0 left promiscuous mode
br-lan: port 1(eth0) entering disabled state
eth0: link up
eth0: link up
device eth0 entered promiscuous mode
br-lan: port 1(eth0) entering forwarding state
root@OpenWrt:/# ifconfig
br-lan    Link encap:Ethernet  HWaddr 00:0C:29:CB:1B:48 
          inet addr:192.168.1.130  Bcast:192.168.1.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

 7. Upload it to ESXi

8. Second Interface

By default, only eth0 is activated as lan interface and assigned ip address 192.168.1.1. It is much easier to add another Interface into OpenWRT through Web UI as shown below screenshot.

References:

• http://wenku.baidu.com/view/a8bbe60516fc700abb68fc8c.html