Pages

Thursday, March 30, 2017

Juniper Space License Issue on Citrix Xen Environment

Based on Juniper "Junos Space Virtual Appliance Installation and Configuration Guide" , JunOS Space " must deploy the virtual appliance on a VMware ESX, VMWare ESXi or KVM server, which provides a CPU, hard disk, RAM, and a network controller, but requires installation of an operating system and applications to become fully functional."

In my test environment, one JunOS Space has been installed on Citrix Xen environment and it is working fine until we tried to import a license.

The license was generated from Juniper License site and emailed to us in a txt file. It used to work on another machine hosted in Vmware ESX environment. Unfortunately, this time, JunOS Space said no.

The License Information windows says:
License upload failed. Please check the following:
1) License data format
2) License Keys
Juniper Space VE at Citrix Xen Server - License Error




Solution:

JTAC was able to help this out using a workaround, basically it just changed JunOS Space Sql database model number from HVM domU to Vmware Virtual Platform. This works.


Space release 16.1R2.7 (381623)

Last login: Thu Mar 23 13:51:52 2017 from 10.94.200.14

Welcome to the Junos Space network settings utility.

Initializing, please wait


Junos Space Settings Menu

1> Change Password
2> Change Network Settings
3> Change Time Options
4> Retrieve Logs
5> Security
6> Expand VM Drive Size
7> (Debug) run shell

A> Apply changes
Q> Quit
R> Redraw Menu

Choice [1-7,AQR]: 7

[sudo] password for admin:
[root@space-c6186f1b3edb ~]#
[root@space-c6186f1b3edb ~]#
[root@space-c6186f1b3edb ~]#
[root@space-c6186f1b3edb ~]#
mysql -u jboss -pnetscreen build_db
Warning: Using a password on the command line interface can be insecure.
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 603536
Server version: 5.6.34-enterprise-commercial-advanced-log MySQL Enterprise Server - Advanced Edition (Commercial)

Copyright (c) 2000, 2016, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql>
select HardwareModel from FABRIC_NODE;
+---------------+
| HardwareModel |
+---------------+
| HVM domU |
| HVM domU |
+---------------+
2 rows in set (0.00 sec)

mysql>
update FABRIC_NODE set HardwareModel = 'VMware Virtual Platform' where HardwareModel='HVM domU';
Query OK, 1 row affected (0.01 sec)
Rows matched: 1 Changed: 1 Warnings: 0

mysql> select HardwareModel from FABRIC_NODE;
+-------------------------+
| HardwareModel |
+-------------------------+
| VMware Virtual Platform |
| HVM domU |
+-------------------------+
2 rows in set (0.00 sec)

mysql>
update FABRIC_NODE set HardwareModel = 'VMware Virtual Platform' where HardwareModel=' HVM domU';
Query OK, 1 row affected (0.01 sec)
Rows matched: 1 Changed: 1 Warnings: 0

mysql> select HardwareModel from FABRIC_NODE;
+-------------------------+
| HardwareModel |
+-------------------------+
| VMware Virtual Platform |
| VMware Virtual Platform |
+-------------------------+
2 rows in set (0.00 sec)

mysql> quit
Bye
[root@space-c6186f1b3edb ~]#
[root@space-c6186f1b3edb ~]#
[root@space-c6186f1b3edb ~]#



License has been accepted as shown in the following screenshot.


Summary for Resolution detailed steps:

Junos Space is Running on Citrix platform, not compatible as per release notes, because of this we were unable to activate the license.
By doing some SQL magic we were able to activate it, steps:
  • ·         Log to Junos space CLI (debugging mode #7)
  • ·         Then, access MySQL 

            #mysql -u jboss -pnetscreen build_db 
  • ·         Then, run: 

              #mysql> select HardwareModel from FABRIC_NODE;
  • ·         To get the following output: 

+-----------------+
| HardwareModel   |
+-----------------+
| HVM domU |
+-----------------+
1 row in set (0.00 sec) 
  • ·         Then, it is required to change the value ‘Virtual Machine’ for 'VMware Virtual Platform' using: 

# update FABRIC_NODE set HardwareModel = 'VMware Virtual Platform' where HardwareModel='HVM domU'; 
  • ·         Copy –paste the license on notepad++ to avoid tabulation / indentation issues
  • ·         License successfully activated.



Brocade Switch Access Through SSH and Web Tools




1. Through SSH
It is pretty straightforward, launch ssh client, enter your switch ip and credential, you will be in the command line.




-----------------------------------------------------------------

TEST1BRO1:FID128:admin>
TEST1BRO1:FID128:admin>
help
aaaconfig Configure RADIUS for AAA services
ad Specifies all administration domain (AD)-level
operations
ag Configure the Access Gateway feature
agautomapbalance Configure the Access Gateway automapbalance feature
agshow Displays the Access Gateway information registered
with the fabric
agtcfgdefault Reset SNMP agent to factory default
aliadd Add a member to a zone alias
alicreate Create a zone alias
alidelete Delete a zone alias
aliremove Remove a member from a zone alias
alishow Print zone alias information
apploginhistory Used to how the login history of external
applications
aptpolicy Get and set Advanced Performance Tuning policy
auditcfg Modifies and displays audit log filter
configuration.
auditdump Display audit log
authutil Get and set authentication configuration
bannerset Set security banner
bannershow Display security banner
bcastshow Display broadcast routing information
bladecfggemode Configure 1GigE/10GigE port configuration for blade
bladedisable Disables all user ports on a blade.
bladeenable Enables all User Ports on a blade.
bladeswap Swaps area numbers for all ports on a blade
bootluncfg Configure boot LUN for an HBA
bottleneckmon Configure port bottleneck monitor
bpportloopbacktest Functional test of port via blade processor path.
bpturboramtest MBIST test for AP blade BP ASICs
bufopmode Configure buffer credit model
ceeportledtest Cycle user port LEDs.
ceeportloopbacktest Functional test of port N->N path.
ceeturboramtest MBIST test for ASICs
cfgactvshow Display Effective zone configuration information
cfgadd Add a member to a configuration
cfgclear Clear all zone configurations
cfgcreate Create a zone configuration
cfgdelete Delete a zone configuration
cfgdisable Disable a zone configuration
cfgenable Enable a zone configuration
cfgmcdtmode Configure legacy MCDT zoning behavior
cfgremove Remove a member from a configuration
cfgsave Save zone configurations in flash
cfgsaveactivetodefined Moves the effective configuration to the defined
configuration
cfgshow Print zone configuration information
cfgsize Print size details of zone database
cfgtransabort Abort zone configuration transaction
cfgtransshow Print zone configurations in transaction buffer
chassisbeacon Set chassis beacon on or off
chassisconfig Display or set the chassis configuration
chassisdisable Disable all ports in chassis
chassisdistribute Distributes data to physical chassis in a fabric
chassisenable Enable all ports in chassis
chassisname Display or set the chassis's Name
chassisshow Display all Field Replaceable Units (FRU)
clihistory Displays a log of most recently executed CLI
commands
cmsh Start CEE Management Shell
configdefault Reset config to factory default
configdownload Load switch config from a server
configlist List the stored config files
configremove Remove a stored config file
configshow Print switch config parameters
configupload Save switch config to a server
configure Set logical switch config parameters
configurechassis Set chassis config parameters
cryptocfg Configure encryption node functions
datatypeshow Display available diagnostic data types.
date Print/set the system date and time
dbgshow Display debug levels of various modules
defzone Activates or deactivates a default zone
configuration.
diagclearerror Clears diagnostics failure status.
diagdisablepost Disable diagnostic POST.
diagenablepost Enable diagnostic POST.
diaghelp Display diagnostic command descriptions.
diagpost Displays the contents of port registers and
memories.
diagretry Sets diagnostic retry mode
diagshow Display diagnostic status.
diagstatus Display currently running diagnostic tests.
distribute Distributes data to switches in a fabric.
dlsreset Configure dynamic load sharing (DLS) option
dlsset Configure dynamic load sharing (DLS) option
dlsshow Display the state of the dynamic load sharing
option
dnsconfig Configure DNS service
enclosureshow Display the switch enclosure attributes
errclear Clear error log
errdelimiterset Sets the error log start and end delimiter for
messages sent to the console and syslog
errdump Print error log (no page breaks)
errfilterset Sets a filter for an error log destination
errmoduleshow Displays all the defined modules in the system
errshow Print error log
fabportshow Display contents of a particular port's data
fabretryshow Display retry count of various fabric commands
fabriclog Displays or manipulates the fabric log
fabricprincipal Configure Principal Switch Selection mode
fabricshow Print fabric membership info
fabstatsshow Displays the fabric statistics information
fabswitchshow Display fabric switch state information
fandisable Disables a fan unit.
fanenable Enables a fan unit.
fanshow Print fan status
fastboot Reboot this switch and bypass POST
fastwritecfg Configure FC fastwrite feature
fcipchiptest Functional Test of Components in FCIP Complex.
fciphelp Print FCIP help info
fcipledtest Cycle user port LEDs.
fcippathtest Data Path Test of the FCIP complex.
fcoe Displays and configures FCoE ports FIP and fcmap
configurations
fcoelogincfg Enable/Disable Save or display FCoE login
configuration
fcoelogingroup Creates or modifies FCoE login configuration
fcping Sends a FC ELS ECHO request to a port
fcplogclear Clear the FCP log
fcplogdisable Disable logging of FCP events
fcplogenable Enable logging of FCP events
fcplogshow Display FCP probing log information
fcpprobeshow Display FCP probing information
fcprlsshow Display FCP RLS (read link state) information
fcrbcastconfig Configure interfabric broadcast frame forwarding
fcrchiptest Functional Test of FCR FPGA.
fcrconfigure Sets FC Router configuration parameters.
fcredgeshow Displays FIDs assigned to defined EX_Port
fcrfabricshow Displays FC Routers on a backbone fabric.
fcrlsan Configure LSAN policies
fcrlsancount Display maximum LSAN zone limit.
fcrlsanmatrix Manage LSAN fabric matrix configuration.
fcrpathtest Data Path Test on Connection between FCR FPGA and
Central ASIC.
fcrphydevshow Displays FC Router physical device information.
fcrproxyconfig Displays or configures proxy devices presented by
an FC Router.
fcrproxydevshow Displays FC Router proxy device information.
fcrresourceshow Displays FC Router physical resource usage.
fcrrouterportcost Modify FC Router port cost configuration.
fcrrouteshow Displays FC Router route information.
fcrxlateconfig Displays or persistently configures a translate
(xlate) domain's domain ID for both
EX_Port-attached fabric and backbone fabric.
fddcfg fddcfg manages the fabric data distribution
configuration parameters.
fdmicacheshow Display abbreviated remote FDMI device information
according to remote domain ID
fdmishow Display detailed FDMI device information
ficoncfg Manage FICON configuration
ficonclear Clears contents of the specified FICON management
database
ficoncupset Sets FICON-CUP parameters for a switch
ficoncupshow Displays FICON-CUP parameters for a switch
ficonhelp Displays FICON commands
ficonshow Displays contents of the specified FICON management
database
fipscfg Configure FIPS mode operation
firmwarecommit Commit firmware to stable storage
firmwaredownload Download firmware into switch
firmwaredownloadstatus Display the progress and status of firmwareDownload
firmwarekeyshow Manage the keys used to check firmware signatures
firmwarekeyupdate Manage the keys used to check firmware signatures
firmwarerestore Restore the old firmware in the switch
firmwareshow Display firmware versions in the switch
fmconfig Configure Frame Monitors
fosconfig Enable/disable FabOS services
fosexec Invoke command in alternate VF context
framerpathtest Tests data path between ASICs
frureplace Replace a Field Replacable Unit
fspfshow Print FSPF global information
fwalarmsfilterset Configure alarms filtering for Fabric Watch
fwalarmsfiltershow Show alarms filtering for Fabric Watch
fwclassinit Initialize all Fabric Watch classes
fwconfigreload Reload Fabric Watch configuration
fwconfigure Configure Fabric Watch
fwfrucfg Configure FRU state and notification
fwhelp Print Fabric Watch help info
fwmailcfg Configure Fabric Watch Email Alert
fwportdetailshow Create a report with detailed port information
fwsamshow Show availability monitor information
fwset Set port persistence time
fwsettocustom Set boundary & alarm level to custom
fwsettodefault Set boundary & alarm level to default
fwshow Show thresholds monitored and port persistence time
grep Linux grep command
h Print shell history
hadisable Disables the High Availability feature in the
switch.
hadump Displays information about the status of the High
Availability feature in the switch.
haenable Enables the High Availability feature in the switch
hafailover Forces the failover mechanism so that the Standby
Control Processor (CP)becomes the Active CP.
hashow Displays control processor (CP) status
hasyncstart Enables HA state synchronization.
hasyncstop Disables the HA state synchronization.
help Print this list
historylastshow Displays last history record
historymode Display or set the mode of the history log
historyshow Displays entire history log
i Display process summary
iclcfg Configure ICL state
ifmodeset Set the link operating mode for a network interface
ifmodeshow Display the link operating mode for a network
interface
interfaceshow Display the FSPF (TM) interface information
interopmode Displays/enables/disables Brocade switch
interoperability with switches from other
manufacturers
ioddelayreset Configure in-order-delivery (IOD) delay parameter
ioddelayshow Configure in-order-delivery (IOD) delay parameter
iodreset Turn off the in-order delivery (IOD) option
iodset Manage the in-order delivery (IOD) option
iodshow Display the state of the in-order delivery option
ipaddrset Set ethernet and FC IP addresses
ipaddrshow Print ethernet and FC IP addresses
ipfilter Manage/display IP filters
ipsecconfig Configure IPSec feature
iscsihelp Print iSCSI help info
islshow Display the current connections and status
killtelnet Terminate telnet/serial login sessions
interactively
ldapcfg Configure LDAP role mapping
ledtest Cycle system LEDs.
lfcfg Configure logical fabrics
lfexec Invoke command in alternate VF context
licenseadd Adds license keys to switch
licensehelp Print license help info
licenseidshow Displays the system license ID
licenseport Configure per-port POD license assignments
licenseremove Removes a license key from this system
licenseshow Displays current license keys
licenseslotcfg Configure licenses on a slot basis
linkcost Set or print the FSPF cost of a link
login Login as a new user
logout Logout from remote session
lscfg Configure logical switches
lsdbshow Displays the FSPF link state database
memshow Display memory usage in the system.
minispropshow Displays ASIC pair properties.
more Linux more command
mscapabilityshow Display Management Server Capability.
msconfigure Configure Management Server
msfr Create a MSFR Zone
msplatshow Display the Management Server Platform Database
msplatshowdbcb Display the Management Server Database Control
Block
msplcleardb Clear the Management Server Platform Database
fabric-wide
msplmgmtactivate Activate the Management Server Platform Database
fabric-wide
msplmgmtdeactivate Deactivate the Management Server Platform Database
fabric-wide
mstddisable Disables the Management Server Topology Discovery
Management Service
mstdenable Enables the Management Server Topology Discovery
Management Service
mstdreadconfig Display status of Management Server Topology
Discovery Service
myid Display the current login session details
nbrstateshow Display FSPF (TM) neighbors' states
nbrstatsclear Reset the FSPF (TM) interface counters
netstat Display network connections and statistics
nodefind Display all the devices' PID matching the given
world wide name
nsaliasshow Display local Name Server information with Aliases
nsallshow Print global Name Server information
nscamshow Print local Name Server Cache information
nsshow Print local Name Server information information
nszonemember Display the information of all the online devices
which are zoned with the given device.
openfr Create a MSFR Zone
opensource Display information on open source software
utilized in FabOS
passwd Set usernames and passwords
passwdcfg passwdcfg manages the password policies.
pathinfo Display routing information between two ports
pdshow Show information from Panic Dump file
perfaddeemonitor Add end-to-end monitor
perfaddipmonitor Add monitor for IP trafic frame count
perfaddreadmonitor Add filter-based monitor - SCSI Read
perfaddrwmonitor Add monitor - SCSI Read and Write
perfaddscsimonitor Add monitor for SCSI frame count
perfaddusermonitor Add filter-based monitor
perfaddwritemonitor Add filter-based monitor - SCSI Write
perfcfgclear Clear Performance settings from FLASH
perfcfgrestore Restore Performance configuration from FLASH
perfcfgsave Save Performance configuration to FLASH
perfclearalpacrc Clears the CRC error count associated with a port
and Arbitrated Loop Physical Address (AL_PA).
perfcleareemonitor Clears statistics counters of one or more
end-to-end performance monitors.
perfclearfiltermonitor Clears counters of one or more filter-based
performance monitors.
perfdeleemonitor Delete end-to-end monitor
perfdelfiltermonitor Delete filter-based monitor
perfhelp Print Performance Monitoring help info
perfmonitorclear Clear end-to-end/filter-based/ISL monitors
perfmonitorshow Show end-to-end/filter-based/ISL monitors
perfresourceshow Show monitor availability
perfsetporteemask Set overall mask for end-to-end monitors
perfshowalpacrc Get ALPA CRC count by port and ALPA
perfshoweemonitor Show user-defined end-to-end monitors
perfshowfiltermonitor Show filter-based monitors
perfshowporteemask Show the current end-to-end mask
perfttmon Configure bandwidth usage monitor
ping Display the link operating mode for a network
interface
ping6 Display the link operating mode for a network
interface
pkicreate Creates public key infrastructure (PKI) objects
pkiremove Removes existing public key infrastructure (PKI)
objects
pkishow Displays existing public key infrastructure (PKI)
objects
policy Configure IPSec encryption and authentication
algorithms
portaddress Configure low 16 bits of Fibre Channel Porrt ID
portalpashow Show the ALPAs of a port
portbeacon Enable or disable port beacon
portbuffershow Print the buffer allocations for the ports of a
quad
portcamshow Display filter utilization
portcfg Create/Delete a new ip interface/route/arp entry on
the GigE port
portcfgalpa Set AL_PA to 0x0 (default) or 0x13
portcfgautodisable Configure auto disable
portcfgcreditrecovery Configure credit recovery
portcfgdefault Restore the port configuration to defaults
portcfgeport Enable/Disable a port from becoming E_Port
portcfgexport Sets a port to be an EX_Port and sets and displays
EX_Port configuration parameters.
portcfgfillword Configure a port to a particular fill word
portcfgfportbuffers Configure F_Port buffers
portcfggemediatype Configure media type for GigE port
portcfggport Lock a port as a G_Port
portcfgislmode Configure a port to be ISL R_RDY Mode
enabled/disabled
portcfglongdistance Configure a Long Distance Port
portcfglosstov Configure frontend port signal debounce
portcfglport Lock a port as a L_Port
portcfgnpivport Configure NPIV functionality on a port.
portcfgnport Enable/disable N_Port capability for a port
portcfgpersistentdisable Persistently disable a port
portcfgpersistentenable Enable a persistently disabled port
portcfgqos Configure QoS feature
portcfgshow Displays port configuration settings.
portcfgspeed Configure a port to a particular speed level
portcfgtrunkport Configure a port for trunking
portcfgvexport Sets a port to be an VEX_Port connected to FC-IP
and sets and displays VEX_Port configuration
parameters.
portcmd Execute commands (ping etc) on the GigE port
portdebug Set debug level and verbosity level of port modules
portdisable Disable a specified port
portenable Enable a specified port
porterrshow Print port error summary
portfencing Configure Fabric Watch port fencing feature
portflagsshow Display the port status bitmaps of all ports in a
switch
portledtest Cycle user port LEDs.
portlogclear Clear port activity log
portlogconfigshow Display portlog configuration
portlogdisable Disable portlog facility
portlogdump Print port log (no page breaks)
portlogdumpport Print port log (no page breaks)
portlogenable Enables port log facility
portlogeventshow Display information about port log events
portloginshow Display port login information
portlogpdisc Set or clear the debug_pdisc_flag
portlogreset Enable portlog facility
portlogresize Resize the port log
portlogshow Print port activity log
portlogshowport Print port activity log
portlogtypedisable Disable port logging
portlogtypeenable Enable port logging
portloopbacktest Functional test of port N->N path.
portmirror Set a port's configuration to be disabled or
enabled
portname Assign or display the name associated with a port
portperfshow Print port throughput numbers
portrouteshow Display various routing tables for a port
portshow Show configured ip interfaces/routes/arp entries on
the GigE Port
portstats64show Display the 64-bit hardware statistics for a port
portstatsclear Clear the status of a specified switch port
portstatsshow Print hardware statistics
portswap Swaps area numbers
portswapdisable Disable portswap feature
portswapenable Enable portswap feature
portswapshow Display port and area information
porttest Functional test on a live fabric. Starts porttest.
porttestshow Retreive information from porttest.
portthconfig Configure Fabric Watch port threshold monitor
porttrunkarea Configure area trunking
portzoneshow Displays the enforced zone type of the F/FL-ports
of a switch.
powerofflistset sets slot power off list order
powerofflistshow Displays slot power-off order.
psshow Print power supply status
reboot Reboot this processor
routehelp Print routing help info
saschiptest Functional Test of Components in SAS Complex.
saspathtest Data Path Test of the SAS complex.
secactivesize Displays size of the active (security) database
secauthcertificate Configure FCAP switch certificate properties
secauthsecret Creates/Manages DHCHAP secret key details
seccertutil Creates/Manages/Displays third party PKI
certificates
secdefinesize Displays size of the defined (security) database
secglobalshow Displays current internal security state
information
sechelp Displays information about security commands
secpolicyabort Aborts changes to defined policy
secpolicyactivate Activates all policy sets
secpolicyadd Adds members to an existing policy
secpolicycreate Creates a new policy
secpolicydelete Deletes an existing policy
secpolicydump Displays all members of existing policies
secpolicyfcsmove Moves a member in the FCS policy
secpolicyremove Removes members from an existing policy
secpolicysave Saves defined policy set and sends to all switches
secpolicyshow Shows members of one or more policies
secstatsreset Resets security statistics
secstatsshow Displays security statistics
sensorshow Display sensor readings
setdbg Set debug level of the specified module
setmodem Enables or disables modem login to a control
processor (CP)
setverbose Set verbosity level of the specified module
sfpshow Print Serial ID SFP information
shellflowcontroldisable Disable xon-xoff flow control on console serial
port
shellflowcontrolenable Enable xon-xoff flow control on console serial port
sleep Linux sleep command
slotpoweroff Removes power from a slot.
slotpoweron Restores power to a slot.
slotshow Displays the status of all slots in the system.
snmpconfig Configure SNMP
snmpmibcapset Set options for configuring SNMP Mib/Trap
Capability
snmpmibcapshow Show SNMP Mib/Trap Configuration
snmptraps Configure SNMP traps
spinfab Functional test of switch to switch ISL cabling and
trunk group operation.
sshutil Configure SSH authentication options
statsclear Clear port and diagnostic statistics.
stopporttest Terminate the running porttest.
supportffdc Modifies or displays first-failure data capture
(FFDC) configuration.
supportftp set support Ftp parameters
supportsave retrieve support data from switch to host
supportshow Prints switch information for debugging purposes.
supportshowcfgdisable Disables a group of commands under supportshow
command.
supportshowcfgenable Enables a group of commands under supportshow
command.
supportshowcfgshow Displays the groups of commands enabled for
display. by the supportshow command.
switchbeacon Set switch beacon on or off
switchcfgpersistentdisable Persistently disable a switch
switchcfgpersistentenable Enable a persistently disabled switch
switchcfgspeed Configures all ports of the switch to a particular
speed level
switchcfgtrunk Configure all ports on the switch for trunking
switchdisable Disable this switch
switchenable Enable this switch
switchname Print/set this switch's name
switchshow Print switch and port status
switchstatuspolicyset Set policy parameters for overall switch status
switchstatuspolicyshow Print policy parameters for overall switch status
switchstatusshow Print overall switch status
switchuptime Display the amount of time for which the switch is
up
switchviolation Display policy violations recorded in RAS log
syslogdfacility Change the syslog facility
syslogdipadd Add syslog daemon IP address
syslogdipremove Remove syslog daemon IP address
syslogdipshow Print the syslog daemon targets
sysmonitor Configure system memory usage monitor
sysshutdown Provides a graceful shutdown to protect the switch
file systems.
tempshow Print temperature readings
thconfig Configure Fabric Watch thresholds
timeout Set/show the IDLE timeout value for a login session
top Linux top command
topologyshow Display the unicast fabric topology
tracedump generate/remove trace dump or display trace dump
status
trackchangeshelp Print Track Changes help info
trackchangesset Configure alert for login/logout/config update
trackchangesshow Displays status of track changes
trunkdebug Debug a trunk link failure
trunkshow Display trunking information
tsclockserver Displays or sets the NTP server address
tstimezone Displays or sets the time zone
turboramtest Turbo SRAM test for bloom ASICs.
uptime Print how long switch has been up
urouteconfig Configure a static route
urouteremove Remove a static route
urouteshow Display unicast routing information
usbstorage Manage USB storage device (available on Brocade
Data Center Director only).
userconfig Display or configure user accounts
userrename Rename user login name
version Print firmware version
wwn Display or set the world wide name
wwnaddress Configure persistent PID assignments
zone Configure zone objects
zoneadd Add a member to a zone
zonecreate Create a zone
zonedelete Delete a zone
zonehelp Print zoning help info
zoneobjectcopy Copies a zone object
zoneobjectexpunge Expunges a zone object
zoneobjectrename Rename a zoning Object
zoneremove Remove a member from a zone
zoneshow Print zone information

TEST1BRO1:FID128:admin>
cfgshow
Defined configuration:
cfg: gdscioffice
backup; clunn1; clunn2; esx5; esx6; exch_backup; exch_phy_1;
exch_phy_2; exch_xen1; exch_xen2; os7015t3P1; test1xenfe2p1_p1;
test1xenfe2p2_p1; test1xenfe2p3_p1; test1xenfe2p4_p1;
test1xenfe2p5_p1; test1xenvmpp2; vtcxmarkham_05; vtcxmarkham_07
zone: backup 1,22; 1,23; ibmtapedrive2; netapp_a_p2; netapp_b_p2;
test1bkpoffp
zone: clunn1 1,22; 1,23; clun1; netapp_a_p2; netapp_b_p2
zone: clunn2 1,22; 1,23; clun2; netapp_a_p2; netapp_b_p2
zone: esx5 1,22; 1,23; esx5_p2; netapp_a_p2; netapp_b_p2
zone: esx6 1,27; 1,31; esx6_p2; netapp_a_p2; netapp_b_p2
zone: exch_backup
1,12; 1,15; 1,21; 1,25; 1,27; 1,31; netapp_b_p2
zone: exch_phy_1
1,14; 1,15; 1,25
zone: exch_phy_2
1,15; 1,20; 1,25
zone: exch_xen1
1,15; 1,24; 1,25
zone: exch_xen2
1,15; 1,25; 1,26
zone: os7015t3P1
1,27; 1,31; netapp_a_p2; netapp_b_p2; os7015t3
zone: test1xenfe2p1_p1
1,22; 1,23; netapp_a_p2; netapp_b_p2; test1xenfe2p1
zone: test1xenfe2p2_p1
1,22; 1,23; netapp_a_p2; netapp_b_p2; test1xenfe2p2
zone: test1xenfe2p3_p1
1,27; 1,31; netapp_a_p2; netapp_b_p2; test1xenfe2p3
zone: test1xenfe2p4_p1
1,22; 1,23; netapp_a_p2; netapp_b_p2; test1xenfe2p4
zone: test1xenfe2p5_p1
1,22; 1,23; netapp_a_p2; netapp_b_p2; test1xenfe2p5
zone: test1xenvmpp2
1,15; 1,22; 1,23; 1,25; netapp_a_p2; netapp_b_p2;
netbackupold; test1xenvmp
zone: vtcxmarkham_05
1,27; 1,31; netapp_a_p2; netapp_b_p2; vtcxmarkham05
zone: vtcxmarkham_07
1,17; 1,27; 1,31; netapp_a_p2; netapp_b_p2
alias: clun1 1,10
alias: clun2 1,11
alias: esx5_p2 1,19
alias: esx6_p2 1,18
alias: ibmtapedrive2
1,1
alias: netapp_a_p2
1,7
alias: netapp_b_p2
1,8
alias: netbackupold
1,2
alias: os7015t3
1,9
alias: test1bkpoffp
1,0
alias: test1xenfe2p1
1,5
alias: test1xenfe2p2
1,28
alias: test1xenfe2p3
1,6
alias: test1xenfe2p4
1,4
alias: test1xenfe2p5
1,29
alias: test1xenvmp
1,3
alias: vtcxmarkham05
1,16
alias: vtcxmarkham07
1,14

Effective configuration:
cfg: gdscioffice
zone: backup 1,22
1,23
1,1
1,7
1,8
1,0
zone: clunn1 1,22
1,23
1,10
1,7
1,8
zone: clunn2 1,22
1,23
1,11
1,7
1,8
zone: esx5 1,22
1,23
1,19
1,7
1,8
zone: esx6 1,27
1,31
1,18
1,7
1,8
zone: exch_backup
1,12
1,15
1,21
1,25
1,27
1,31
1,8
zone: exch_phy_1
1,14
1,15
1,25
zone: exch_phy_2
1,15
1,20
1,25
zone: exch_xen1
1,15
1,24
1,25
zone: exch_xen2
1,15
1,25
1,26
zone: os7015t3P1
1,27
1,31
1,7
1,8
1,9
zone: test1xenfe2p1_p1
1,22
1,23
1,7
1,8
1,5
zone: test1xenfe2p2_p1
1,22
1,23
1,7
1,8
1,28
zone: test1xenfe2p3_p1
1,27
1,31
1,7
1,8
1,6
zone: test1xenfe2p4_p1
1,22
1,23
1,7
1,8
1,4
zone: test1xenfe2p5_p1
1,22
1,23
1,7
1,8
1,29
zone: test1xenvmpp2
1,15
1,22
1,23
1,25
1,7
1,8
1,2
1,3
zone: vtcxmarkham_05
1,27
1,31
1,7
1,8
1,16
zone: vtcxmarkham_07
1,17
1,27
1,31
1,7
1,8

TEST1BRO1:FID128:admin>
TEST1BRO1:FID128:admin>
thconfig
Fabric Watch is not licensed.
TEST1BRO1:FID128:admin>
TEST1BRO1:FID128:admin>
switchshow
switchName: TEST1BRO1
switchType: 66.1
switchState: Online
switchMode: Native
switchRole: Principal
switchDomain: 1
switchId: fffc01
switchWwn: 10:00:00:05:33:7d:b2:b0
zoning: ON (gdscioffice)
switchBeacon: OFF
FC Router: OFF
Allow XISL Use: OFF
LS Attributes: [FID: 128, Base Switch: No, Default Switch: Yes, Address Mode 0]

Index Port Address Media Speed State Proto
==============================================
0 0 010000 id N8 Online FC F-Port 21:00:00:24:ff:25:34:61
1 1 010100 id N8 Online FC L-Port 1 public
2 2 010200 id N8 No_Light FC
3 3 010300 id N8 Online FC F-Port 21:00:00:24:ff:25:33:fe
4 4 010400 id N8 Online FC F-Port 50:01:43:80:24:29:6b:90
5 5 010500 id N8 Online FC F-Port 50:01:43:80:24:29:5d:10
6 6 010600 id N8 Online FC F-Port 50:01:43:80:24:29:60:44
7 7 010700 id N8 No_Light FC
8 8 010800 id N8 No_Light FC
9 9 010900 id N4 Online FC F-Port 21:01:00:1b:32:b7:c7:f4
10 10 010a00 id N8 Online FC F-Port 50:01:43:80:24:29:5e:be
11 11 010b00 id N8 Online FC F-Port 50:01:43:80:24:29:60:ee
12 12 010c00 id N8 Online FC F-Port 10:00:00:90:fa:cb:b7:f7
13 13 010d00 -- N8 No_Module FC
14 14 010e00 id N8 Online FC F-Port 10:00:00:90:fa:cb:c2:10
15 15 010f00 id N8 Online FC F-Port 1 N Port + 3 NPIV public
16 16 011000 id N8 Online FC F-Port 50:01:43:80:24:29:5e:74
17 17 011100 id N8 Online FC F-Port 50:01:43:80:24:29:61:30
18 18 011200 id N8 Online FC F-Port 21:00:00:24:ff:32:6c:c5
19 19 011300 id N8 Online FC F-Port 21:00:00:24:ff:25:33:5a
20 20 011400 id N8 Online FC F-Port 10:00:00:90:fa:cb:c2:5e
21 21 011500 id N8 Online FC L-Port 1 public
22 22 011600 id N8 Online FC F-Port 1 N Port + 2 NPIV public
23 23 011700 id N8 Online FC F-Port 1 N Port + 2 NPIV public
24 24 011800 id N8 Online FC F-Port 10:00:00:90:fa:cb:b7:de
25 25 011900 id N8 Online FC F-Port 1 N Port + 3 NPIV public
26 26 011a00 id N8 Online FC F-Port 10:00:00:90:fa:cb:df:5f
27 27 011b00 id N8 Online FC F-Port 1 N Port + 2 NPIV public
28 28 011c00 id N8 Online FC F-Port 50:01:43:80:28:cf:97:9e
29 29 011d00 id N8 Online FC F-Port 50:01:43:80:33:16:c2:2c
30 30 011e00 id N8 No_Light FC
31 31 011f00 id N8 Online FC F-Port 1 N Port + 2 NPIV public
32 32 012000 -- N8 No_Module FC (No POD License) Disabled
33 33 012100 -- N8 No_Module FC (No POD License) Disabled
34 34 012200 -- N8 No_Module FC (No POD License) Disabled
35 35 012300 -- N8 No_Module FC (No POD License) Disabled
36 36 012400 -- N8 No_Module FC (No POD License) Disabled
37 37 012500 -- N8 No_Module FC (No POD License) Disabled
38 38 012600 -- N8 No_Module FC (No POD License) Disabled
39 39 012700 -- N8 No_Module FC (No POD License) Disabled

TEST1BRO1:FID128:admin>
switchname
TEST1BRO1
TEST1BRO1:FID128:admin> 


2. Through Web Tools in Firefox

It cost me two hours to find out a solution for v6.4.1b Web Tools working in my browsers. Here is what I experienced. I was using latest Java 1.8 and quickly found it does not work properly. Based on  Internet search results, 1.7 or 1.6 suppose to work. Quickly installed both and enabled 1.6 / 1.7 , but it seems only 1.7 works with special tuning.
Java Runtime Environment Settings

It needs special url, not default http://<Brocade SW ip address>. It has to be http://<Brocade SW IP Address>/switchExplorer_installed.html. If you are using wrong url, you will always be directed to Java download site. 

Even with 1.7 and with this special url, I still experienced famous "Failed to validate certificate" error. I tried to disable certificate verification, low down Java security level to medium. 



But still same. It downloaded application, but failed to launch it because certificate could not be verified.



Failed validate certificate


More information on Failed to validate certificate error


Eventually I found another small tuning on file java.security located in your windows machine's c:/program files/Java/JRE installed directory (jre/lib/security/java.security).

searching for this line:

jdk.certpath.disabledAlgorithms=MD2, RSA keySize < 1024

Then change 1024 to 256 and save.


Finanlly got it working.
Brocade Web Tools Verifying Application Window


Brocade Switch Login Window


Initializing Brocade Web Tools

Finally, Brocade Web Tools Interface shows up. 
Brocade Web Tools Interface










Monday, March 27, 2017

Cisco Switch 2960x Memory Increasing Issue Troubleshooting - Memory Leak

Symptoms 

Our network monitoring software found memory usage on some new production switches keep increasing. Those switches are Cisco 2960X and coming with 15.0(2) EX3 IOS.

As we know there are two types of memory in Cisco IOS: process memory and IO memory.
  • When a feature is enable on an IOS device (e.g. PIM, HSRP, and etc), IOS allocates process memory for the process.
  • IO memory is used when a software switched traffic hits the CPU. The CPU allocates IO memory to store the frame temporarily.
Our case is process memory increasing. What we need to do is to find out which process.




Switch1# show version 
Cisco IOS Software, C2960X Software (C2960X-UNIVERSALK9-M), Version 15.0(2)EX3, RELEASE SOFTWARE (fc1)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2013 by Cisco Systems, Inc.
Compiled Wed 11-Sep-13 02:04 by prod_rel_team

ROM: Bootstrap program is C2960X boot loader
BOOTLDR: C2960X Boot Loader (C2960X-HBOOT-M) Version 15.0(2r)EX, RELEASE SOFTWARE (fc1)

CSWP3024 uptime is 5 days, 23 hours, 48 minutes
System returned to ROM by power-on
System restarted at 05:47:41 EST Sat Mar 11 2017
System image file is "flash:/c2960x-universalk9-mz.150-2.EX3/
c2960x-universalk9-mz.150-2.EX3.bin"


This product contains cryptographic features and is subject to United
States and local country laws governing import, export, transfer and
use. Delivery of Cisco cryptographic products does not imply
third-party authority to import, export, distribute or use encryption.
Importers, exporters, distributors and users are responsible for
compliance with U.S. and local country laws. By using this product you
agree to comply with applicable laws and regulations. If you are unable
to comply with U.S. and local laws, return this product immediately.

A summary of U.S. laws governing Cisco cryptographic products may be found at:
http://www.cisco.com/wwl/export/crypto/tool/stqrg.html

If you require further assistance please contact us by sending email to
export@cisco.com.

cisco WS-C2960X-48TS-LL (APM86XXX) processor (revision B0) with 262144K bytes of memory.
Processor board ID FOC1744W3V5
Last reset from power-on
2 Virtual Ethernet interfaces
1 FastEthernet interface
50 Gigabit Ethernet interfaces
The password-recovery mechanism is enabled.

512K bytes of flash-simulated non-volatile configuration memory.
Base ethernet MAC Address : 5C:A4:8A:5C:6F:80
Motherboard assembly number : 73-14196-03
Power supply part number : 341-0537-02
Motherboard serial number : FOC17443CWD
Power supply serial number : LIT17320EU6
Model revision number : B0
Motherboard revision number : B0
Model number : WS-C2960X-48TS-LL
Daughterboard assembly number : 73-14200-03
Daughterboard serial number : FOC17440VRF
System serial number : FOC1744W3V5
Top Assembly Part Number : 800-38011-02
Top Assembly Revision Number : B0
Version ID : V02
CLEI Code Number : CMMNG00ARA
Daughterboard revision number : A0
Hardware Board Revision Number : 0x05


Switch Ports Model SW Version SW Image
------ ----- ----- ---------- ----------
* 1 50 WS-C2960X-48TS-LL 15.0(2)EX3 C2960X-UNIVERSALK9-M


Configuration register is 0xF

Switch1#
show memory statistics history
------------------ History of Processor Mempool ------------------




777777777777777777777777777777777777777777777777777777777777
222222222222222222222222222222222222222222222222222222222222
100
90
80
70 **********************************************************
60 **********************************************************
50 **********************************************************
40 **********************************************************
30 **********************************************************
20 **********************************************************
10 **********************************************************
0....5....1....1....2....2....3....3....4....4....5....5....6
0 5 0 5 0 5 0 5 0 5 0
Free memory per second (last 60 seconds)




777777777777777777777777777777777777777777777777777777777777
222222222222222222222222222222222222222222222222222222222222
100
90
80
70 ##########################################################
60 ##########################################################
50 ##########################################################
40 ##########################################################
30 ##########################################################
20 ##########################################################
10 ##########################################################
0....5....1....1....2....2....3....3....4....4....5....5....6
0 5 0 5 0 5 0 5 0 5 0
Free memory per minute (last 60 minutes)
* = maximum # = average




777777777777777777777777777777777777777777777777777777777777777777777777
222222222233333333333333444444444444444555555555555555666666666666667777
100
90
80 **#############################
70 ######################################################################
60 ######################################################################
50 ######################################################################
40 ######################################################################
30 ######################################################################
20 ######################################################################
10 ######################################################################
0....5....1....1....2....2....3....3....4....4....5....5....6....6....7..
0 5 0 5 0 5 0 5 0 5 0 5 0
Free memory per hour (last 72 hours)
* = maximum # = average


------------------ History of I/O Mempool ------------------




555555555555555555555555555555555555555555555555555555555555
888888888888888888888888888888888888888888888888888888888888
100
90
80
70
60 **********************************************************
50 **********************************************************
40 **********************************************************
30 **********************************************************
20 **********************************************************
10 **********************************************************
0....5....1....1....2....2....3....3....4....4....5....5....6
0 5 0 5 0 5 0 5 0 5 0
Free memory per second (last 60 seconds)




555555555555555555555555555555555555555555555555555555555555
888888888888888888888888888888888888888888888888888888888888
100
90
80
70
60 ##########################################################
50 ##########################################################
40 ##########################################################
30 ##########################################################
20 ##########################################################
10 ##########################################################
0....5....1....1....2....2....3....3....4....4....5....5....6
0 5 0 5 0 5 0 5 0 5 0
Free memory per minute (last 60 minutes)
* = maximum # = average




555555555555555555555555555555555555555555555555555555555555555555555555
888888888888888888888888888888888888888888888888888888888888888888888888
100
90
80
70
60 ######################################################################
50 ######################################################################
40 ######################################################################
30 ######################################################################
20 ######################################################################
10 ######################################################################
0....5....1....1....2....2....3....3....4....4....5....5....6....6....7..
0 5 0 5 0 5 0 5 0 5 0 5 0
Free memory per hour (last 72 hours)
* = maximum # = average


switch1#show controllers utilization
Port Receive Utilization Transmit Utilization
Gi1/0/1 0 0
Gi1/0/2 0 0
Gi1/0/3 0 0
Gi1/0/4 0 0
Gi1/0/5 0 0
Gi1/0/6 0 0
Gi1/0/7 0 0
Gi1/0/8 0 0
Gi1/0/9 0 0
Gi1/0/10 0 0
Gi1/0/11 0 0
Gi1/0/12 0 0
Gi1/0/13 0 0
Gi1/0/14 0 1
Gi1/0/15 0 0
Gi1/0/16 0 0
Gi1/0/17 0 0
Gi1/0/18 0 0
Gi1/0/19 0 0
Gi1/0/20 0 0
Gi1/0/21 0 0
Gi1/0/22 0 0
Gi1/0/23 0 0
Gi1/0/24 5 4
Gi1/0/25 0 0
Gi1/0/26 0 0
Te1/0/1 0 0
Te1/0/2 0 0

Total Ports : 28
Switch Receive Bandwidth Percentage Utilization : 0
Switch Transmit Bandwidth Percentage Utilization : 0

Stack Ring Percentage Utilization : 0




From our management software , the free memory diagram shows it is continuous reducing until whole system lost responding .

Free Switch Memory Going Down in Monitoring Tools

Troubleshooting

There are two main reasons why a switch runs out of memory.

1. A process does not free up the memory after the memory is no longer needed. This is generally known as memory leak.

2. A process does not limit the amount of memory it allocates. Eventually, the process uses up all process memory on the switch.

The goal is to identify the misbehaving process. There are two approaches.
  •  Identify any process leaking memory
Show process memory debug leak  

Notes: This command only to be run during maintenance window or when the device is not in production .This command is handy to identify any process leaking memory. However, it does not help if the memory problem is caused by a process grabs all memory
  • Identify a process grabs all memory 


Potential fix:

After googled and I did find many related Cisco bug and suggestions.

1.  Cisco Bug: CSCut06609 - Memory leak seen in Auth Manager process- leaks from ASP component (Not our case)
https://quickview.cloudapps.cisco.com/quickview/bug/CSCut06609

Last Modified

Jan 31, 2017

Product

Cisco IOS

Known Affected Releases

15.0(2)SE5

Description (partial)

Symptom:
++ Very slow memory leak in "Auth Manager" process seen on 2960s/2960x switches running 15.0(2)SE5.
++ Everyday we see about 64KB leak in Auth Manager process.
++ Leak is seen on a setup having no "Auth session" present (Only auth hidden sessions) and no ports running MAB or dot1X. It is not same as CSCuq53377.
++ The leak is seen in "Auth Manager" process but it is actually being caused by ASP component.

Conditions:
"macro auto monitor" need to be enabled globally, it could be seen even when no switch port is running MAB/dot1x.
The Fix below fixed all out problems so far:

Apparently, in the IOS there's an "Auth manager" that can monitor all
sessions in a switch. Starting from the 15.0 IOS stream this feature is
enabled by default. As the bug describes:
"Auth Manager continues to hold more memory in Processor Pool locking
out access to the 3750 stack unless switch is rebooted"

To prevent this 'leakage' of memory to the processor, you have to
disable this session monitoring by issuing the following command:

no macro auto monitor

Tried this, but did not help in this case.

2. Cisco Bug: CSCut92215 - Memory Leak in hrpc_incoming_request_handler_process - 15.0(2)EX5 (Not our case)
https://quickview.cloudapps.cisco.com/quickview/bug/CSCut92215

Last Modified

Jan 23, 2017

Product

Cisco Catalyst 2960-X Series Switches

Known Affected Releases

15.0(2)EX

Description (partial)

Symptom:
Show process memory sorted will display MallocLite holding memory but no freeing it

SW#show process memory sorted

Processor Pool Total: XXXXXXXX Used: XXXXXXXX Free: XXXXXXX
I/O Pool Total: XXXXXXXX Used: XXXXXXXX Free: XXXXXXX
Driver te Pool Total: XXXXXXX Used: XX Free: XXXXXXX

PID TTY Allocated Freed Holding Getbufs Retbufs Process
0 0 XXXXXXXX XXXXXXXX XXXXXXXX 0 0 *Init*
0 0 X 0 XXXXXXXX 0 0 *MallocLite*


//////////////////
show mem lite-chunk statistics will show a large disparity between Inuse and Free


SW#show mem lite-chunk statistics

Pool Inuse Free
8-Bytes XXX XXXX
20-Bytes XXXXXX XXXX <<<<<<
44-Bytes XXXX XXX
68-Bytes XXX XXX
96-Bytes XXX XXX
128-Bytes XX XXX

Conditions:
SW running 15.0(2)EX5

3. SNMP disable (Not our case)
Suggestion to run following three commands to reduce memory usage
- no macro auto monitor
- no cdp running
- snmp completely disabled
Tried those three commands / methods, but still did not help.
4. Memory leak with hulc_flash init process (Our case)
Switch1#show processes memory sorted | i Total
Processor Pool Total: 174442388 Used: 50534880 Free:
123907508
I/O Pool Total: 16777216 Used: 6924400 Free: 9852816
Driver te Pool Total: 1048576 Used: 40 Free: 1048536
57447240 Total

Switch1#show processes memory sorted | i Total
Processor Pool Total: 174442388 Used: 50661608 Free:
123780780
I/O Pool Total: 16777216 Used: 6924400 Free: 9852816
Driver te Pool Total: 1048576 Used: 40 Free: 1048536
57573808 Total

You can see Total processor Pool Free memory is decreasing, Free I/O and Driver te Pool memory keep same.

Check more details from sorted processes memory command, *Dead* process memory is increasing by comparing the holding memory data with the data collected 5 minutes ago.
Switch1#show processes memory sorted 
Processor Pool Total: 174442388 Used: 50519544 Free: 123922844
I/O Pool Total: 16777216 Used: 6924400 Free: 9852816
Driver te Pool Total: 1048576 Used: 40 Free: 1048536

PID TTY Allocated Freed
Holding Getbufs Retbufs Process
0 0 59344172 24818044 31046036 0 0 *Init*
0 0 0 0 19543880 0 0 *MallocLite*
0 0 40461580 39067064 1530300 6881007 949166 *Dead*
87 0 715684 146228 451160 0 0 Stack Mgr Notifi
289 0 267228 0 277388 100548 0 EEM ED Syslog
1 0 234876 36256 205780 0 0 Chunk Manager
68 0 206732 232 188500 0 0 USB Startup
205 0 1520792 1070688 154044 0 0 802.1x switch
65 0 108532 400 118292 0 0 EEM ED Identity
180 0 153020 8916 109604 0 0 DHCPD Receive
102 0 93356 212 100304 53760 0 HRPC emac reques
153 1 3418792 3336728 94288 0 0 SSH Process
300 0 70460 0 77620 0 0 EEM Server
178 0 13629416 300 75732 802008 0 CDP Protocol
138 0 65812 232 72740 0 0 REP Helper Proc
53 0 108644 84688 71160 0 0 IF-MGR control p
175 0 530220 373064 62440 0 0 Auth Manager
128 0 232 2204 61160 0 0 HACL Acl Manager
159 0 499433728 499416280 61160 10152 0 HRPC qos request
173 0 49364 0 56524 0 0 ACCT Periodic Pr
280 0 237076 202296 48632 0 0 Crypto CA
21 0 48776 64728 45920 0 9620 Entity MIB API
17 0 211669728 211633028 43860 0 0 ARP Input
189 0 33020 0 40180 0 0 IP ARP Retry Age
297 0 12408 0 37440 0 0 LICENSE AGENT
230 0 50872 9248 34396 0 0 CEF: IPv4 proces
8 0 1547520 3356300 32952 1401600 2400103 Pool Manager

Switch1#show processes memory sorted          
Processor Pool Total: 174442388 Used: 50659912 Free: 123782476
I/O Pool Total: 16777216 Used: 6924400 Free: 9852816
Driver te Pool Total: 1048576 Used: 40 Free: 1048536

PID TTY Allocated Freed
Holding Getbufs Retbufs Process
0 0 59344172 24818044 31046036 0 0 *Init*
0 0 0 0 19675040 0 0 *MallocLite*
0 0 44037920 42642260 1531252 6881007 949166 *Dead*
87 0 715684 146228 451160 0 0 Stack Mgr Notifi
289 0 267228 0 277388 100548 0 EEM ED Syslog
1 0 234876 36256 205780 0 0 Chunk Manager
68 0 206732 232 188500 0 0 USB Startup
205 0 1520792 1070688 154044 0 0 802.1x switch
65 0 108532 400 118292 0 0 EEM ED Identity
180 0 153020 8916 109604 0 0 DHCPD Receive
102 0 93356 212 100304 53760 0 HRPC emac reques
153 1 193080 111648 93592 0 0 SSH Process
300 0 70460 0 77620 0 0 EEM Server
178 0 13658484 300 75732 802008 0 CDP Protocol
138 0 65812 232 72740 0 0 REP Helper Proc
53 0 108644 84688 71160 0 0 IF-MGR control p
175 0 530220 373064 62440 0 0 Auth Manager
159 0 510696628 510679180 61160 10152 0 HRPC qos request
128 0 232 2204 61160 0 0 HACL Acl Manager
173 0 49364 0 56524 0 0 ACCT Periodic Pr
280 0 237076 202296 48632 0 0 Crypto CA
21 0 48776 64728 45920 0 9620 Entity MIB API
17 0 212097396 212060696 43860 0 0 ARP Input
189 0 33020 0 40180 0 0 IP ARP Retry Age
297 0 12408 0 37440 0 0 LICENSE AGENT
230 0 50872 9248 34396 0 0 CEF: IPv4 proces
112 0 4349612 232 33052 0 0 HLFM address lea
8 0 1547520 3356300 32952 1401600 2400103 Pool Manager
321 0 288460 157320 31420 0 0 SNMP Traps
199 0 5724 0 30884 0 0 SMI Backup Proce
197 0 3160 592 28360 0 0 SMI Director DB
60 0 74608 52800 27056 0 0 Per-Second Jobs
306 0 232 0 25392 0 0 tHUB





From the command 'show memory dead totals' output showed there were many instances of hulc_flash init.

Switch1#show memory dead totals 
Head Total(b) Used(b) Free(b) Lowest(b) Largest(b)
Processor 49B2FA0 174442388 50508688 123933700 123791240 103785944
I/O E000000 16777216 6924444 9852772 9736080 9805164
Driver te 2B00000 1048576 44 1048532 1048532 1048532



Dead Proc Summary for: Processor

PC Total Count Name
0x020CC35C 942068 47 mifs NOR Sector
0x0036E178 201252 20 SSH Process
0x020D35AC 100220 5 mifs sector buffers
0x01B31C10 65580 1 AAA Large Chunk
0x0017E2D4 64696 517 hulc_flash init
0x020D6A08 25220 5 mifs cache_entry buffers
0x020CC91C 20044 1 mifs NOR Sector Info Pointers
0x0204DE2C 10576 1 master cfg mgr init process
0x02050388 10548 1 HTTP SSL PKI CERTIFICATE CREATE PROC
0x0099C134 9756 1 EEM Shell Director
0x020CC974 8656 10 hulc_flash init
0x0021CBE8 7316 54 Smartport macro autoexec subblock
0x02304ED0 6888 51 LLDP-MED local annex
0x020B0788 6188 1 hulc_flash init
0x01B5F2DC 4652 1 Watcher Info
0x022E3640 4100 52 LIM Index
0x014829EC 3716 3 OBFL UPTIME obfl0
0x020D3DE0 2780 5 hulc_flash init
0x020D3684 2720 5 mifs file_map buffers
0x02883318 2352 7 platform OBFL
0x00477224 2196 1 HTTP SSL PKI CERTIFICATE CREATE PROC
0x004ADE94 2124 1 HTTP SSL PKI CERTIFICATE CREATE PROC
0x020D3618 2040 5 mifs file_info buffers
0x014618C4 1796 5 OBFL UPTIME obfl0
0x020C7A68 1708 24 hulc_flash init
0x020C79F8 1668 24 hulc_flash init
0x020C79BC 1632 24 hulc_flash init

0x01B5E85C 1544 1 Watched Semapho
0x005C40A8 1460 4 HTTP SSL PKI CERTIFICATE CREATE PROC
0x020D6928 1020 15 hulc_flash init
0x020D3DA4 980 5 hulc_flash init
0x020CCA94 460 5 hulc_flash init

0x004ADD80 444 1 HTTP SSL PKI CERTIFICATE CREATE PROC
0x020CCB3C 400 5 hulc_flash init
0x020D08E0 340 5 hulc_flash init
0x020D6C40 340 5 hulc_flash init

0x000730D4 308 1 platform OBFL
0x000739B0 308 1 platform OBFL
0x00073814 308 1 platform OBFL
0x00073B14 308 1 platform OBFL
0x000732A4 308 1 platform OBFL
0x00072E5C 308 1 platform OBFL
0x00343B40 276 3 TTY timer block
0x0045A4EC 232 1 HTTP SSL PKI CERTIFICATE CREATE PROC
0x01489A94 216 1 OBFL Infra Device info
0x01B31A48 208 1 AAA Large Chunk
0x00C1869C 144 1 HTTP
0x003877F0 136 1 Parser Mode
0x020B0734 112 1 hulc_flash init
0x00BF1E80 104 1 HTTP
0x00BFE2FC 100 1 HTTP
0x0059FE88 92 1 crypto_handle_table
0x00BF1B6C 84 1 HTTP
0x020B0808 80 1 hulc_flash init
0x00363AB4 76 1 Parser Linkage
0x002FD83C 68 1 AAA MI SG NAME
0x0038785C 68 1 Parser Mode Q1
0x00387880 68 1 Parser Mode Q2


Dead Proc Summary for: I/O

PC Total Count Name


Dead Proc Summary for: Driver text

PC Total Count Name


Switch1#show memory debug leak
Adding blocks for GD...

Driver text memory


Address Size Alloc_pc PID Alloc-Proc Name

I/O memory


Address Size Alloc_pc PID Alloc-Proc Name

Processor memory


Address Size Alloc_pc PID Alloc-Proc Name
Switch1#

Unfortunately, 'show memory debug leaks' did not find any process having leak issue.

I were able to find a post from High Memory Utilization due to hulc_flash to mention CSCth60511 and CSCua52463 are relating to this leak. It seems upgrade will fix this issue, but he did not post if issue resolved after upgraded.

Here is what I searched from Cisco Bug Search Website and there are three related bugs:
Cisco Bug Search Tool
Results from Cisco Bug Search Tool




4.1 CSCth60511 - Memory leak seen at hulc_flash init process

Description
Symptom :

Upon bootup a switch might leak a small amount of memory for hulc_flash init
Process


Further Info :

Since the leak is small it does not have any functionality impact.
Customer Visible

Was the description about this Bug Helpful?
(0)
Last Modified:
Feb 6,2017
Status:
Fixed
Severity:
3 Moderate
Product:
(1)
Cisco Catalyst 3750-E Series Switches
Support Cases:
14
Known Affected Releases:
(4)
12.2
15.0DPA
15.0DPB
15.0SID
Known Fixed Releases:
(23)
15.2(6.0.21i)E
15.2(5.1.57i)E
15.2(5.0)ST
15.2(4.0.64a)E
15.2(4.0)ST
15.2(1.30)PSR
15.2(1.1)EY
15.1(2)SG1.90
15.1(1)SG5.170
15.1(1)SG5.169
15.1(1)SG5.163
15.1(1)SG5.124
15.1(1)SG5.103
15.1(1)SG5.83.1
15.1(1)SG5.78.11
15.1(1)SG5.75
15.1(1)SG5.74
15.1(1)SG15.3
15.0(5.2)DPB1
15.0(4.6)SID
15.0(2)SE
15.0(2)EX
15.0(1)SE


Unfortunately our devices is 2960x and running on 15.0(2)EX3 , which is different from what this bug says.


4.2 CSCua52463 - C2960 - Memory leak in dead memory - not able to telnet/ssh to switch

C2960 - Memory leak in dead memory - not able to telnet/ssh to switch
CSCua52463
C2960 is having memory leak in *dead* memory.
Problem observed on multiple switches.

arnsw0034#sh mem dead
Head Total(b) Used(b) Free(b) Lowest(b) Largest(b)
Processor 25D8FA0 23727744 19629156 4098588 3362220 4034820
I/O 2C00000 4194304 2473496 1720808 1646820 1609296
Driver te 1800000 1048576 44 1048532 1048532 1048532

-----------

arnsw0034#sh proc mem sor
Processor Pool Total: 23727744 Used: 19628532 Free: 4099212
I/O Pool Total: 4194304 Used: 2473356 Free: 1720948
Driver te Pool Total: 1048576 Used: 40 Free: 1048536

PID TTY Allocated Freed Holding Getbufs Retbufs Process
0 0 22527504 4649688 14708156 0 0 *Init*
0 0 3014844 1663092 1290864 3804375 1341026 *Dead*
150 0 14687348 8409544 659316 2268 0 Auth Manager
71 0 736300 143256 472556 0 0 Stack Mgr Notifi
0 0 0 0 394484 0 0 *MallocLite*
157 0 25864596 43517164 335060 0 0 EAP Framework
179 0 2289784 23580 261832 0 0 CDP Protocol
57 0 198956 1686364 208516 0 0 EEM ED ND
-----------


From "show mem dead" we can see one process continuously increasing:
...
026A5598 0000000056 026A5538 026A55F8 001 -------- -------- 000196E0 hulc_flash init
026A55F8 0000000056 026A5598 026A5658 001 -------- -------- 000196E0 hulc_flash init
026A5658 0000000040 026A55F8 026A56A8 001 -------- -------- 005360BC hulc_flash init
026A56A8 0000000028 026A5658 026A56EC 001 -------- -------- 00539ED0 hulc_flash init
026A56EC 0000000044 026A56A8 026A5740 001 -------- -------- 00E4C48C hulc_flash init
...
Customer Visible

Was the description about this Bug Helpful?
(0)
Last Modified:
Mar 17,2017
Status:
Other
Severity:
2 Severe
Product:
(104)
Cisco IOS
Cisco Catalyst 3560E-12D-E Switch
Cisco Catalyst 3560E-24TD-S Switch
Cisco Catalyst 2960S-24PD-L Switch
Cisco Catalyst 2960-48TT-S Switch
Cisco Catalyst 3750V2-24PS Switch
Cisco Catalyst 2960-24-S Switch
Cisco Catalyst 3750V2-24FS Switch
Cisco Catalyst 3560E-48PD-SF Switch
Cisco Catalyst 2960-24TC-L Switch
Cisco Catalyst 3560E-48PD-E Switch
Cisco Catalyst 3560X-48P-S Switch
Cisco Catalyst 3560-12PC-S Compact Switch
Cisco Catalyst 3560G-48PS Switch
Cisco Catalyst 2960G-24TC-L Switch
Cisco Catalyst 3560X-48PF-S Switch
Cisco Catalyst 2960S-48TD-L Switch
Cisco Catalyst 3750X-24T-S Switch
Cisco Catalyst 3750X-48PF-L Switch
Cisco Catalyst 3750-24FS Switch
Cisco Catalyst 3560G-24TS Switch
Cisco Catalyst 2960S-48LPS-L Switch
Cisco Catalyst 2960-24TC-S Switch
Cisco Catalyst 3750G-12S-SD Switch
Cisco Catalyst 3560X-48PF-L Switch
Cisco Catalyst 2960-24PC-L Switch
Cisco Catalyst 2960-48PST-S Switch
Cisco Catalyst 2960PD-8TT-L Compact Switch
Cisco Catalyst 2960G-48TC-L Switch
Cisco Catalyst 3750E-24TD-E Switch
Cisco Catalyst 3560X-48T-L Switch
Cisco Catalyst 3750E-48PD-E Switch
Cisco Catalyst 3750X-48PF-S Switch
Cisco Catalyst 3750X-24S-S Switch
Cisco Catalyst 2960S-48FPS-L Switch
Cisco Catalyst 2960-48PST-L Switch
Cisco Catalyst 3750G-24TS-1U Switch
Cisco Catalyst 3560E-24PD-S Switch
Cisco Catalyst 2960-48TC-L Switch
Cisco Catalyst 3750X-48T-S Switch
Cisco Catalyst 2960S-24TD-L Switch
Cisco Catalyst 3750E-48PD-EF Switch
Cisco Catalyst 3750V2-48TS Switch
Cisco Catalyst 3560E-12D-S Switch
Cisco Catalyst 3560E-48PD-EF Switch
Cisco Catalyst 3560G-24PS Switch
Cisco Catalyst 2960S-48LPD-L Switch
Cisco Catalyst 2960S-24TS-L Switch
Cisco Catalyst 3750G-48TS Switch
Cisco Catalyst 2960S-48TS-S Switch
Cisco Catalyst 3750X-24T-L Switch
Cisco Catalyst 3560G-48TS Switch
Cisco Catalyst 3560X-24P-L Switch
Cisco Catalyst 3750V2-48PS Switch
Cisco Catalyst 3560V2-24PS Switch
Cisco Catalyst 3750X-48T-L Switch
Cisco Catalyst 3560E-24PD-E Switch
Cisco Catalyst 2960G-8TC-L Compact Switch
Cisco Enhanced Layer 2/3 EtherSwitch Service Module
Cisco Catalyst 3750E-24TD-S Switch
Cisco Catalyst 2960S-48TS-L Switch
Cisco Catalyst 3560X-48P-L Switch
Cisco Catalyst 3750V2-24TS Switch
Cisco Catalyst 3560E-24TD-E Switch
Cisco Catalyst 2960S-24PS-L Switch
Cisco Catalyst 3560V2-24TS Switch
Cisco Catalyst 2960-8TC-S Compact Switch
Cisco Catalyst 3750X-24S-E Switch
Cisco Catalyst 3750E-48TD-S Switch
Cisco Catalyst 3560V2-24DC Switch
Cisco Catalyst 3750E-24PD-E Switch
Cisco Catalyst 2960-8TC-L Compact Switch
Cisco Catalyst 3750X-48P-L Switch
Cisco Catalyst 3560X-48T-S Switch
Cisco Catalyst 3750E-48PD-SF Switch
Cisco Catalyst 3750E-24PD-S Switch
Cisco Catalyst 3560E-12SD-S Switch
Cisco Catalyst 3560V2-48TS Switch
Cisco Catalyst 3750X-24P-L Switch
Cisco Catalyst 3750X-12S-E Switch
Cisco Catalyst 2960-48TT-L Switch
Cisco Catalyst 2960-48TC-S Switch
Cisco Catalyst 3560X-24T-S Switch
Cisco Catalyst 3750X-48P-S Switch
Cisco Catalyst 3560-8PC Compact Switch
Cisco Catalyst 2960S-48FPD-L Switch
Cisco Catalyst 3560X-24T-L Switch
Cisco Catalyst 2960-24LC-S Switch
Cisco Catalyst 3560V2-48PS Switch
Cisco Catalyst 3750G-48PS Switch
Cisco Catalyst 3750X-12S-S Switch
Cisco Catalyst 2960-24PC-S Switch
Cisco Catalyst 2960-24LT-L Switch
Cisco Catalyst 3560E-48PD-S Switch
Cisco Catalyst 3560E-12SD-E Switch
Cisco Catalyst 3750G-24PS Switch
Cisco Catalyst 3560E-48TD-S Switch
Cisco Catalyst 2960-24TT-L Switch
Cisco Catalyst 3560X-24P-S Switch
Cisco Catalyst 3750X-24P-S Switch
Cisco Catalyst 3750E-48PD-S Switch
Cisco Catalyst 3560E-48TD-E Swtch
Cisco Catalyst 3750E-48TD-E Switch
Cisco Catalyst 2960S-24TS-S Switch
Support Cases:
10



4.3 CSCua52463 - C2960 - Memory leak in dead memory - not able to telnet/ssh to switch

hulc_flash init and init memory leaks seen in 3750 during dot1x testing
CSCuw93176
Symptom:
hulc_flash init and init memory leaks seen during dot1x testing on 3750G

Conditions:
This issue can be seen with or without any dot1x configured on the 3750G switch. Observed this even with the default config.

Workaround:
No workaround exists for this.

Further Problem Description:
Customer Visible

Was the description about this Bug Helpful?
(0)
Last Modified:
Feb 1,2017
Status:
Other
Severity:
3 Moderate
Product:
(143)
Cisco Catalyst 3750 Series Switches
Cisco Catalyst 2960-48TT-S Switch
Cisco Catalyst 2960-24-S Switch
Cisco Catalyst 3560E-12D-E Switch
Cisco Catalyst 2960S-24PD-L Switch
Cisco Catalyst 3560E-48PD-E Switch
Cisco Catalyst 3560X-48T-E Switch
Cisco Catalyst 3560E-48PD-SF Switch
Cisco Catalyst 3560X-48P-S Switch
Cisco ME 3400-24TS-A Switch
Cisco ME 3400EG-2CS-A Switch
Cisco Catalyst 2960G-24TC-L Switch
Cisco Catalyst 3560-12PC-S Compact Switch
Cisco Catalyst 3750G-12S Switch
Cisco Catalyst 3750V2-24FS Switch
Cisco Catalyst 2960-24TC-L Switch
Cisco Catalyst 3560E-24TD-S Switch
Cisco Catalyst 3750V2-24PS Switch
Cisco Catalyst 3560G-48PS Switch
Cisco Catalyst 3560G-24TS Switch
Cisco Catalyst 3560X-48PF-S Switch
Cisco Catalyst 2960S-48LPS-L Switch
Cisco ME 3400E-24TS-M Switch
Cisco IE 3000-8TC Industrial Ethernet Switch
Cisco Catalyst 3750X-48P-E Switch
Cisco Catalyst 3750G-16TD Switch
Cisco Catalyst 3750 Metro 24-AC Switch
Cisco Catalyst 2960S-48TD-L Switch
Cisco Catalyst 2960-24TC-S Switch
Cisco Catalyst 3750-48PS Switch
Cisco Catalyst 3750-24FS Switch
Cisco Catalyst 3750X-48PF-L Switch
Cisco Catalyst 3750X-24T-S Switch
Cisco Catalyst 3560-24TS Switch
Cisco Catalyst 3750X-24S-S Switch
Cisco Catalyst 3560X-48PF-L Switch
Cisco Catalyst 2960G-48TC-L Switch
Cisco Catalyst 2960-48PST-S Switch
Cisco Catalyst 3750X-48PF-S Switch
Cisco Catalyst 3750X-48PF-E Switch
Cisco Catalyst 3750G-24TS-1U Switch
Cisco Catalyst 2960S-48FPS-L Switch
Cisco Catalyst 3750E-24TD-E Switch
Cisco Catalyst 2960-48PST-L Switch
Cisco Catalyst 3750-48TS Switch
Cisco Catalyst 2960PD-8TT-L Compact Switch
Cisco Catalyst 3750E-48PD-E Switch
Cisco Catalyst 2960-24PC-L Switch
Cisco Catalyst 3750 Metro 24-DC Switch
Cisco ME 3400G-12CS-A Switch
Cisco Catalyst 3750G-12S-SD Switch
Cisco Catalyst 3560X-48T-L Switch
Cisco Catalyst 3560E-24PD-S Switch
Cisco ME 3400G-2CS-A Switch
Cisco Catalyst 2960S-24TD-L Switch
Cisco Catalyst 3750G-48TS Switch
Cisco Catalyst 2960S-48LPD-L Switch
Cisco Catalyst 3750V2-48PS Switch
Cisco Catalyst 2350-48TD-SD Switch
Cisco Catalyst 3560-48TS Switch
Cisco Catalyst 3750X-48T-S Switch
Cisco Catalyst 2350-48TD-S Switch
Cisco Catalyst 2960-48TC-L Switch
Cisco Catalyst 3560E-48PD-EF Switch
Cisco ME 3400EG-12CS-M Switch
Cisco Catalyst 3750X-24T-L Switch
Cisco Catalyst 2960S-24TS-L Switch
Cisco Catalyst 3560X-24T-E Switch
Cisco Catalyst 3750V2-48TS Switch
Cisco Catalyst 3750E-48PD-EF Switch
Cisco Catalyst 3560E-12D-S Switch
Cisco Catalyst 3560G-24PS Switch
Cisco Catalyst 3560X-24P-L Switch
Cisco Catalyst 3560G-48TS Switch
Cisco Catalyst 3750G-24TS Switch
Cisco Catalyst 2960S-48TS-S Switch
Cisco Catalyst 2960S-24PS-L Switch
Cisco Catalyst 3750V2-24TS Switch
Cisco Catalyst 3560X-24P-E Switch
Cisco Catalyst 3750X-24P-E Switch
Cisco Catalyst 3560-24PS Switch
Cisco Catalyst 3560E-24PD-E Switch
Cisco Catalyst 3560E-24TD-E Switch
Cisco Catalyst 3750X-48T-L Switch
Cisco Enhanced Layer 2/3 EtherSwitch Service Module
Cisco Catalyst 3560V2-24PS Switch
Cisco Catalyst 2960S-48TS-L Switch
Cisco Catalyst 3560X-48P-L Switch
Cisco Catalyst 3750E-24TD-S Switch
Cisco Catalyst 2960G-8TC-L Compact Switch
Cisco Catalyst 3750X-24T-E Switch
Cisco Catalyst 3750X-48P-L Switch
Cisco Catalyst 2960-8TC-L Compact Switch
Cisco Catalyst 3560E-12SD-S Switch
Cisco ME 3400-24FS-A Switch
Cisco Catalyst 3560X-48T-S Switch
Cisco Catalyst 3750E-48TD-S Switch
Cisco Catalyst 3750E-48PD-SF Switch
Cisco Catalyst 3560-48PS Switch
Cisco Catalyst 3560V2-24DC Switch
Cisco Catalyst 3560V2-24TS Switch
Cisco Catalyst 3750E-24PD-E Switch
Cisco Catalyst 3750G-24T Switch
Cisco Catalyst 3750E-24PD-S Switch
Cisco Catalyst 3750X-24S-E Switch
Cisco Catalyst 2960-8TC-S Compact Switch
Cisco Catalyst 3560X-48P-E Switch
Cisco Catalyst 2975 Switch
Cisco Catalyst 3750X-48T-E Switch
Cisco Catalyst 3750-24TS Switch
Cisco Catalyst 3560V2-48PS Switch
Cisco Catalyst 2960-24LC-S Switch
Cisco Catalyst 3750X-48P-S Switch
Cisco Catalyst 2960-48TT-L Switch
Cisco Enhanced Layer 2 EtherSwitch Service Module
Cisco Catalyst 3750X-12S-E Switch
Cisco Catalyst 3560V2-48TS Switch
Cisco Catalyst 3560X-24T-S Switch
Cisco Catalyst 2960S-48FPD-L Switch
Cisco Catalyst 3750X-24P-L Switch
Cisco Catalyst 3560-8PC Compact Switch
Cisco Catalyst 2960-48TC-S Switch
Cisco Catalyst 3560X-24T-L Switch
Cisco Catalyst 3750X-12S-S Switch
Cisco Catalyst 3560X-24P-S Switch
Cisco Catalyst 3750-24PS Switch
Cisco Catalyst 3560E-48TD-S Switch
Cisco Catalyst 3560E-12SD-E Switch
Cisco Catalyst 2960-24TT-L Switch
Cisco Catalyst 2960S-24TS-S Switch
Cisco Catalyst 3750E-48PD-S Switch
Cisco ME 3400G-12CS-D Switch
Cisco Catalyst 3750E-48TD-E Switch
Cisco Catalyst 3560E-48PD-S Switch
Cisco IE 3000-4TC Industrial Ethernet Switch
Cisco Catalyst 3750X-24P-S Switch
Cisco Catalyst 2960-24LT-L Switch
Cisco ME 3400-24TS-D Switch
Cisco Catalyst 3750G-24PS Switch
Cisco Catalyst 2960-24PC-S Switch
Cisco Catalyst 3560X-48PF-E Switch
Cisco Catalyst 3560E-48TD-E Swtch
Cisco Catalyst 3750G-48PS Switch
Support Cases:
1

Solution


Sadly although we found on bug CSCua52463 seems perfectly matching our case, Cisco does not post any solution and fix for it.


Downgrading to 12.2 or upgrading to 15.2(2)E3  will be our last step to try at this moment. Actually upgrading to 15.2(2)E3 resolved this issue.

Here is the monitoring information from NetMRI. After upgraded IOS to 15.2(2)E3, the free memory is getting stable. It was reducing slowly every minute until upgraded to 15.2(2)E3.
Type:Switch (99%)Vendor:Cisco
O/S Version:15.2(2)E3Model:catWsC2960x48tsLL
Up Time:1d 00h 17m 42sSNMP Status:Enabled
Last Communication:2017-03-27 10:13:30MAC Address:5C:A4:8A:5C:70:C1
Discovery Blackout:N/AChange Blackout:N/A

Free Switch Memory Status in Monitoring Tools





Reference: