This article describes how to identify hardware issues like faulty HDD, PSU, SSL Card, Network Interfaces etc without collecting the support bundle and waiting for offline analysis.
This script can be run directly from the NetScaler shell to verify hardware problem.1- Open a NetScaler CLI window and go to Shell:
>
> shell
Copyright (c) 1992-2013 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
The Regents of the University of California. All rights reserved.
root@NSMPX-8200#
2- Run the script ns_hw_err.bash:
(Also we can get this output from ns_offline_analysis script)
[collector_S_147.46.10.192_25Dec2016_10_33]$ ns_offline_analysis.bash -o.
root@NSMPX-8200# ns_hw_err.bash
NetScaler NS10.5: Build 59.13.nc, Date: Nov 5 2015, 10:26:52
platform: Manufacturer Citrix Inc.
platform: NSMPX-8000 4*CPU+12*E1K+1*E1K+4*CVM 1620 675300 (26), manufactured at 4/15/2013
platform: bios version 2.1d dated 09/06/2012 found at 0xffffff00000e9786
platform: serial 234XXXXXXR
platform: smbios_read failed
platform: sysid 675300 - NSMPX-8000 4*CPU+12*E1K+1*E1K+4*CVM 1620
platform: serial 234XXXXXXR
platform: sysid 675300 - NSMPX-8000 4*CPU+12*E1K+1*E1K+4*CVM 1620
HDD MODEL: Device Model: SAMSUNG SSD 830 Series
Generating the list of newnslog files to be processed...
Generating the events from newnslog files...
Checking for HDD errors...
Checking for HDD SMART errors...
Checking for Flash errors...
Checking for Mega Raid Controller errors...
Checking for SSL errors...
Checking for BIOS errors...
Checking for SMB errors...
Checking for MotherBoard errors...
Checking for CMOS errors...
License year: 2016: OK
Checking for SFP/NIC errors...
Checking for Firmware errors...
Checking for License errors...
Checking for Undetected CPUs...
Checking for DIMM flaps...
Checking for LOM errors...
Checking the Power Supply Errors...
root@NSMPX-8200#
To check for HDD issues you can also use the smartcl command:
1- You can also run smartcl :
root@NSMPX-8200# smartctl --scan
/dev/ad4 -d ata # /dev/ad4, ATA device
root@NSMPX-8200#
2- Select the HD to check:
root@NSMPX-8200# smartctl -a /dev/ad4
smartctl 5.42 2011-10-20 r3458 [FreeBSD 8.4-NETSCALER-10.5 amd64] (local build)
Copyright (C) 2002-11 by Bruce Allen,
=== START OF INFORMATION SECTION ===
Model Family: Samsung based SSDs
Device Model: SAMSUNG SSD 830 Series
Serial Number: SXXXXXXXXXXX666
LU WWN Device Id: 5 002538 043584d30
Firmware Version: CXM03B1Q
User Capacity: 256,060,514,304 bytes [256 GB]
Sector Size: 512 bytes logical/physical
Device is: In smartctl database [for details use: -P show]
ATA Version is: 8
ATA Standard is: ACS-2 revision 2
Local Time is: Tue Jan 3 13:53:03 2017 CET
SMART support is: Available - device has SMART capability.
SMART support is: Enabled
=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED
General SMART Values:
Offline data collection status: (0x02) Offline data collection activity
was completed without error.
Auto Offline Data Collection: Disabled.
Self-test execution status: ( 0) The previous self-test routine completed
without error or no self-test has ever
been run.
Total time to complete Offline
data collection: ( 1020) seconds.
Offline data collection
capabilities: (0x5b) SMART execute Offline immediate.
Auto Offline data collection on/off support.
Suspend Offline collection upon new
command.
Offline surface scan supported.
Self-test supported.
No Conveyance Self-test supported.
Selective Self-test supported.
SMART capabilities: (0x0003) Saves SMART data before entering
power-saving mode.
Supports SMART auto save timer.
Error logging capability: (0x01) Error logging supported.
General Purpose Logging supported.
Short self-test routine
recommended polling time: ( 2) minutes.
Extended self-test routine
recommended polling time: ( 17) minutes.
SMART Attributes Data Structure revision number: 1
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE
5 Reallocated_Sector_Ct 0x0033 100 100 010 Pre-fail Always - 0
9 Power_On_Hours 0x0032 093 093 000 Old_age Always - 31251
12 Power_Cycle_Count 0x0032 099 099 000 Old_age Always - 22
177 Wear_Leveling_Count 0x0013 098 098 000 Pre-fail Always - 68
179 Used_Rsvd_Blk_Cnt_Tot 0x0013 100 100 010 Pre-fail Always - 0
181 Program_Fail_Cnt_Total 0x0032 100 100 010 Old_age Always - 0
182 Erase_Fail_Count_Total 0x0032 100 100 010 Old_age Always - 0
183 Runtime_Bad_Block 0x0013 100 100 010 Pre-fail Always - 0
187 Uncorrectable_Error_Cnt 0x0032 100 100 000 Old_age Always - 0
190 Airflow_Temperature_Cel 0x0032 071 046 000 Old_age Always - 29
195 ECC_Rate 0x001a 200 200 000 Old_age Always - 0
199 CRC_Error_Count 0x003e 253 253 000 Old_age Always - 0
235 Unknown_Attribute 0x0012 099 099 000 Old_age Always - 21
241 Total_LBAs_Written 0x0032 099 099 000 Old_age Always - 8730895758
SMART Error Log Version: 1
No Errors Logged
SMART Self-test log structure revision number 1
Num Test_Description Status Remaining LifeTime(hours) LBA_of_first_error
# 1 Extended offline Completed without error 00% 9264 -
# 2 Short offline Completed without error 00% 8883 -
SMART Selective self-test log data structure revision number 1
SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS
1 0 0 Not_testing
2 0 0 Not_testing
3 0 0 Not_testing
4 0 0 Not_testing
5 0 0 Not_testing
Selective self-test flags (0x0):
After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.