Resolve Incorrect Nexus 9000 SSD Partitioning

Introduction

This document describes corrective action to be taken when incorrect SSD partitioning is seen on the Nexus 9000 Series Switch running NX-OS software.

Prerequisites

Requirements

It is recommended that users are familiar with NX-OS fundamentals, and recognize the Nexus 9000 hardware version and configuration present on the NX-OS to determine the corrective action.

Components Used

The information in this document is based on the Nexus 9000 Series switches running NX-OS software release 10.5(1)F, 10.4(4)M and later versions.

The information in this document was created from the devices in a specific lab environment. All of the devices used in this document started with a cleared (default) configuration. If your network is live, ensure that you understand the potential impact of any command.

Background Information

Enhancements were added starting in 10.5(1)F & 10.4(4)M to automatically detect SSD partition size on the Nexus 9000 to match the expected configured size. If this syslog has been seen during bootup in show logging log or show logging nvram, this indicates the NX-OS Nexus 9000 booted with an unexpected SSD partitioning size.

%PLATFORM-2-SSD_PARTITION_CHECK: Incorrect <device> partition size detected - please contact Cisco TAC for additional information

Partition issues can happen for various reasons. Smaller partitions can lead to issues where NX-OS images cannot be saved to bootflash for upgrades, show techs cannot be saved properly, loss of important log history, bootup can fail due to insufficient free space, and so on. This document focuses on recovering the SSD partition to the correct size.

Recovery Steps

Cisco recommends you use this Nexus 9000 SSD partitioning method to recover. SSD Re-Partitioning has the best chance of preserving critical files. This recovery process can only be required once, if an incorrect partition is seen.  Subsequent reloads need to preserve this configuration. If the SSD Re-partitioning method does not resolve the incorrect SSD size, contact Cisco Technical Assistance Center (TAC).

Note: This requires the Nexus 9000 to reload.

SSD Re-Partitioning

The command system flash sda resize can be used to perform a re-partition of the SSD based on the default partition scheme for the platform or to a specified partition scheme.

Partition scheme changes are designed to enhance the SSD capacity utilization of the platform and enable support for future features.

The NX-OS system makes the best effort to preserve critical files by using this SSD Re-partitioning option. The critical files include NX-OS boot image, running-config, bootflash contents, and logflash contents. 

Caution: Cisco recommends ALWAYS backing up critical files to an external source.

switch# system flash sda resize ?
  <CR>     
  extended  Cfg=1GB, logflash=39GB
  standard  Cfg=64MB, logflash=4|8GB

Standard partition scheme allocates 64MB for configuration partitions & either 4 or 8GB for logflash depending on SSD capacity.

Extended partition scheme allocates 1GB for configuration partitions & 39GB for logflash.

Standard is the default SSD configuration for these platforms:

• Nexus 9300-FX/FXP/FX2/GX/GX2
• Nexus 92348GC-X
• Nexus 93180YC-FX3
• Nexus 93108TC-FX3P
• Nexus 9332C
• Nexus 9364C
• Nexus 9408
• Nexus 9500s

Note: Extended is supported with these platforms containing a SSD greater than 64GB.

Extended is the default SSD configuration for these platforms. Standard is not supported with these platforms:

• Nexus 9348GC-FX3
• Nexus 9348GC-FX3PH
• Nexus 92348GC-FX3
• Nexus 93108TC-FX3
• Nexus 9332D-H2R
• Nexus 93400LD-H1
• Nexus 9364C-H1
• Nexus 9800s
• Nexus 9364E-SG2-Q
• Nexus 9364E-SG2-O

Note: Starting in 10.5(1)F & 10.4(4)M, the default Extended partition scheme for these platforms has been modified resulting in a greater bootflash size. To take advantage of the larger bootflash size, please run the command 'system flash sda resize'.

SSD Repartitioning for Dual Supervisor Nexus 9500/9800

For Nexus 9500 and 9800 platforms with a single supervisor, the resize operation results in a chassis reload.

For Nexus 9500 and 9800 platforms with dual supervisors running 10.4(x) or 10.5(1)F releases, the resize operation needs to be executed twice.

The process begins with repartitioning the current Active Supervisor, which causes a chassis reload.  Once the chassis is powered online, the previous Standby Supervisor will become Active.  Once the Supervisors are back in HA, the resize command needs to be executed once more on the current Active Supervisor resulting in another chassis reboot.  Once resize operation is completed, both Active and Standby Supervisors must have updated partitions.

For Nexus 9500 and 9800 platforms wth dual supervisors, starting with release 10.5(2)F, the resize operation only needs to be executed once and does not require a chassis reload. The process begins with repartitioning and reloading the Standby Supervisor, followed by an automatic switchover once the Supervisor returns to the HA-Standby state, allowing the previous Active Supervisor to undergo repartitioning.

Review the detailed explanation in the SSD Re-partitioning section of the Cisco Nexus 9000 Series NX-OS Fundamentals Configuration Guide, Release 10.4(x).

Additional Verification/Identification

This is an example of a Nexus 9000 with SSD partitioning that is incorrect.  When running the command system flash sda resize standard, you can see that the current scheme partitions do not match the target schemes for a standard partition configuration.

switch# system flash sda resize standard

!!!! WARNING !!!!

Attempts will be made to preserve drive contents during
the resize operation, but risk of data loss does exist.
Backing up of bootflash, logflash, and running configuration
is recommended prior to proceeding.

!!!! WARNING !!!!


current scheme is
sda 8:0 0 119.2G 0 disk
|-sda1 8:1 0 1.2G 0 part
|-sda2 8:2 0 9.6G 0 part /mnt/plog
|-sda3 8:3 0 1.2G 0 part /mnt/pss
|-sda4 8:4 0 11.9G 0 part /bootflash
|-sda5 8:5 0 1.2G 0 part /mnt/cfg/0
|-sda6 8:6 0 1.2G 0 part /mnt/cfg/1
|-sda7 8:7 0 39.4G 0 part /logflash
|-sda8 8:8 0 11.9G 0 part
`-sda9 8:9 0 23.9G 0 part


target scheme is
sda 8:0 0 64G|120GB|250GB 0 disk
|-sda1 8:1 0 512M 0 part
|-sda2 8:2 0 32M 0 part /mnt/plog
|-sda3 8:3 0 128M 0 part /mnt/pss
|-sda4 8:4 0 110.5G 0 part /bootflash
|-sda5 8:5 0 64M 0 part /mnt/cfg/0
|-sda6 8:6 0 64M 0 part /mnt/cfg/1
|_sda7 8:7 0 8G 0 part /logflash

To verify that a Nexus 9000 SSD partitioning has been corrected, you can use the same command system flash sda resize standard to verify that the current scheme partitions match the target scheme.  Additionally, you can see a message at the bottom of the command indicating that the system is already in standard scheme. 

switch# system flash sda resize standard

!!!! WARNING !!!!

Attempts will be made to preserve drive contents during
the resize operation, but risk of data loss does exist.
Backing up of bootflash, logflash, and running configuration 
is recommended prior to proceeding.

!!!! WARNING !!!!


current scheme is 
sda 8:0 0 119.2G 0 disk 
|-sda1 8:1 0 512M 0 part 
|-sda2 8:2 0 32M 0 part /mnt/plog
|-sda3 8:3 0 128M 0 part /mnt/pss
|-sda4 8:4 0 110.5G 0 part /bootflash
|-sda5 8:5 0 64M 0 part /mnt/cfg/0
|-sda6 8:6 0 64M 0 part /mnt/cfg/1
`-sda7 8:7 0 8G 0 part /logflash


target scheme is 
sda 8:0 0 64G|120GB|250GB 0 disk 
|-sda1 8:1 0 512M 0 part 
|-sda2 8:2 0 32M 0 part /mnt/plog 
|-sda3 8:3 0 128M 0 part /mnt/pss 
|-sda4 8:4 0 rem 0 part /bootflash
|-sda5 8:5 0 64M 0 part /mnt/cfg/0
|-sda6 8:6 0 64M 0 part /mnt/cfg/1
|_sda7 8:7 0 8G 0 part /logflash


System is already in Standard scheme Use force option to proceed

system flash sda resize can also be executed on the Standby Supervisor for validation purposes only. This command will not execute the resize operation on the non-Active Supervisor.

Additionally, these commands can be used to view and verify that the Nexus 9000 SSD partitionings are configured as expected.

switch# show system internal flash | i i dev/sda
/bootflash 113795280 38647924 75147356 34 /dev/sda4
/cmn/cfg/0 58090 872 53942 2 /dev/sda5
/cmn/cfg/1 58090 872 53942 2 /dev/sda6
/cmn/pss 121299 10546 104200 10 /dev/sda3
/mnt/cfg/0 58090 872 53942 2 /dev/sda5
/mnt/cfg/1 58090 872 53942 2 /dev/sda6
/mnt/plog 27252 16025 9589 63 /dev/sda2
/mnt/pss 121299 10546 104200 10 /dev/sda3
/logflash 8107484 1501688 6186368 20 /dev/sda7

Note: feature bash-shell must be enabled globally to use bash-shell.

switch# run bash
bash-4.4$ lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
loop0 7:0 0 165.8M 0 loop /usr_ro
loop1 7:1 0 116.9M 0 loop /isan_lib_ro
loop2 7:2 0 48.2M 0 loop /isan_bin_ro
loop3 7:3 0 43.9M 0 loop /isan_bin_eth_ro
loop4 7:4 0 11.3M 0 loop /isan_lib_eth_ro
loop5 7:5 0 4.2M 0 loop /isan_lib_n9k_ro
loop6 7:6 0 4K 0 loop /isan_bin_n9k_ro
loop7 7:7 0 195.3M 0 loop /bootflash/.rpmstore/patching
loop8 7:8 0 57.6M 0 loop 
loop9 7:9 0 144.4M 0 loop 
loop10 7:10 0 221.2M 0 loop 
sda 8:0 0 119.2G 0 disk 
├─sda1 8:1 0 512M 0 part 
├─sda2 8:2 0 32M 0 part /mnt/plog
├─sda3 8:3 0 128M 0 part /mnt/pss
├─sda4 8:4 0 110.5G 0 part /bootflash
├─sda5 8:5 0 64M 0 part /mnt/cfg/0
├─sda6 8:6 0 64M 0 part /mnt/cfg/1
└─sda7 8:7 0 8G 0 part /logflash
mmcblk0 179:0 0 3.7G 0 disk 
├─mmcblk0p1 179:1 0 32M 0 part 
├─mmcblk0p2 179:2 0 32M 0 part 
├─mmcblk0p3 179:3 0 32M 0 part /mnt/pstore
└─mmcblk0p4 179:4 0 3.6G 0 part