FN-2018-425, VDX ISSU Firmware Upgrade May Fail
- Article Type:
- Field Notice
- Article Number:
- Last Modified:
ISSU Firmware upgrade may fail from NOS 7.0.0/7.0.1/7.0.1x to NOS 7.0.2/7.0.2x.
Timing issues can lead to a failure while upgrading the software version and a failure of one guest operating system (GOS), either SW/0 or SW/1.
In the best case scenario, the firmware download fails and the switch recovers and runs in normal condition afterwards. In the worst case scenario, the firmware download fails, and at least one GOS goes into rolling reboots and fails to boot up completely.
All VDX Platforms except VDX 8770 Product Family
When a GOS enters into a rolling reboot condition, it prints messages such as the following on its console:
qman_recovery_exit_local: DEBUG: the FQID 776 has dest_wq as channel: 44 and wq: 7 qman_recovery_exit_local: DEBUG: the WQ lengths for pool channel of portal 1 on cpu1 are: 0:0:0:0:0:0:0:0 qman_recovery_exit_local: DEBUG: the FQID 777 has np fileds state 1, frame count 0
If the partition SW/0 is consistently visible in "show version" and SW/1 is not, the console of SW/1 can be observed over the serial console port by pressing the keys ‘Ctrl+Shift+6’ all at once, releasing them, and then hitting the 2 key. When the messages above appear, they will appear in a burst, once approximately every fifteen minutes. The serial console port connection can be returned to SW/0 by pressing the keys ‘Ctrl+Shift+6’, releasing, and then hitting the 1 key.
There is no workaround that guarantees successful ISSU from any 7.0.0/7.0.1 release to any 7.0.2 release.
The coldboot option must be used when upgrading from any 7.0.0/7.0.1 release to any 7.0.2 release.
NOTE: For best practices regarding NOS upgrades, refer to the Release Notes of the version you are upgrading to, as well as reference soon to be released Extreme Knowledge Article #35934 “NOS Upgrade Best Practices” (eta 10/17/18).