SummitStack upgrade fails : "No space left on device"

[Gabriel3]

Hi everybody,

I'm upgrading my production SummitStacks from EXOS version 15.1.4.3 to 16.1.3.6.
The upgrade was successful on every stack except one, which fails and displays this error message :

Failed to download image on Slot-7 - tftp: can't open '/scratch/summitX-16.1.3.6-patch1-11.xos': No space left on device

It's a 8-member stack, i tried to launch the upgrade several times and it always fails on Slot 7.

Here is the complete output displayed in CLI :

Slot-1 # download image X.X.X.X extreme/summitX-16.1.3.6-patch1-11.xos vr vr-Default secondary
Do you want to install image after downloading? (y - yes, n - no, - cancel) Yes
Downloading to Slot-1(Master): ..................................................................................................................................................................................
Downloading to Slot-6(Standby):
Downloading to Slot-5(Standby):
Downloading to Slot-7(Standby):
Downloading to Slot-8(Standby):
Downloading to Slot-2(Backup):
Downloading to Slot-3(Standby):
Downloading to Slot-4(Standby): Failed to download image on Slot-7 - tftp: can't open '/scratch/summitX-16.1.3.6-patch1-11.xos': No space left on device
.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*.*..***Error: Download image failed (see above error message)

I rebooted the slot 7 but it doesn't solve the problem.
I even tried to remove slot 7 from the stack, do "unconfigure switch all" on slot 7 to put it back to factory default, and then put it back in the stack. The issue is still here.

I wonder if any of you ever faced this problem.
Thanks in advance for the help

Gabriel

[Gabriel3]

Hi,

All slots are X460-48p

Gabriel

[Stephane_Grosj1]

out of curiosity, what switch it is (slot 7)?

[Gabriel3]

Hello,

Thanks for your help.
When i run "ls" and "ls internal-memory" on the Master slot, it seems no files are using a lot of space.
Then from slot 1 I telnet into slot 7 and run the same commands, i find that internal memory has only 38% space left. I try to erase the content of internal memory but it seems I can't do that from a Standby slot :

Slot-7 > ls internal-memory
-rwxr-xr-x 1 root 0 35645 Feb 24 17:28 old_traces.tgz

1K-blocks Used Available Use%
118160 44474 73686 38%
Slot-7 >
Slot-7 > rm internal-memory old_traces.tgz
Remove old_traces.tgz from internal storage? (y/N) Yes
This command cannot be executed on a standby node.

I ran the "rm internal-memory old_traces.tgz" on the Master slot but it didn't remove the files from slot 7.
Is there a way to remove files that are on a standby slot ?

Best regards,
Gabriel

[BradP]

Hi Gabriel,

Please first try to run a "ls" and "ls internal-memory" on the switch. Both of these will still retain files after "unconfigure switch all"

Slot-1 Stack.11 # ls internal-memory
-rwxr-xr-x 1 root 0 23427 Feb 25 09:59 show_tech.log.gz
-rwxr-xr-x 1 root 0 153482 Feb 25 09:58 trace.devmgr.1417
-rwxr-xr-x 1 root 0 135079 Feb 25 09:58 trace.nodemgr.1421
-rwxr-xr-x 1 root 0 5622 Feb 25 09:58 trace.vlan.1437
Then, at the prompt, type "rm internal-memory *.*"
Slot-1 Stack.17 # rm internal-memory *.*
NOTE: This will remove all files from internal storage.
Continue? (y/N) Yes
Slot-1 Stack.18 # -rwxr-xr-x 1 root 0 135079 Feb 25 09:58 trace.Slot-1 Stack.18 # -rwxr-xr-x 1 root 0 153482 Feb 25 09:58 trace.Slot-1 Stack.18 # -rwxr-xr-x 1 root 0 23427 Feb 25 09:59 show_tSlot-1 Stack.18 # ls internal-memory
1K-blocks Used Available Use%
49038 716 48322 1% Then, at the prompt, "ls" and "rm" to any files you feel comfortable removing. I wouldn't remove the configuration file that you're using.

Hope this helps.

Thanks
Brad

[Dave_Hammers]

You could try BootRom rescue using the summitX-16.1.3.6-patch1-11.xos.
https://gtacknowledge.extremenetworks.com/articles/How_To/How-to-Use-the-Boot-Rom-Menu-to-Download-a...