serious issue after enabling udp policy

  • 0
  • 1
  • Problem
  • Updated 6 months ago
  • Solved
Hi guys, we need some help!

We have serious problems with our X690-G2 switches after enabling a udp policy. In the last weeks we had brought a bulk of x690 Switches to production with the same policy without  any problems

In this case we are using at our location two x690 as core switches with mlag enabled

The policy is in our configurations since years in production and works well without problems

Here the output at the cli after enabling the policy
############################################################################
drw-r--r--    2 root     root         1024 May 14 14:33 cc_logs
-rw-r--r--    1 root     root       970838 May 17 11:40 primary.cfg
drw-r--r--    4 root     root         1024 May 14 14:13 ssl
-rw-r--r--    1 admin    admin        1248 May 17 12:39 udp_profile_client.pol
drwxr-xr-x    2 root     root         1024 May 17 12:23 vmt

 1K-blocks      Used Available Use%
g
    245679      3072    229500   1%
W_MP_2.3.9 #
W_MP_2.3.9 # check policy udp_profile_client
Policy file check successful.
W_MP_2.3.10 # conf vlan cli01 udp-profile udp_profile_client.pol
netTools: mallocProcess netTools pid 2706 died with signal 6
.c:2369: sysmallCode:
f778abe0:oc: Assertion `(  b8   ad   00 old_top == (((mb  00   00   cd   80   90
f778abe8:inptr) (((char *  90   8d   b4   26   00   00   00   00 ) &((av)->bins[(
f778abf0:  51   52   55   89   e5 (1) - 1) * 2]))   0f   34   90
f778abf8:  90   90 - __builtin_offs  90   90   90   90   cd   80 etof (struct mal
f778ac00: <5d>  5a   59   c3   ac loc_chunk, fd)))  fa   ff   ff
f778ac08:  14   00   00 ) && old_size ==  00   71   00   03   03
 0) || ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 *(sizeof(size_t))) - 1)) & ~((2 *(sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long) old_end & pagemask) == 0)' failed.
W_MP_2.3.11 # netTools: mallocProcess netTools pid 3209 died with signal 6
Code:
f76ebbe0:  b8   ad   00 .c:2369: sysmall  00   00   cd   80   90 oc: Assertion `(
f76ebbe8:  90 old_top == (((mb  8d   b4 inptr) (((char *  26   00   00   00   00
) &((av)->bins[(f76ebbf0:  51 (1) - 1) * 2]))   52 - __builtin_offs  55 etof (struct mal  89   e5   0f loc_chunk, fd)))  34   90
f76ebbf8:  90   90 ) && old_size ==  90  0) || ((unsigne  90   90   90   cd   80
f76ebc00: <5d>d long) (old_siz  5a e) >= (unsigned   59 long)((((__built  c3   ac   fa   ff   ff
in_offsetof (strf76ebc08:  14   00   00   00   71   00 uct malloc_chunk  03   03
, fd_nextsize))+((2 *(sizeof(size_t))) - 1)) & ~((2 *(sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long) old_end & pagemask) == 0)' failed.
Sending SIGKILL to all processes
Requesting system reboot
PEI Phase:
Initializing..................   0%
Initializing..................  15%
Initializing..................  21%
Initializing..................  84%


############################################################################

After this happens the policy rather was writen in the edit mode then copied into the switch 

but this was not really helpful

The next error happend after configuring the vlans to the udp-profile

Here the output

* (Software Update Required) W_MP_2.2.525 #
* (Software Update Required) W_MP_2.2.525 # conf vlan cli01 udp-profile udp_profile_client.pol
netTools: mallocProcess netTools pid 2649 died with signal 6
.c:2369: sysmallCode:
f775dbe0:  b8 oc: Assertion `(  ad old_top == (((mb  00   00 inptr) (((char *  00   cd   80   90
f775dbe8:  90   8d   b4 ) &((av)->bins[(  26   00   00   00   00
f775dbf0:  51   52   55   89   e5   0f   34   90
f775dbf8:(1) - 1) * 2]))   90   90   90   90   90   90   cd   80
f775dc00: <5d>  5a   59   c3   ac   fa   ff   ff
f775dc08:  14   00   00   00   71   00   03   03
- __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 *(sizeof(size_t))) - 1)) & ~((2 *(sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long) old_end & pagemask) == 0)' failed.
The system is going down NOW!W_MP_2.2.526 #

Our SL3 technician from our distributor does not know about such problems with udp policys
Photo of HarrySo

HarrySo

  • 482 Points 250 badge 2x thumb

Posted 6 months ago

  • 0
  • 1
Photo of Ronald Dvorak

Ronald Dvorak, Embassador

  • 51,328 Points 50k badge 2x thumb
* (Software Update Required) W_MP_2.2.525 #
I'm not an XOS expert but that doesn't look OK.

https://community.extremenetworks.com/extreme/topics/new-switch-show-software-update-required
Photo of Grosjean, Stephane

Grosjean, Stephane, Employee

  • 13,676 Points 10k badge 2x thumb
The switch is not using a GA version, but a beta. You need to upgrade before anything else. Install 22.4p1-2.
Photo of HarrySo

HarrySo

  • 482 Points 250 badge 2x thumb
here the "show switch" output

SysName:          xxxx
SysLocation:      xxxx
SysContact:       xxx
System MAC:       00:04:96:xx:xx:xx
System Type:      X690-48x-2q-4c

SysHealth check:  Enabled (Normal)
Recovery Mode:    All
System Watchdog:  Enabled

Current Time:     Thu May 17 13:43:44 2018
Timezone:         [Auto DST Enabled] GMT Offset: 60 minutes, name is mez.
                  DST of 60 minutes is currently in effect, name is not set.
                  DST begins every last Sunday March at 2:00
                  DST ends every last Sunday October at 3:00

Boot Time:        Thu May 17 13:00:13 2018
Boot Count:       67
Next Reboot:      None scheduled
System UpTime:    43 minutes 30 seconds

Current State:    OPERATIONAL
Image Selected:   secondary
Image Booted:     secondary
Primary ver:      22.3.0.39
Secondary ver:    22.4.1.4
                  patch1-2

Config Selected:  primary.cfg
Config Booted:    primary.cfg
Config Automatic: NONE (Disabled)

primary.cfg       Created by ExtremeXOS version 22.4.1.4
                  981546 bytes saved on Thu May 17 13:13:58 2018

LAA MAC:          Locally Administered MAC Address Disabled
Photo of Ron Huygens

Ron Huygens, Employee

  • 3,360 Points 3k badge 2x thumb
Ronald is correct, the switch prompt tells us thet it is shipped with a pre-GA version of code. (22.3.0.39) but after the reboot it looks like it is now running  22.4.1.4patch1-2.
When the issue is still present on the GA version of code, I recommend that you open a case at GTAC to have this investigated.