Default gateway on VSP not working after reboot of default gateway

[BRMS]

We are using an SPBM-Cloud of 4 VSP 8600 as our Backbone. Two of them are connected to a layer 2 transport net in which the firewall is used as default gateway. last week we had a power shortage and a few weeks ago i rebooted the firewall at night. both times the VSP stopped using the firewall as gateway. clients that tried to ping something behind the firewall got an “time to life exceeded” error. The VSPs itself were able to ping devices behind the firewall.

By using different VRFs on the VSPs we are creating different security domains. all other VRFs didn’t suffer from that problem although they get routed by the same firewall, albeit another IP.

the solution to the problem was deleting the route and recreating it.

this is the route we are using:

ip route 0.0.0.0 0.0.0.0 172.28.2.1 weight 1 preference 5

show ip route
************************************************************************************
        Command Execution Time: Mon Jun 08 12:42:44 2020 CEST
************************************************************************************
=====================================================================================================
                                       IP Route - GlobalRouter
=====================================================================================================
                                                     NH                      INTER   
DST             MASK            NEXT                 VRF/ISID         COST   FACE     PROT AGE TYPE PRF
-----------------------------------------------------------------------------------------------------
0.0.0.0         0.0.0.0         172.28.2.1           GlobalRouter     1      135      STAT 0   IB   5

this is the routing table on one of the VSPs that is not directly connected to the firewall:

show ip route
************************************************************************************
        Command Execution Time: Mon Jun 08 12:40:27 2020 CEST
************************************************************************************
=====================================================================================================
                                       IP Route - GlobalRouter
=====================================================================================================
                                                     NH                      INTER   
DST             MASK            NEXT                 VRF/ISID         COST   FACE     PROT AGE TYPE PRF
-----------------------------------------------------------------------------------------------------
0.0.0.0         0.0.0.0         pik                  GlobalRouter     10     4051     ISIS 0   IBSE 7  
0.0.0.0         0.0.0.0         kreuz                GlobalRouter     10     4051     ISIS 0   IBSE 7  
0.0.0.0         0.0.0.0         pik                  GlobalRouter     10     4052     ISIS 0   IBSE 7  
0.0.0.0         0.0.0.0         kreuz                GlobalRouter     10     4052     ISIS 0   IBSE 7  

What could possibly be the reason for this strange behavior?

[BRMS]

im using the most recent one: 6.3.4.0

the firewall is connected via smlt with 2 of the VSPs. vrrp is used for ip redundancy:

this is the vlan config of both connected VSPs:

VSP1:

vlan create 135 name "tr_firewall" type port-mstprstp 0 
vlan mlt 135 16
vlan mlt 135 109
vlan mlt 135 110
vlan mlt 135 111
vlan mlt 135 112
vlan mlt 135 113
vlan mlt 135 114    
vlan mlt 135 115
vlan members 135 2/4,7/1-7/7 portmember
vlan i-sid 135 10135
interface Vlan 135
ip address 172.28.2.4 255.255.255.240 53 
ip vrrp version 3 
ip vrrp address 2 172.28.2.2
ip vrrp 2 backup-master enable
ip vrrp 2 enable
exit

show ip vrrp address 
************************************************************************************
		Command Execution Time: Mon Jun 08 14:14:10 2020 CEST
************************************************************************************

====================================================================================================
                            VRRP Info - GlobalRouter
====================================================================================================

VRRP ID  P/V      IP              MAC                STATE    CONTROL  PRIO  ADV VERSION 
----------------------------------------------------------------------------------------------------
[...]
2        135      172.28.2.2      00:00:5e:00:01:02  Backup   Enabled  100   1   3       

2 out of 2 Total Num of VRRP Address Entries displayed.


VRRP ID  P/V      MASTER          UP TIME               HLD DWN  CRITICAL IP(ENABLED)  VERSION
----------------------------------------------------------------------------------------------------
[...]
2        135      172.28.2.5      7 day(s), 01:11:12    0        0.0.0.0         (No)  3       

2 out of 2 Total Num of VRRP Address Entries displayed.

VSP2:

vlan create 135 name "tr_firewall" type port-mstprstp 0 
vlan mlt 135 16
vlan mlt 135 109
vlan mlt 135 110
vlan mlt 135 111
vlan mlt 135 112
vlan mlt 135 113
vlan mlt 135 114
vlan mlt 135 115
vlan members 135 2/4,7/1-7/7 portmember
vlan i-sid 135 10135
interface Vlan 135
ip address 172.28.2.5 255.255.255.240 52 
ip vrrp version 3 
ip vrrp address 2 172.28.2.2
ip vrrp 2 backup-master enable
ip vrrp 2 priority 200
ip vrrp 2 enable
exit

show ip vrrp address 
************************************************************************************
		Command Execution Time: Mon Jun 08 14:14:10 2020 CEST
************************************************************************************

====================================================================================================
                            VRRP Info - GlobalRouter
====================================================================================================

VRRP ID  P/V      IP              MAC                STATE    CONTROL  PRIO  ADV VERSION 
----------------------------------------------------------------------------------------------------
[...]
2        135      172.28.2.2      00:00:5e:00:01:02  Master   Enabled  200   1   3       

2 out of 2 Total Num of VRRP Address Entries displayed.


VRRP ID  P/V      MASTER          UP TIME               HLD DWN  CRITICAL IP(ENABLED)  VERSION
----------------------------------------------------------------------------------------------------
[...]
2        135      172.28.2.5      7 day(s), 01:34:20    0        0.0.0.0         (No)  3       

2 out of 2 Total Num of VRRP Address Entries displayed.

[Martin_Sebek]

Hello,

what version of VOSS are you using on VSP 8600?  Do you use VRRP on VSPs in transport network?  I’ve seen broken routing in GRT prior to version 6.2.0.3.