Non-default SNMP User in XMC = headache with Traps... FAD?

[Tomasz]

Hello,

I’ve a bit long story to a short question (tl;dr: GOTO the last paragraph). 🙂

I’ll share my thoughts below according to the troubleshooting steps I took today, hoping that someone can have a look and make sure it’s FAD and not some tricky internals in XMC that are to be modified in a future release.

I am playing around in my lab trying to configure EXOS + XMC/EAC/EAN + few other things in the way I consider a set of deployment good practices. I decided to use a non-default SNMPv3 user for XMC-EXOS communication, along with SHA and AES (FYI: 128b) instead of MD5/DES.

XMC: 8.5.4.23

EXOS: 30.7 and 31.2

I wanted to play with SNMP Traps today as someone told me they don’t work with ELRP. I remember they work well in the EXOS S&R training so tried to do it quick and spent few hours on that lol. 😉

I used “Configure Trap Receiver” option in XMC to configure EXOS.

Toggling ports, saving config, inducing ELRP to trigger port disabling, custom trap - nothing shown up in XMC Events when limiting the view to Traps. Syslog was good though.

Double-checked device profile in XMC and SNMP config in EXOS. All was fine. XMC and EXOS in the same subnet thru VR-Default.

Unfortunately I spotted wrong time on XMC and it distracted me for a while (took another quarters to understand chronyd doesn’t like Windows Server for time sync unless additional measure is taken). I thought perhaps that’s the reason why traps are not showing. Definitely looked in a wrong place, tcpdump would be better to see if anything comes to XMC as the first step.

I tried to fix things like limit to VR-Default in snmpv3 target-params as I didn’t see any traps sent under neither ‘show snmp vr-def’ nor ‘show snmp vr-mgmt’. I thought they are not sent as it tried VR-Mgmt by default and it was down so nothing to send. But when I limited the VR scope, it didn’t work.

Moved between 30.7 and 31.2 just to check for any potential fw issue.

After a bit of lurking around the same show commands again and again and staring into the void I have spotted that XMC’s right-click option put a username into EXOS config that I don’t want to use (ie. I have a different username in SNMP part of device profile):

configure snmpv3 add target-params "TV1v3snmpuser" user "snmpuser" mp-model snmpv3 sec-model usm sec-level priv

And it was wrong, because I have created SNMPv3 user called xmc for XMC-EXOS communication and I don’t have snmpuser account in SNMP config of EXOS at all.

So I have modified this line in EXOS. The switch started sending traps! But… Nothing in XMC.

With tcpdump on XMC (helped a lot as ‘show snmpv3 counters’ are good in default communication direction, not relevant for Traps/Informs) I could see this kind of stuff:

21:53:51.339422 00:04:96:d7:85:28 (oui Unknown) > 00:0c:29:c3:94:f8 (oui Unknown), ethertype IPv4 (0x0800), length 271: 172.16.101.1.44417 > xmc.lab.local.snmp-trap:  F=apr U="xmc" [!scoped PDU]01_5[ABBREVIATED - Tomasz]

21:53:51.339607 00:0c:29:c3:94:f8 (oui Unknown) > 00:04:96:d7:85:28 (oui Unknown), ethertype IPv4 (0x0800), length 148: xmc.lab.local.snmp-trap > 172.16.101.1.44417:  F= U="xmc" E=_80_00_1f_88_80_2b_a1_56_5d_46_43_30_60 C="" Report(28)  S:snmpUsmMIB.usmMIBObjects.usmStats.usmStatsUnknownUserNames.0=23

That was strange indeed. I have decided to run through /usr/postinstall/snmpconfig to change the default snmpuser to my good ol’ pal ‘xmc’. During deployment I didn’t change it as I considered it just for SNMP agent on XMC in case of anything above trying to poll XMC server.

Restarted the service, restarted the server. No friggin’ luck this time.

After digging more I have realized that postinstall script is modifying /etc/snmp/snmpd.conf but it doesn’t touch /usr/local/Extreme_Networks/NetSight/appdata/snmptrapd.conf. I have added a line to cover my non-default user (‘createUser xmc SHA <authcred> AES <privcred>’). Traps finally got succesfully processed by XMC and displayed under Events tab.

My quesion is - is this FAD? I had to manually modify /usr/local/Extreme…/appdata/snmptrapd.conf with my non-default EXOS SNMPv3 user (although the file comments wanted to discourage me) as right-click ‘Register Trap Receiver’ wasn’t using my Device Profile SNMP user and that user was neither reflected in the config to receive traps.

Thanks,

Tomasz

P.S. If you find that useful for non-defaults in SNMP config, perhaps it’s worth noting in the KB until XMC behavior is adjusted.

[Tomasz]

Hi Stefan, Peter,

Dang! So simple! I was looking at SNMP options for XMC but didn’t look at Traps...

I have adjusted these options, removed trap config from EXOS and re-registered thru the GUI. It used the correct one.

It works kinda strange but it works indeed. When you choose something else than default_snmp_v3 another record is just added in new line to /usr/local/Extreme_Networks/NetSight/appdata/snmptrapd.conf.

I’d still ask for registering the trap receiver according to snmp user already set on a device (+ if I want to have creds for all kinds of devices in the snmptrapd.conf file, I have to pick each one and save the options several times; EXOS snmp user is ‘xmc’ but for WiNG it’s that ‘snmpmanager’ thing).

Kind of workaround but it’s fair enough.

Thanks!

Cheers,
Tomasz

[Stefan_K_ · Accepted solution]

So what is the purpose of this?

[PeterK]

I can confirm, that you need to manually mod the snmptrad.conf for using snmpV3 Traps.

In legacy Java console client you were able to mod this file from GUI with choosing snmp profile. But this function was never ported to WEB-GUI. I don’t know why.

If you use “register trap receiver” in web-gui it is configured as v1/v2c trap or as snmpv3 Inform, which is mostly not working, from my experience.

[Stefan_K_]

Hi Tomasz,

last time I did something with SNMP Traps in EXOS/XMC it was working as expected, but it was ~2 years ago. Did you take a look at Administration → Options → Trap?

But I had another fatal problem with this: You can only define one global SNMP credential. When you have different snmp-profiles for different subsidaries, this will fail. I don’t know why there isn’t an option to use the SNMP user for traps, that is configured in the selected SNMP profile of the switch.

Best regards
Stefan

[Tomasz]

Now I look at this traps view in XMC like Thanos stared at some sights after wiping out half of life in the universe.