topic Logging Best Practice - NVRAM vs Syslog in ExtremeSwitching (EXOS/Switch Engine)

Logging Best Practice - NVRAM vs Syslog

RobertWilkinson — Mon, 18 Sep 2023 09:49:11 GMT

Hi Community,

A topic that i think will generate alot of discussion.

What level of logging to implement at the core, dist and edge and does it differ for NVRAM vs a Syslog server?

Re: Logging Best Practice - NVRAM vs Syslog

OscarK — Mon, 18 Sep 2023 12:22:52 GMT

The nvram for logging on EXOS switches is small, and is not what you see in show log, nvram is what you see in show log messages nvram and these are the ones that remain after a reboot.

ideally you should have a centralized syslog server like for example XIQ Site Engine, as normal log entries will vanish after some time or after a reboot.

Re: Logging Best Practice - NVRAM vs Syslog

RobertWilkinson — Wed, 20 Sep 2023 12:48:43 GMT

Thanks Oscar.

Any recommendations on logging severity on each log type (NVRAM, Memory, Console Syslog)?

Re: Logging Best Practice - NVRAM vs Syslog

Junespiano — Wed, 24 Apr 2024 12:50:24 GMT

How many messages NVRAM is able to storage, OscarK?

Re: Logging Best Practice - NVRAM vs Syslog

Gabriel_G — Fri, 03 May 2024 14:58:40 GMT

The defaults are fine.
Console -> Default Disabled -> Down to Severity Info
Memory-Buffer -> Default Enabled (show log) -> Down to Severity Debug-Data (Assuming log debug-mode is enabled)
NVRAM -> Default Enabled (show log messages nvram) -> Down to Severity Warning
Syslog -> Default Disabled -> Down to Severity Debug-Data (Assuming log debug-mode is enabled)

Notes:
-Debug logs only show when a debug log event has been added to the relevant log filter (not done by default), log debug-mode has been enabled (not done by default), and the log target is configured for debug level severities (done by default for memory-buffer and syslog). This is usually only needed for advanced troubleshooting. Optionally omit debug logs being sent to syslog by limiting the severity to 'info' in the 'configure log target syslog' command.
-You can increase the size of the memory-buffer up to 20000 messages with minimal impact on RAM usage:
configure log target memory-buffer number-of-messages 20000

Hope that helps!

Re: Logging Best Practice - NVRAM vs Syslog

Gabriel_G — Fri, 03 May 2024 14:55:50 GMT

Around 200ish, I believe there are some dynamic factors and this is not configurable.

Re: Logging Best Practice - NVRAM vs Syslog

FredrikB-NN2 — Sat, 04 May 2024 06:15:29 GMT

Some things to consider:

Enable logging of commands excuted in the CLI:

enable cli config-logging
enable cli config-logging expansion

Generic syslogging:

create log filter commandfilter
configure log filter DefaultFilter add exclude events cli.logRemoteCmd
configure log filter DefaultFilter add exclude events cli.logLocalCmd
configure log filter commandfilter add events cli.logLocalCmd
configure log filter commandfilter add events cli.logRemoteCmd
configure syslog add 10.199.0.11:514 vr VR-Default local0
enable log target syslog 10.199.0.11:514 vr VR-Default local0
configure log target syslog 10.199.0.11:514 vr VR-Default local0 filter DefaultFilter severity Debug-Data
configure log target syslog 10.199.0.11:514 vr VR-Default local0 match Any
configure log target syslog 10.199.0.11:514 vr VR-Default local0 format timestamp seconds date Mmm-dd event-name none priority tag-name

configure syslog add 10.199.0.11:514 vr VR-Default local1
enable log target syslog 10.199.0.11:514 vr VR-Default local1
configure log target syslog 10.199.0.11:514 vr VR-Default local1 filter commandfilter severity Debug-Data
configure log target syslog 10.199.0.11:514 vr VR-Default local1 match Any
configure log target syslog 10.199.0.11:514 vr VR-Default local1 format timestamp seconds date Mmm-dd event-name none priority tag-name

Using the above, you can choose to separate command logs and normal logs in different files on the syslog server (selecting on facility local0 or local1 in the example).

I don't think it's worth the effort of separating various tiers in the network when it comes to syslogging. It's easy enough to filter out the host you want to look at when you need the data. If you have lots and lots of switches (hundreds or more), a syslog server needs to have some serious disks performance wise. An old Linux laptop is good for many purposes, but massive syslogging to it's internal 2.5" SATA drive isn't one of them...

The only thing to log to internal memory is the default as it catches boot messages and anything sent to syslog while the network is down. You will rarely need it, but when you do, it may be worth a lot. Remember to set the number of lines to something as high as your platform can handle, ex:

configure log target memory-buffer number-of-messages 20000

Sure, if you're space constrained, you can filter some thing out on various levels, but I haven't bothered with that. "logrotate" does the job of handling files on the Linux server for me.