*Content migrated from LinkedIn Group- Enterasys OneFabric Connect Central
By: Salvador Ferrer, Director, Solutions Architecture
Sometimes we face some scalability decisions when designing applications that use our web services architecture.
Some design decisions in the MDM plugin and DCM come from scalability considerations. Trying to avoid a full query of netsight database every poll time is an example that comes easily to our mind.
I did some informal scalability tests that show that NetSight is able to handle ~80 group change operations with reauthentication per second. But the fact that I managed to build a system that can handle that load doesn't mean that this is a desired setup in a vanilla NetSight install.
Most of our web services access a table in NetSight database named the end system table. That table contains all the end systems in the network and all the info that NetSight has about them. This table can grow very big in big deployments, searching and modifying it can't be taken lightly.
For instance a non recommended architecture would be one that has to update the whole table every access cycle. Imagine you have a visitor system to authenticate your guests and distribute single use passwords and you want to integrate that info with Netsight, for instance adding info about the user in one of the custom fields.
A simple loop to do that could be:
Every poll cycle
For all endsystem MAC in Netsight
Get info about MAC in visitor system
Write MAC's info in custom field 1
End For
End Poll
The problem with this simplistic approach is that we query all NetSight Database, get info and update all the database. We never check if the info is new or if MACs are new. This simplistic approach might run into scalability issues in the long run as more MACs are added to the system.
If we think about it it is very easy to come to the conclusion that the number of devices in the visitor's system is probably a subset of the ones in NetSight. If there is not way to trigger updates when new MAC's are added (unfortunately this happens more often than you can imagine) doing the poll in the opposite direction will save some updates.
If you can keep some local table with the result from the previous update, you can detect when new MACs are added to the visitors system and when the information about the existing ones changes. This way, we only update NetSight the for new MACs or those whose information changed.
It can look something like this:
Every Poll cycle
Get all MAC from visitors system
For all MACs in list
if MAC is new or MAC info changed
update MAC information in Netsight
end if
end for
end poll
This way a little more code will future proof out application for future growth of the network.
An easy conclusion from the previous is that if your application requires to update a significant amount of end systems every poll cycle, we'd better look at it in advance to make sure you won't find any issue in the future.
An example of this kind of applications is data gathering applications. If you plan to get the bandwidth on every port every minute and add it to the custom field of every end system, that will eventually run into scalability issues.
Let's say that the end system table is no place to store short lived information.
If you think that the application that you have in mind may face these issues open a post and discuss it, we'll be glad to review your concept and help you with the design.
