The most common question I get asked is “When troubleshooting, where do you start?”.
Other than the "client interview", the answer is simple, “start with an inventory”. When I say “Inventory” it is an all-encompassing term. It covers the physical equipment, connectivity, layer 2, 3, and any other layer it takes to get from point a to point b. Many times we can get a 'visual' from the client interview but other times we need more info, or have to confirm what they believe is out there.
I always say, “We can't fix what we can't see”. If the client already has documentation, great, let's review it with respect to the issue at hand. I am not surprised how many times I am presented with out-of-date documentation. In some cases, entire sites are omitted, not added or equipment upgrades are not reflected. I never consider this scenario a negative one and explain “all we have to do is update what we have”.
It's important that this does not turn into a “who didn’t update the documentation and why witch hunt”, let's just get the documentation updated and move on. This is always the interesting part of the documentation exercise; watching the client argue with colleagues about “who moved that”, “I thought we replaced that box”, and the ever-popular “that’s an old config”. Many times, we actually find and fix the problem as we go through the documentation exercise.
When I talk about the documentation process, the most common response I get is “I don’t have time to do that”, and I respond with “how much time do you have to update the documentation during an outage or troubleshooting?”
The best place to start with is at the physical layer, at the clients' desk and the destination could be a server/application you use. The client the best place to start because you can verify the cabling, switch configuration, etc easier. Then work your way up the physical layer, to layer2, then 3, etc. If you get stuck or aren’t sure, feel free to reach out to various people/departments for clarification.
You can literally start with a piece of blank paper, Powerpoint, Draw, Impress, heck, even Paint will do. As you build the maps, and more people see the value, you can consider a different application and how to share the information.
In this video, I was doing a physical inventory at a remote office and there was no documentation, so I grabbed a couple of my Cisco switches, enabled SNMP, swapped out the client’s switch, and ran a discover with my Optiview XG. Then I moved on to the next closet.
In this case, I took photos before I started, and documented all the current switch connections. Then I had to come in when no one was in the office, run my discovery and then put everything back. The whole process took an hour and about another 30 minutes putting the documentation together using Powerpoint.