In this little write-up, I wanted to share my latest experience at a customer site.
As part of troubleshooting, I always suggest we ‘clean-up’ and refresh or validate their documentation. I can’t remember how many times this simple exercise resulted in actually ‘fixing’ the problem. I love watching the look of confusion in the customer’s face as they mutter, “that’s not supposed to be connected to that”, “this was supposed to be decommissioned a long time ago”, or the ever popular, “who plugged this in?”.
Label, label, who’s got the label
In this case, I noticed some switches had labels indicating the IP address of the switch. Great, a label, not a common site, and I was impressed. The odd thing about the label was that I noticed it wasn’t within their addressing range. I asked if they had a management VLAN with a different IP addressing scheme, to which the customer replied, “yes”. When I probed a bit deeper and queried if the address on the label was within the Management VLAN addressing, the answer was, No.” Of course the IP address on the label was not reachable, nor within the customer’s addressing ranges.
One of the technicians recognized the IP address and said that the IP range was the LAB subnet. It seems that the LAB technicians labeled everything when they were testing and configuring the switches, but the labels were never physically removed when the equipment was deployed to the production environment. Not a big deal if you work there, but very confusing for contractors like me.
Passwords
This one is becoming my favourite; as I worked towards cleaning up the computer room, I noticed this curled up label on the core switch and similar labels on other equipment. Can you guess what the labels were? Hold on to your socks, it was the telnet/enable password of the equipment. AND THEY WORKED!!!! When I asked the customer why they thought this was a good idea, they said this was done because the analysts kept forgetting the passwords, so it was easier to put them on a label. He added that since the computer room has restricted access, having the passwords written on the labels posed no security risk.
This next part is so crazy, its believable. I telnetted into the switch and this was the banner configuration;
The telnet and enable password was part of the switch banner configuration. Hmmmmmmmmmmmmm…. It seems this is common practice in the lab but someone forgot to remove that part of the switch configuration when it was put into production. Of course everyone thought someone else would review or clean up the configuration.
The moral of the story is to keep your eyes open and review configurations every so often.