When I first signed up for the Telecommunications course in University, nobody prepared me for the kind of job I’m signing up to adopt for the rest of my life. Don’t get me wrong, I love being a network engineer, but back in the beginnings of my career, I was crying for help. I was cursing the moment I decided to pursue a career in this domain and felt completely hopeless.
Ah, those days.
Although I had multiple chances to just give up and become a doctor or something, it has never been in my blood to quit. So, I took the challenge: shift life, network problems I never heard of and of course, following the weather cast.
A lot of my friends had no idea what a network engineer does for life. And after many visual explaining sessions – they still don’t. So, I this post is for them and everyone who wants to know what the professional life of a network engineer is like.
Network Engineers never sleep
Well not literally. Networks and data centres need constant monitoring, as the main source of problems that may disrupt their stability is very unpredictable. The environments that use NOCs need very high and stable availability, usually meaning uptime – much higher than normal.
Keeping in mind that the source of problems for networks is often the weather, power outage, human element and incompatible network changes that often aren’t evaluated properly and cause severe incidents – it’s only expected for a NOC to operate 24/7/365.
You may think that a network problem can’t be that big of a deal if it’s not predicted and stopped on time, so let me just roughly visualize the horror a network failure can bring to companies:
- An entire fleet of airplanes can be grounded urgently because of a data centre problem. Delta Airlines is an example;
- A one-day service outage at Salesforce.com cost the company $20 milion.
- A series of delays and slowdowns from the Bank of America’s website that lasted for 6 consecutive days, affected 29 million online customers – not so good for the Bank’s image nor for its operations.
Conclusion: if the network doesn’t sleep, neither do we. Welcome to the life of a network engineer.
Having said this, working in shifts means going to work in the middle of the night and coming home at dawn or even when the day has long begun, and all your neighbours are gone to work. Sometimes it’s like everyone else’s work day: a 9-5 job. And sometimes, you’re left with just a few hours at night to see your loved ones, grab a bite and crush in bed.
But that’s just shift life.
NOC Duties and Responsibilities – The Fun Part
We don’t just stare at screens, if that’s what you thought we do. The duties and responsibilities of a noc engineer are derived from the main goal of the position: making sure the network we’re monitoring is stable and provides high availability at all times.
- When PC techs escalate a Network, VPN or a Wireless issue, we’re the guys that fix those issues fast and clear;
- Monitoring the Bandwidth, IPFlow as well as the Syslogs and Firewall to detect and fix issues is in our mental box, a way of thinking;
- Creating and proposing projects for improving network performance since who doesn’t want to be better than they were yesterday ;
- Troubleshooting problems with the Dynamic VPN and IPSEC tunnels;
- Day-to-day operational support of network infrastructure and services maintaining, highly-available network infrastructure;
- Where we stumble, we troubleshoot technical issues together with more experienced engineers – always a rich experience;
- Monitor, manage and implement network security, traffic, and performance;
- Sharing the knowledge with co-workers and give the term teamwork new meaning – we don’t just robotically do our tasks – we learn from each and every one of them.
The Setup – Our Cool Offices
Part of what makes my job fun is that the offices we’re located in aren’t just a vague NOC with the cold ambiance with all the screens and phones around. As a team, we’re tightly connected, and it matters to all of us that our working environment works with us, especially when we need to spend a night together (no pun intended)!
So, the basic NOC is often a centralised location where the NOC technicians are based and from where the monitoring, troubleshooting and maintenance of a network’s endpoints is done. That location can be as large as an entire building floor, or a small office – depending on the complexity of the network in question as well as how many engineers are employed to work there.
Regardless of the size, this centre usually would have at least 2 status monitors in front of every engineer, phones for communication, desks with under space for backups, passwords or specific hardware cryptographic devices.
There would also be a TV set for news broadcasting, as engineers need to keep track of potential weather or other (un)announced threats to the network’s uptime and stability.
I can’t say that stress levels are high for a noc engineer; maybe because it’s never the network! Most of the issues that come to you have already been picked up by the Sys Admins and the PC Techs, so you’re often left with little to do.
However, even if the tasks that come to you seem small and you give it little credit in complexity, that doesn’t mean you don’t have to pay proper attention to it. Remember, human element is one of the factors that causes the problems in this domain.
So, if you’re inexperienced and never worked with networks before, you may want to think of an alternative option, like outsourcing noc to experienced technicians with who you can work with.
Network Operations Centres best practices
There are 5 elements that make a NOC truly effective: a solid and fast ticketing system, a centralised knowledge base with all documentation available to the entire team for reference, reporting, effective monitoring of the infrastructure and the user experience and IT Process Automation. In addition, your engineers’ technical knowledge needs to be supported with very strong analytical and problem-solving skills, so they’re able to pick up issues and react to them in timely manner.
I know just how hard it is for companies to build a strong NOC team. Shift life isn’t for everyone, and in most cases that is the main contributing factor to the high costs per network engineer. But thanks to the nature of the work and the fast digitalisation, today you can still have your noc dedicated to you and working your work hours, by outsourcing it.
Depending on the provider you choose, outsourcing noc usually implies a fully equipped centre and a team made of experienced engineers who know where to look for the problem and how to best fix it.
Of course, there are the outsourcing providers who offer a slightly different, and more advanced outsourcing, more like a hybrid model of traditional outsourcing to a third party and growing their in-house teams with external resources – far more effective, as the dedicated employees are assigned exclusively to one project.
So in conclusion, if you’re in telecommunications, you want to have a solid and well-organised NOC. It will be so much easier to have support in resolving the complex network issues and it will save you a fortune as big issues will be quickly resolved. Think of it as an inevitable problem-detection and resolving machinery working 24/7 to keep your business up and running.