“You’re cutting into my overtime. But if I can schedule upgrades to happen overnight and sleep better, I’m in!” This is what a network engineer recently told me as I was discussing network automation.
Network infrastructure owners I speak with are looking for ways to maximize the value from their investments. Meanwhile, their teams are looking for relief from repetitive tasks so they can focus on strategic initiatives. Unfortunately, many of the approaches they attempt do not effectively address these challenges.
Manual Limitations
Historically, keeping devices up to date and reducing risk to the organization has been done manually, one device at a time. These tasks include ensuring configurations are compliant, updating software and/or firmware, backup and recovery, and patching critical vulnerabilities.
A manual approach is effective when managing a dozen or fewer devices. But when you have tens, hundreds, or thousands of devices across multiple vendors, manual methods become impractical. NetOps teams spend nights and weekends on these repetitive tasks, yet they still struggle to keep up and face burnout. Organizations have two options: hire more people or automate. Hiring additional resources is great, but even the largest teams struggle to keep up.
So, clearly, it’s time to automate. Why, then, are 67% of enterprise networking activities still being performed manually?
The answer lies in the paths to automation that many teams try first.
Brush up on Coding Skills or Ask AI
If you are a network engineer with some coding background or experience with scripting languages such as Python or Shell, you might be comfortable scripting a single task. For example, pushing an upgrade or configuration change across multiple devices or validating configurations to ensure compliance.
Alternatively, you could ask AI to write code for you, and it will provide a result. However, if you are not an expert in coding, you can’t be sure the AI output is correct. Having a confidence level of 70% to 90% doesn’t cut it if your organization relies on the network to deliver critical services. A network outage in a hospital could mean tracking requisitions and lab results on pen and paper, requiring them to hand-deliver information between floors. Similarly, an outage for a food manufacturer could stop production, disrupting the entire food supply chain. Many organizations recognize and can calculate the cost of every minute of downtime, and the potentially significant implications for patients, customers, and partners.
The challenge with coding is amplified when you need to automate multiple tasks across various device types. If you’re not an expert coder, it’s easy to make mistakes when tweaking or updating scripts for different scenarios. It’s essential to include built-in checks to ensure your scripts are working correctly. Suddenly, that “quick script” turns into a thousand lines of code, leading to headaches in maintenance.
Use Tools Provided by Device Vendors
Device vendors offer tools to automate certain tasks. However, if you work with multiple vendors, you need to learn to use and manage multiple tools.
You also need to do tasks sequentially, moving from vendor to vendor as you back up or update devices. Depending on the sheer number of devices and vendors in your environment, a simple task like a backup can take days or weeks to complete. Multistep workflows, such as multi-version upgrades or configuration changes, may take even longer. What started off as an efficient tool for one-off tasks ultimately doesn’t scale.
Getting to Business Value
The corollary to 67% of tasks being performed manually is that 33% are being automated. Often this is done with a sustainable approach using pre-built automation and no-code customization. These organizations are leaning into automation to achieve immediate business value, understanding that it serves as a stepping stone to more strategic value AI can unlock. Here are a few examples of the impact NetOps teams are seeing.
- Faster change implementation: A small team may only be able to manually upgrade devices at a rate of two devices a week. This means they face a choice: either spend years completing the upgrade cycle or wait until the end of life for some devices. While waiting can save time, it also comes at the cost of reduced device performance and functionality, as well as risk exposure to unmitigated vulnerabilities. In contrast, by using automation, a financial institution successfully upgraded 22,000 devices in just six days.
- Improve service reliability: It’s important to verify that backups are successful. However, when it comes to disaster recovery, what about the ability to restore? Verifying that restoration works correctly at scale is very challenging. A multi-step automation process can validate restoration after nightly backups as a best practice to strengthen resilience.
- Ensure compliance: Rather than writing scripts to check device configurations against standards like CIS benchmarks, teams are now automating the workflow. The automation matches each device to the correct configuration, verifies devices, notifies engineers of configuration drift, and can even automatically remediate misconfigurations.
- Reduce security risk: Many network teams lack confidence in the results of their weekly or monthly vulnerability scans, leading them to conduct manual validation. This verification can take months to complete, leaving the organization exposed to threats during that time or a false sense of risk if the findings turn out to be false positives. Some organizations completely ignore network devices, only tracking vulnerability risk on servers and workstations, which itself is a risk. Automation can seamlessly track, prioritize, and remediate vulnerabilities across the network infrastructure, enabling teams to demonstrate their impact on reducing security risks.
- Lower operating costs: The operational costs associated with building and maintaining network devices are significant and increase with the complexity of modern networks. Automation offers a sustainable way to reduce overall costs. For example, a financial institution is automating two tasks per day across all their network devices, resulting in an average annual savings of $6 million.
- Increase agility: A network admin recently mentioned they have a strict service level agreement of 24 hours to address critical vulnerabilities and had no reliable way to upgrade devices at scale. Implementing automation for this purpose has been life-changing.
Streamlining overtime without cutting corners ensures the business is able to rely on the availability and resilience of the network to run its business. Network automation frees teams to focus on what drives the business forward, while infrastructure owners demonstrate risk reduction and solid value from critical investments.
Irfahn Khimji is a seasoned technology leader with a deep background in cybersecurity and go-to-market strategy. In his current role as Field CTO at BackBox, he leverages his extensive expertise to provide technical leadership and strategic direction. Recently, Irfahn served as Managing Director for Critical Start in Canada. His career progression highlights a strong focus on sales and leadership, including his tenure as Vice President at Tripwire, where he led sales, sales engineering, and channel teams for North America. He also served as the Managing Director for Qualys in Canada, overseeing technical sales, marketing, and services.
The opinions expressed in this post belong to the individual contributors and do not necessarily reflect the views of Information Security Buzz.