Unpacking the Patch Management Process for Operations

You can’t patch what you can’t see

The idea of an asset inventory isn’t new, and you may have already tried this exercise internally, or even enlisted outside help. But to capture everything is no easy task, and many are still working to get it right.

There are two ways to take inventory, and to set the right foundation for your ICS cybersecurity program, you need both.

• Electronic interrogation tools can scan your network and automatically identify assets, getting you most of the way there.
• Manual identification will catch the rest, but requires someone to literally walk around, open panels and do a physical survey of what’s out there.

A watch out here is to take both approaches at all of your locations. If only complete at nine of your 10 sites, I can just about guarantee the breach is coming through the one that was overlooked.

Setting a comprehensive patching strategy

Following the inventory, you may be left with a list of thousands of assets to wrap your head around. Luckily, not all assets are created equal. The next step is performing a risk analysis to identify the high priority assets to patch based on their criticality, exposure, age, anticipated risk, etc. Some assets aren’t even on the network, so are they really a risk?

There are two types of patches you’ll need to address:

1. Operating system (OS) patching is commonplace for IT, so much so that Microsoft Patch Tuesday has been around for more than 15 years. You’ll have to time plant floor OS patching with scheduled downtime for minimal disruption. Some proactive IT/OT collaboration can take care of this in many instances.
2. Application-level patching is a different story. There could be literally hundreds of applications from different vendors with different patches. So it’s incumbent upon you to go find patches on vendor websites, understand the vulnerabilities they protect against and if they are needed or not.

Because each application is configured differently, patching the application layer warrants a very deliberate, consistent testing standard. One conducted in a lab environment prior to implementation on the plant floor where you could run the risk of unintentionally shutting down production.

A systematic approach to patch management

The “fingers crossed” approach is common throughout the food and beverage industry. Not for lack of trying, but for lack of the right resources and specialized expertise. Generally what I see in the field today is reactive. Responding to a high-priority patch notification and accomplished by shutting down production on a weekend as needed.

And the common progression looks like this:

• Operations enlists IT to help manage OT patching.
• IT fills in, but doesn’t have the ICS expertise or resources to manage the unique requirements and constraints.
• So they hire a hybrid IT/OT resource, or more often, outsource to a company like Rockwell Automation or others.

If going the third-party route, seek a partner grounded in operations. One telltale sign is their service level agreement (SLA) response time. Traditional IT providers measure response in hours. But that kind of downtime in consumer goods production can mean millions of dollars lost. SLAs measured in minutes represent an operations-friendly approach.

The ICS cybersecurity end game

Patch management is one step on your way to getting a security operations center (SOC) up and running. An SOC can provide a holistic dashboard view of your security posture, include a disaster recovery strategy and ensure optimal operation of your connected factory.

Additionally, there are solutions available today that are designed for end point protection or “whitelisting.”  While these solutions do not entirely eliminate the need for patching, they are an effective solution to protect and buy you time while formulating a patching strategy.

The truth is, there is no silver bullet to effective cybersecurity. That is what defense-in-depth is all about. But with more than the bottom line at risk (think food and employee safety), reaction and a little luck is no longer a viable approach. If you’re looking for a little help kicking off your program, or bringing it to the next level, we’re here to help.