IT terms such as “bug” and “debugging” have their origins in the era of giant mechanical computers that more closely resembled factory floors than modern devices. Bugs would literally be found in IT systems and prevent them from working as they should. They were the original messy technology issue, quite literally, but they are far from the last. 

We like to think of the manufacturing sector as a slick, well-oiled machine where tasks are completed like clockwork by man and machine working together. However, when you look closer, you start to see the bugs in the machine everywhere. Even the most modern and technologically advanced sites have operational, technical and personnel issues that make operational priorities like cybersecurity a lot more complex, and a lot more chaotic, than outsiders imagine. 

Toyota is a perfect example of cyber security complexities and their consequences. A data breach impacting a single plastics supplier put the brakes on Toyota’s global production of vehicles in March 2022. The breach delayed the production and sale of tens of thousands of cars, at great financial and operational cost to the company. It’s proof of how little room for error is involved with just-in-time production control systems, and the urgent need for digital assets to be protected. Most factories have this issue, as well as challenges around basic cyber security hygiene, M&A activity, patching, insider threats, espionage and employee training. The end result is that cybersecurity in manufacturing is anything but clockwork. It’s seriously hard work. 

Below are some of the most common people, process and technology challenges facing manufacturing security teams, as well as some advice for how they can be mitigated. 

Technical debt in manufacturing

Manufacturing is a data-rich sector, and this data is used as an intelligent resource to guide decision-making and optimise operations. It is one of the reasons that modern manufacturing companies have hundreds, if not thousands, of connected devices and industrial controls on site. 

Unfortunately each one is a potential entry point for attackers or a point of failure if they are not kept up to date and secured. Patch management is therefore essential for security teams, ensuring that known vulnerabilities are identified and fixed continuously. However, old and outdated machines are a big problem. Even manufacturers at the cutting edge of robotics and IoT tech will usually have some legacy machines on the floor too. Often these are machines that are too old to be updated with the latest security patches, but too expensive or important to be replaced. Many won’t have been designed with cyber and data privacy in mind. In critical infrastructure and manufacturing, sometimes devices cannot be updated and restarted because organisations cannot afford the downtime. 

ICS/SCADA equipment is behind the curve when it comes to security, and could never be described as “secure by design”. To make matters worse, ICS/SCADA systems rely upon niche protocols which were designed to work effectively in environments with low computational power. Infosec was never a consideration in their design. These niche protocols can present challenges to standard infosec tooling.

Manufacturing also has a device variety and complexity issue that is very different to the average office workplace. It is not a simple case of securing a laptop, PC or smartphone per employee plus a few other connected devices – which can all invariably run similar platforms and security controls. Factories have myriad connected devices of every size, shape, purpose, and manufacturer. They may be accessed more easily and readily than the CEO’s laptop, they can be damaged on site, and getting data from these devices to see if they are behaving as expected isn’t always straightforward.   

It’s hard for manufacturers to go on the front foot against potential attackers and adopt the latest security controls used by other industries such as finance when they are in this state of ‘technical debt’. CISOs will be acutely aware of AI and automation tech and want to invest more in security, but the priority for them must be the basic security hygiene of patch management and locking down machines. In this scenario, security teams need to focus on walking before they can run. The pursuit of future perfection cannot come at the cost of achieving good security processes in the here and now. This can mean ignoring more advanced security tools in the short term and focusing on the basics that align with business priorities – eg. ensuring that cyber incidents do not impact a factory’s ability to run on schedule and on budget. In practice, this hinges on an organisation’s ability to detect threats and potential cyber incidents in their infancy. It also means having a detailed plan in place to respond to breaches, and what the company will do in the event of a ransomware attack, for example. 

It should be noted that the manufacturing industry is estimated to have spent more than any other sector last year on ransomware payments. This suggests that the industry is a primary target, due to its soft underbelly. Few manufacturing companies have a plan for how to combat the threat beyond paying the demand. 

Business priorities in manufacturing

The priorities of manufacturing bosses and security teams broadly align. Both want to eliminate downtime and ensure that sites and teams run as efficiently as possible. However, other business priorities around growth can become major issues for security teams. 

Mergers and acquisitions are extremely common in manufacturing, which inevitably introduces new people, new systems, and new technologies to an organisation’s operations. Suddenly, a security team may be responsible for securing a whole other company, site, or even an entire region. Management naturally wants to break down silos as quickly as possible. They want everyone and every department to be using the same tech, processes and systems as each other for the sake of efficiency. 

However, security teams know that each new department adds to their IT scale and complexity, increasing their potential attack surface even further. Their primary concern is that a potential breach in one of these previously unconnected departments can have serious ramifications for others. They may also be inheriting the IT and security issues of another company, on top of their own problems.     

When you work in an environment where you cannot patch, then the primary solution must be segmentation. Rather than “patch, patch, patch”, the correct motto should be “segment, segment, segment” – ideally following the Purdue model. Opp codes will need to be linked, but thankfully the Purdue model for separating layers of technology in fields like Industrial Controls (IC), Supervisory control and data acquisition (SCADA) and Internet of Things IoT has existed for a long time. It has stood the test of time because of its effectiveness. 

It’s also vital that security teams have complete oversight of the entire estate, no matter how big it has grown. They must have the controls put in place to analyse traffic, spot malicious activity and limit traffic 24/7. Often this Security Operations Control (SOC) is outsourced if a manufacturer does not have the skills or manpower to monitor this activity in house.  

A common security issue in manufacturing stems from the common use of Systems Integrators. Vendors are generally abstracted from customers by a systems integrator, which tend to have a very rudimentary infosec understanding. Even though vendor knowledge is improving, that is effectively nullified by the SI.

When a production machine is installed by a systems integrator, it will be delivered in an insecure, yet functionally tested state. The systems integrator may not allow patching irrespective of the CVSS score of a vulnerability present within their software. They are too concerned by the extensive regression testing required. Sis and management may also insist on remote access, regardless of their ability to deliver remote access in a secure manner. 

People problems and insider threats in manufacturing

Manufacturing’s people problems broadly reflect its connected device problems – namely scale, variety and complexity. Manufacturing employees are a varied group, including on-site, office-based, and roaming employees, as well as third-party contractors and suppliers. 

Most workforces are not chained to their desk and a single device, and it’s likely that they are less IT security savvy than their office counterparts who spend more of their time online. Manufacturing also has a physical security / access issue. People need to move freely on site and between multiple locations and devices, while simultaneously ensuring that devices are only accessed by people with the right permissions. Again, all this creates a large attack surface for cybercriminals.  

The sector also suffers from a lack of skilled ICS/SCADA infosec professionals. All sites need experienced people who understand that the normal rules of “patch, patch, patch” don’t always apply. All employees need formal training in how infosec works in an ICS/SCADA environment.

Meanwhile, there is also the insider threat issue. Manufacturers often have extremely valuable IP that may be the result of years of R&D. It is a lucrative target for competitors and cybercriminals alike, and it is common for them to target someone on the inside to give them the access they need. According to Verizon’s Data Breach Investigations Report, 28% of manufacturing industry data breaches were motivated by espionage in 2021.

To combat personnel issues and insider threats, manufacturers should prioritise regular employee training sessions so that staff know how to identify phishing attempts and what to do if they receive any suspicious communication. The ability to monitor all employee devices and detect unusual activity among staff is also vital. Companies need to know if someone is accessing files or machines that they are not permitted to use, or don’t usually. Likewise, they need to spot abnormal behaviour that may indicate an insider threat, such as downloading files en masse, or accessing files from another location/out of office hours. These are all activities that warrant closer scrutiny and investigation.    

A bug free future for manufacturing

While we no longer have the distasteful task of clearing dead bugs from giant computers, new, complex challenges emerge every day for manufacturing security teams to contend with. 

While great strides have been made in recent years in regards to security controls and resources, we have to admit that most organisations in this sector are not ready for them yet. There is a messy reality to contend with now, but doing so will ensure that cyber incidents do not impede an organisation’s ability to deliver goods and services on schedule and on budget.

Andrew Kays is the CEO of Socura