Link to the original

AI in OT Is Cool. Now We Just Need to Make Sure It Does Not Break Anything

In December 2025, CISA and Australia’s ACSC, together with the NSA Artificial Intelligence Security Center (NSA AISC), the FBI, the Canadian Cyber Centre, Germany’s BSI, and the national cybersecurity centers of the Netherlands, New Zealand, and the United Kingdom, released a document titled Principles for the Secure Integration of Artificial Intelligence in Operational Technology.

The document addresses what to do when artificial intelligence starts appearing not in yet another chatbot for the sales department, but alongside industrial automation, SCADA, controllers, safety systems, and processes where a software error can end not with a “something went wrong” message, but with very real physical consequences.

In a conventional information system, people can argue at length about hallucinations, prompt injection, and data leakage through a model. In industry, however, another unpleasant variable is added to all of this: the actual physical process.

In short, the document’s main point is roughly this:

The closer AI gets to controlling a physical process, the less it can be taken at its word and the more important it becomes to understand what will happen when it makes a mistake. Not “if,” but “when.”

Do We Even Need AI Here?

Perhaps one of the most sensible questions considered by the authors is to answer not “How do we implement AI?” but “Why are we bringing it in here at all?”

It sounds obvious, but against the backdrop of today’s enthusiasm for attaching a large language model to almost everything, it is a magnificent piece of engineering wisdom.

Before implementation, the document recommends comparing AI with conventional, proven solutions and considering:

  • accuracy and performance;
  • complexity;
  • cost;
  • impact on functional safety;
  • new network connections;
  • data requirements;
  • the organization’s ability to maintain such a system for years.

In other words, if a deterministic algorithm solves the problem perfectly well, replacing it with a neural network solely for the sake of a nice “we implemented AI” slide is not technological progress at all. It is more likely just a new way to add several unknown failure states to the system.

The document provides a fictional example assessing whether AI could be used for predictive maintenance of an industrial generator. It defines the problem, goal, and risk; identifies the stakeholders; and specifies data, performance, safety, and network requirements, as well as success metrics. Such an assessment is needed before selecting the technology to determine whether AI is genuinely better suited to the task than existing proven solutions.

The Closer to the Hardware, the More Interesting It Gets

The authors map potential AI applications to the Purdue Model.

At the lower levels—close to sensors, controllers, and the physical process—predictive machine learning models are predominantly used. Examples include anomaly detection, equipment condition forecasting, telemetry processing, and operator support. Large language models and AI agents are generally expected to remain higher up, in the enterprise environment, where they work with data that has already been exported from the OT network.

This is an important distinction: at the enterprise level, a model’s output can usually be checked before it is passed into the operational environment. At the lower levels, especially when AI participates in local control, an error can directly affect the physical process. The document therefore takes a cautious approach to active control by AI and recommends limiting such scenarios, particularly when no human is in the loop.

In other words, giving a language model access to an OT process control system simply because it understands natural language well remains at roughly the same level of engineering maturity as using admin/admin on an external interface.

Food for the Model

The next problem is OT data.

Industrial AI needs telemetry, archives, events, configurations, process diagrams, maintenance logs, operating-mode data, and much more. The document separates this data into two categories.

  1. Engineering data:
  • network diagrams;
  • asset inventories;
  • logic diagrams;
  • sequences of operations;
  • safety-related information;
  • safety documentation.

Even without AI, this data is highly valuable to an attacker. If it is handed over to an external model provider, uploaded to the cloud, or used in a poorly controlled training pipeline, the attack surface expands.

  1. Process telemetry:
  • voltages;
  • currents;
  • temperatures;
  • pressures;
  • flow rates;
  • frequency;
  • vibration.

These values are generally short-lived, but if a model is trained or fine-tuned on them, the data gains a second life. In other words, if another location appears where the data is sent and stored, that location must also be protected.

Introducing AI changes not only how data is processed but also its lifecycle. A temperature reading might previously have been just one point in a historian. Now it may become part of a training dataset, a copy held by a contractor, an input to a remote service, and a statistical trace within a model.

Bad Data

Classical machine learning likes the phrase “garbage in, garbage out.” In OT, the garbage coming out can sometimes affect real equipment. The document separately considers data quality, data poisoning, and model drift.

If an industrial facility changes—equipment ages, operating modes are reconfigured, new setpoints are introduced, raw materials change, machinery is repaired, and control systems are modernized—a model trained two years ago may continue producing an extremely precise result with six decimal places. That result will simply correspond less and less closely to reality.

This creates several scenarios with similar outcomes:

  • the data was deliberately manipulated → incorrect result;

  • the process gradually changed → incorrect result;

  • the model had never encountered a rare emergency condition → incorrect result.

The first is a cybersecurity incident. The second is an operational problem. The third is a problem with model quality and test coverage. But the operator and the physical process do not particularly care what the department responsible for preventing it was called, because the outcome is the same: the system produced an incorrect recommendation or control action. AI safety, cybersecurity, and operational reliability are gradually merging into a single engineering problem.

Vendors

A separate section of the document is devoted to equipment vendors, which seems particularly relevant for the coming years. The document notes that some OT devices already ship with built-in AI and that their capabilities will become increasingly sophisticated. In practice, this means that AI will not necessarily arrive at a facility as a separate project with a ceremonial launch, a budget, and an “AI” label. It may arrive with the next update or as part of:

  • SCADA;
  • engineering software;
  • a diagnostic tool;
  • an intelligent device;
  • a cloud monitoring system;
  • new equipment firmware.

This raises some highly practical questions.

  • What model does it use?
  • Where does it run?
  • Does the device require a permanent internet connection?
  • Where does the operational data go?
  • Does the vendor use it to train its own models?
  • What happens if the remote service becomes unavailable?
  • Can the AI feature be disabled?
  • Who will notify the asset owner, and how, if the model is found to be capable of producing dangerous recommendations?

The document explicitly recommends requiring transparency from vendors, descriptions of the software component supply chain, data usage policies, and the ability for the asset owner to disable AI features. The last point seems quite reasonable. If a device communicates with the cloud without the owner’s consent, uses an unknown model, and does not allow its “intelligent” feature to be disabled, it begins to look less like an additional product capability.

The Best Channel from AI into OT Is the One That Does Not Exist

One of the document’s most practical recommendations is to avoid giving an external AI system a persistent path into the OT network wherever possible. Data is pushed outward, but the AI system itself does not receive persistent access back into the network. Where possible, one-way transfer, staging buffers, and controlled exchange points should be used.

The logic is simple: if a system only needs to analyze telemetry and advise an operator, it generally has no reason to send anything back. If that capability is nevertheless required, it becomes a separate design decision that needs its own justification, because compromising an external AI system must not automatically create a new attack path into OT.

Human in the Loop

The document says a great deal about human involvement in critical decisions. The more strongly AI can influence a physical process, the more important it becomes to have points at which an action can be checked, approved, or stopped. But there is a catch…

This leads to a practical conclusion that goes beyond the literal text of the document: merely having a human in the process guarantees nothing. A system can formally declare that a human always makes the decision, only for the model to produce hundreds of recommendations whose origins the operator does not understand, while the interface displays a green “everything is OK” message and allows three seconds for review. The human in such a system is mostly decorative. What is therefore needed:

  • clear boundaries on the AI’s authority;
  • independent data sources;
  • predefined safe limits;
  • the ability to reject a recommendation;
  • clear indicators of model failure;
  • decision logs;
  • procedures for operating without AI.

The document specifically emphasizes the last point through the problem of skill erosion. If personnel rely on automated recommendations for years, an emergency may suddenly reveal that the system is no longer working and that people have gradually lost the ability to operate without it.

Testing

For industrial use, showing an impressive metric on a holdout dataset is not enough. The document proposes an engineering approach: begin with a test environment, move on to increasingly realistic testbeds, use hardware-in-the-loop testing where necessary, and transition to the production system only after sufficient verification. Testing should cover far more than accuracy, including:

  • latency;
  • compatibility with existing equipment;
  • false positives;
  • missed events;
  • behavior under unfamiliar operating conditions;
  • model degradation over time;
  • the consequences of losing connectivity;
  • corruption or manipulation of input data;
  • network load;
  • safe system shutdown.

This is a natural approach for OT. With AI, however, some failure states are less obvious, which is precisely why it needs no less testing and, in some cases, even more.

If no one in their right mind would install a new protective relay device or controller at a facility merely because the developer showed an impressive chart in a presentation, why should a neural network be treated differently?

The AI Itself Must Also Be Monitored

After an AI system is deployed, the document recommends maintaining an inventory of all AI components, logging their inputs and outputs, and monitoring acceptable behavioral bounds and deviations.

The model therefore becomes yet another object of technical monitoring. The situation is rather ironic: we deploy AI to detect anomalies, and then deploy another mechanism to detect anomalies in the AI itself.

To prevent a future investigation from turning into archaeology, the authors specifically recommend:

  • monitoring outbound traffic;
  • controlling access to data;
  • conducting adversarial resilience assessments;
  • monitoring model drift;
  • distinguishing AI actions in logs from those of users and conventional service accounts.

When AI Breaks

The authors recommend designing for AI system failure in advance and having immediate answers to the following questions:

  • What happens if the AI becomes unavailable?
  • What happens if the model starts making mistakes?
  • How will drift be detected?
  • How will the system be taken out of the loop?
  • Can it fall back to conventional automation?
  • Is manual control preserved?
  • What will the operator see?
  • How will this be incorporated into the incident response plan?

In other words, AI must not only be able to work; it must also be able to not work correctly. For industrial automation, this is standard engineering logic: the failure of a component should not itself cause an uncontrolled failure of the entire system. The document therefore recommends providing a fallback to conventional automation or manual control and incorporating AI failures into existing functional safety and cyber incident response procedures.

Trust is nice, but a safe state when that trust is lost seems more useful.

What About Standards?

The authors explicitly acknowledge a problem: existing international technical standards for AI are primarily oriented toward the use of AI in information systems rather than industrial automation. Therefore, there is currently no ready-made standard resembling “IEC 62443, but entirely for AI”—this is a practical conclusion drawn from the situation described by the authors, not a literal statement from the document.

The document recommends layering AI requirements onto existing cybersecurity and functional safety processes, while using additional materials from NIST, MITRE ATLAS, ETSI, and other organizations for AI-specific threats.

The result is roughly the following:

AI safety + OT cybersecurity + functional safety = security of AI-enabled OT

The links between these three areas are still described much less thoroughly than each area individually. It is fairly easy to write a requirement stating that a model must be protected against data poisoning, but much harder to answer what exactly will happen to the physical process if poisoning does occur, which control will detect it, what the safe response will be, and which requirement of the existing safety system should cover it.

The Bottom Line

What makes Principles for the Secure Integration of Artificial Intelligence in Operational Technology valuable is its grounded engineering perspective: industrial AI should no longer be treated as a magic box, but as another potentially failing component of a complex system—one that people quickly grow accustomed to and begin to trust. This component:

  • depends on data;
  • can degrade over time;
  • may behave nondeterministically or unpredictably outside tested operating conditions;
  • may require external infrastructure;
  • does a poor job of explaining its errors;
  • creates new attack paths.

The main question to ask before introducing an AI system into OT is: “What will happen to the physical process if the model makes a mistake, breaks down, or an attacker manages to alter its behavior?” If the answer is “well, we hope that never happens,” then perhaps it is still too early to give AI access to a real facility.