Early steel-frame high-rises were not trusted simply because steel was new and modern. For example, concerns were contemporaneously raised in Chicago in March of 1884 regarding the fire safety of one of the world’s first steel “high rise” buildings. These now common modern buildings became trusted because the full system was engineered to handle uncertainty through sound engineering, safety margins, inspection, monitoring, and failure containment. Similar trust issues currently surround AI’s use in many security domains. For AI-powered security systems to become trusted, a similar systems engineering approach must be applied.
Engineers liked the steel framing of high rises because it solved real problems of height and load, but critics worried about fireproofing, wind, and the quality of early steel. These were variable, safety-impacting factors that had to be addressed in the design of the building as a system. Similarly, in enterprise IT and security, AI is showing real promise in improving the prevention, detection, and response to threats, but it is also raising concerns about reliability, transparency, bias, data leakage, and the risk of overdependence on systems that can behave unpredictably. These are not reasons to reject AI outright; rather, they are factors that must be understood and managed through architecture, governance, controls, and ongoing oversight, if AI is to become a trusted part of enterprise security systems.
Why Systems Engineering?
Systems engineering is an interdisciplinary field focused on designing, integrating, and managing complex systems over their full life cycle. It brings together technical, organizational, and human factors to ensure that all parts of a system work together effectively. Security teams and vendors are increasingly being tasked with what is essentially a systems engineering challenge. They are being asked to trust systems with AI analytics inside the SOC, within IAM systems, and integrated into many other security controls. Once security systems use these non-deterministic AI elements, they introduce valid questions about reliability, consistency, explainability, and control.
Implications for Market Participants
This has direct implications for security vendors. AI should not be seen as a magic analytic layer. Instead, it should be designed and presented as a vital part of a larger security system and process. Buyers need transparency about how AI is trained, updated, constrained, and monitored. While explainability with AI will never be perfect due to the complexity of the underlying models, operational clarity is still achievable. Vendors should clarify where AI is advisory, where it is assistive, and where it can act independently. In practice, data engineering, guardrails, workflow design, and monitoring may be just as important as the sophistication of the models.
The implications for enterprise security leaders are just as crucial. AI security analytics systems should be evaluated similarly to how critical infrastructure is assessed. What decisions does the AI system influence? What are the consequences when it makes mistakes? What data quality does it rely on? How is its performance tracked over time? When is human validation and oversight necessary? What operational limits should be set for autonomous AI agents?
Safety Margins and Redundancy
The high rises of both the past and present are not trusted because every component is perfect. They are trusted because the entire system is designed to handle uncertainty and variability in its operating conditions. Modern buildings incorporate safety margins, redundancy, layered controls, inspection, testing, and ongoing monitoring, and are designed to fail gracefully rather than catastrophically. Above all, the modern skyscraper is trusted because the system is validated in real-world operation over time, not just through theoretical engineering calculations.
In building design, structural engineers work from assumptions about expected loads, wind, occupancy, and other variable conditions. In AI security analytics, vendors and enterprises need the same clarity about model assumptions, intended use, operating conditions, and likely failure modes. If those factors are poorly understood, the overall system's reliability is weakened.
Buildings also rely on safety factors and redundancy. No civil engineer assumes that every component will always perform perfectly. Not every piece of steel has the same strength characteristics. AI security systems need the same mindset. Confidence thresholds, deterministic fallbacks, layered controls, and clear limits on automated action are not optional add-ons. They are a required part of the system design.
Building codes and related inspections (audits in GRC terminology) provide another obvious parallel. Buildings are governed by standards, review processes, and inspection regimes. AI-based security analytics likewise need governance, testing, validation, auditing, and documented quality controls. In the current security market, many products present AI as a key feature. Buyers should instead ask whether it has been properly engineered as a subsystem as part of a larger system.
Instrumentation and Maintenance
Modern high-rise buildings are also comprehensively instrumented. They include structural monitoring such as strain gauges, accelerometers, tilt, vibration, and displacement sensors. They rely on fire and life-safety systems, HVAC telemetry, electrical monitoring, elevator diagnostics, plumbing and water monitoring, and security and occupancy systems. Together, these monitoring systems provide continuous visibility into whether the building is operating within expected parameters. AI security systems require the same kind of instrumentation. Telemetry quality, runtime monitoring, model drift detection, and continuous performance reviews are essential if these inherently probabilistic components are going to be trusted as part of a system in continuous use.
The same applies to maintenance and containment. Buildings are not finished once initial construction ends. AI models also need recalibration, retraining, and periodic review as attack patterns, data quality, and operating environments change. And just as buildings are designed so that localized failures do not become disasters, AI-based security systems must be designed so that the wrong outputs do not cause unacceptable business disruption, especially as AI agents move closer to executing more autonomous actions.
We do not reject skyscrapers because elevators occasionally fail, components require regular replacement and maintenance, or roofs sometimes leak. We trust them because the overall system is engineered to operate safely despite uncertainty, component limitations, and periodic subsystem failures.
AI-based security systems should be judged not by whether they are perfect, but by whether they improve outcomes when engineered with controls, transparency, and continuous monitoring. That is the shift explored more fully in the Advisory Note, From Deterministic to Probabilistic Security: Why AI Is Foundational to Cybersecurity.
