In The Oil and Water Moment in AI Architecture, Shweta Vohra articulates a tension many architects are already encountering: Deterministic systems and probabilistic AI do not naturally coexist.
Traditional software is built on predictability, where the same input produces the same output. This predictability permits testing, debugging, and clear guarantees. AI systems, particularly those powered by LLMs, break this model. Their outputs are inherently variable, shaped by probabilities rather than fixed logic. The result is an "oil and water" moment, where two fundamentally different behaviors must operate within a single architecture.
The implication is a shift in design mindset. We are no longer engineering purely for correctness, but for bounded uncertainty. This shift defines acceptable outcomes rather than exact ones, and introduces guardrails such as validation layers, human-in-the-loop checks, and deterministic fallbacks when confidence is low.
Observability also evolves, with latency and error rates that are no longer sufficient. Architects must track output quality, drift, and alignment with user intent. In practice, tracking pushes teams toward tighter feedback loops and more continuous evaluation, closer to managing a sociotechnical system than a traditional service.
A pragmatic pattern is emerging. Rather than embedding AI deep within critical paths, teams encapsulate it behind well-defined interfaces. This approach helps contain unpredictability while still enabling high-value use cases such as summarization, recommendation, and decision support.
The takeaway is clear: AI is not just another component. It introduces a different operational model, and architects who succeed will embrace probabilistic thinking, design for uncertainty, and treat AI as a constrained collaborator rather than a deterministic service.
This content is a short summary of a recent InfoQ article by Shweta Vohra, "The Oil and Water Moment in AI Architecture".
To get notifications when InfoQ publishes content
on these topics, follow "architecture and design", "sociotechnical architecture", and "culture and methods" on InfoQ.
Missed a newsletter? You can find all of the
previous issues
on InfoQ.
Sponsored
|
|
Struggling with brittle, hard-to-debug data pipelines and inconsistent orchestration across your stack? Learn how to design, build, and operate reliable workflows with Apache Airflow. This comprehensive eBook walks through DAG design, scheduling, testing, and production best practices—equipping architects to standardize data pipelines, improve observability, and scale data-driven systems with confidence.
Download "Data Pipelines with Apache Airflow (By Manning)", sponsored by Astronomer
|
|
