Beyond the Model: The Industry Playbook for Scalable AI Systems

As artificial intelligence transitions from experimental deployments to mission-critical production infrastructure, organizations face a fundamental shift from model-centric optimization to system-centric engineering. While advances in model architectures and accelerator technologies have driven recent AI breakthroughs, long-term performance, reliability, and sustainability increasingly depend on the interaction between compute, memory, networking, software runtimes, operations, and governance. This talk presents an industry roadmap for building scalable AI systems that move beyond isolated model optimization toward adaptive, software-defined AI platforms. The roadmap explores five interconnected layers—compute, memory and data, interconnect, runtime and operating systems, and operations and governance—and demonstrates how these layers collectively influence throughput, latency, cost, energy efficiency, reliability, and compliance. The discussion introduces workload-aware architectures for inference, retrieval-augmented generation (RAG), agentic workflows, multimodal applications, and edge AI, highlighting the growing importance of memory hierarchies, topology-aware scheduling, adaptive control loops, and cluster-scale orchestration. A practical AI systems maturity model is proposed to help organizations assess current capabilities and prioritize investments, progressing from ad hoc experimentation to autonomous, policy-governed AI fabrics. The presentation concludes with a pragmatic execution framework and industry best practices for achieving predictable service levels, operational resilience, and sustainable AI economics. The central thesis is that future AI leadership will be determined not by model performance alone, but by the ability to design, operate, and govern AI as an integrated systems platform
Co-sponsored by: Vishnu S. Pendyala, San Jose State University
Speaker(s): Sujit Reddy Thumma
Virtual: https://events.vtools.ieee.org/m/563401