Principal AI Hardware Architect

Job Title

Principal AI Hardware Architect

Role Summary

Join Microsoft Azure Hardware Systems and Infrastructure (AHSI) — AI Systems Architecture (ASA) group — to define and optimize next-generation AI accelerator platforms and large-scale AI systems. The role focuses on performance modeling, workload characterization, profiling, and end-to-end performance analysis across GPU and accelerator architectures.

Experience Level

Senior (Principal-level). Typical expectation: 7+ years of technical engineering experience; equivalent practical experience accepted.

Responsibilities

Analyze and optimize hardware and system-level performance for AI workloads; collaborate across hardware, software, and systems teams to influence architecture and product decisions.

• Lead performance analysis, profiling, benchmarking, and analytical modeling for GPU and AI accelerator architectures.
• Characterize end-to-end AI workloads including model execution, runtime behavior, memory systems, communication collectives, and workload mapping strategies.
• Develop system-level and efficiency models to evaluate architectural features, memory/interconnect innovations, and TCO/perf-per-watt trade-offs.
• Correlate silicon measurements, software traces, and kernel execution with models and simulators to validate assumptions and improve fidelity.
• Drive kernel-, runtime-, and system-level performance optimizations for training and inference workloads.
• Design and implement tooling for data analysis, correlation, visualization, and performance modeling.
• Partner with architecture, microarchitecture, compilers, runtimes, networking, and systems teams to evaluate trade-offs and influence roadmaps.
• Present technical findings and recommendations to senior technical leadership through reports and architecture reviews.

Requirements

Minimum qualifications and skills required to perform the role.

• Minimum 7+ years of technical engineering experience (equivalent practical experience accepted).
• Proven experience in performance profiling, benchmarking, and root-cause analysis using hardware counters, software traces, and workload measurements.
• Hands-on experience analyzing and optimizing AI kernels and relating kernel behavior to system-level performance.
• Experience developing performance, efficiency, or TCO models and using architectural simulation or analytical modeling.
• Proficient programming skills in Python and C/C++ for tooling, benchmarking, automation, and data analysis.
• Strong written and verbal communication skills; experience presenting technical analyses to stakeholders and leadership.
• Ability to meet Microsoft Cloud background check and other required security screenings for this role.

Nice-to-have:

• Deep understanding of GPU and AI accelerator architectures: compute pipelines, memory hierarchies, interconnects, collective communication, and parallel execution models.
• Experience with workload characterization, silicon correlation, and performance modeling for accelerators and large-scale AI deployments.
• Familiarity with AI frameworks and serving stacks (for example, PyTorch, vLLM) and distributed training/inference frameworks.
• Knowledge of modern AI optimizations (quantization, sparsity, sharding, KV-cache, Flash Attention) and communication-computation overlap strategies.

Education Requirements

Degrees mentioned in the posting include Bachelor's and Master's (MS) and PhD in Electrical Engineering, Computer Engineering, Mechanical Engineering, Computer Architecture, Computer Systems, Machine Learning, High-Performance Computing, or a related technical field. The posting also accepts equivalent practical experience in lieu of formal degrees.

About the Company

Company: Microsoft

Headquarters: Redmond, Washington, United States

Microsoft is a global technology company that develops and sells software, services, devices, and solutions. Known for its Windows operating system, Office suite, and Azure cloud platform, Microsoft aims to empower individuals and organizations around the world to achieve more. The company fosters a culture of innovation and inclusion, focusing on delivering trusted experiences to customers and partners globally.