Principal Software Developer – AI/ML Performance Validation & Systems Testing

Job Title

Principal Software Developer – AI/ML Performance Validation & Systems Testing

Role Summary

Senior individual-contributor responsible for technical leadership of ROCm validation across compute workloads and server-class systems. Own the validation strategy, test infrastructure, and release-qualification criteria for ROCm on AMD Instinct GPU platforms.

Work includes end-to-end testing (unit to multi-node systems), performance and stability characterization, and cross-team debugging to ensure production readiness for hyperscalers, OEMs, and open-source users. Location: San Jose, CA (hybrid).

Experience Level

Senior-level (Principal). Typical experience for this role is 7+ years in software validation, SDET, or quality engineering; equivalent demonstrated experience accepted.

Responsibilities

Lead validation architecture, test infrastructure, and release gates; drive debugging and qualification of ROCm across platforms.

• Define and own end-to-end validation architecture: unit, integration, workload, performance, stress, stability, scale-out, and system-level tests across GPU generations and servers.
• Establish release qualification gates and exit criteria (functional coverage, performance regression thresholds, stability hours, scale targets, RAS criteria).
• Architect distributed test infrastructure: CI (GitHub Actions/Jenkins), test runners, hardware lab orchestration, result data lakes, flaky-test detection, and bisection automation.
• Promote shift-left quality practices: test pyramids, hermetic environments, deterministic reproducers, and continuous validation in trunk.
• Set GitHub-based quality workflows: PR gating, required checks, code coverage standards, and disciplined issue management.
• Lead complex escalations and cross-team debug with development, hardware, firmware, and customer teams; translate findings into durable test coverage.
• Influence product and platform roadmaps to ensure validation readiness before tape-in and silicon arrival.
• Mentor and elevate senior validation engineers, SDETs, and SQA leads through reviews and technical guidance.
• Represent validation externally with customers, OEMs, and open-source communities.
• Lead system-level testing for server nodes: multi-GPU topologies, interconnects (PCIe/xGMI), BMC/IPMI, thermal/power, firmware, and multi-node fabric bring-up.
• Drive compute workload validation and characterization for LLM training/inference, recommender systems, HPC kernels, and MLPerf-class benchmarks.

Requirements

Must-have

• Proven software engineering experience in validation, SDET, or quality engineering; experience leading complex systems validation programs.
• Expert Python for test automation and infrastructure; strong C++ for debugging and extending production code.
• Experience leading validation or qualification programs for hyperscalers, OEMs, or Tier-1 customers.
• Experience with performance validation, profiling tools, and regression analysis.

Nice-to-have

• Validation expertise in GPU software stacks (ROCm, CUDA, oneAPI, SYCL) and AI/ML frameworks (PyTorch, TensorFlow, JAX, Triton, vLLM).
• Experience with HPC runtimes and communication libraries (MPI, RCCL/NCCL, UCX, Libfabric) and large-scale cluster software.
• Experience validating multi-GPU, multi-node server platforms including stress, soak, fault injection, and RAS testing.
• Contributions to ROCm, PyTorch, LLVM, Triton, vLLM, or similar open-source validation/CI projects.
• Experience with agentic AI workflows for testing and debugging (automated testing, AI-driven debugging, RAG/MCP workflows).
• Experience operating large-scale GPU clusters (256+ GPUs), fabric bring-up, health monitoring, and diagnostics.
• Familiarity with hardware lab automation (BMC/IPMI/Redfish, PDU control, serial consoles, automated re-imaging, topology-aware scheduling).
• Experience supporting pre-silicon, emulation, and first-silicon accelerator bring-up.

Education Requirements

BS, MS, or PhD in Computer Science, Computer Engineering, or related technical discipline — or equivalent demonstrated experience.

About the Company

Company: Advanced Micro Devices

Headquarters: Sunnyvale, California, USA

Advanced Micro Devices, or AMD, is a global semiconductor company that designs and manufactures microprocessors, graphics processors, and related technologies for a variety of computing devices. Known for pushing the boundaries of innovation, AMD's mission is to deliver high-performance computing solutions for AI, data centers, gaming, and embedded applications. They foster a collaborative, inclusive culture focused on creativity and problem-solving, aiming to drive progress and excellence in technology.