Technical Product Manager - AI Infra Resilience
NVIDIAJob Title
Technical Product Manager - AI Infra Resilience
Role Summary
Own the product roadmap for workload failure attribution on GPU clusters: determine whether a failed job was caused by hardware, system software, or application code and ensure those verdicts are surfaced to operators, schedulers, and an open-source developer surface. This role reports into NVIDIA's DSX Platform and works directly with engineering, operations, and cloud partners who deploy and extend the platform.
Experience Level
Senior. Requires 12+ years total experience and at least 5 years in product management, solutions architecture, software engineering, or site reliability engineering on technical infrastructure products.
Responsibilities
Lead product definition, delivery, and community engagement for automated GPU workload failure attribution.
- Define vision and roadmap for attribution capabilities and prioritize features with engineering.
- Specify attribution rules and APIs that distinguish hardware, system software, and application faults (e.g., NCCL timeout attribution).
- Work with cloud partners and cluster operators to translate operational triage requirements into platform features.
- Design which telemetry signals belong in public APIs versus privileged runtime extensions.
- Lead community engagement and contribution models for the open-source attribution project.
Requirements
Must-have technical experience and skills for immediate impact; candidates without a prior PM title who meet technical criteria are considered.
- 12+ years total professional experience and 5+ years in product management, solutions architecture, software engineering, or SRE on infrastructure products.
- Proven technical depth in GPU or AI infrastructure and operational triage of GPU clusters.
- Familiarity with GPU failure modes and signals: XID codes, ECC correctable/uncorrectable errors, DCGM metrics, NVLink and PCIe health signals.
- Understanding of distributed training failure patterns (NCCL timeouts, framework errors) and telemetry sources (DCGM, sysfs, dmesg).
- Experience engaging senior technical customers and translating operational requirements into product decisions.
- Strong written and verbal communication for technical and non-technical audiences.
- Nice-to-have: former SRE or infra engineer on GPU clusters; experience with GPU scheduling/topology-aware placement (Slurm, Kueue, Kubernetes); prior work on structured diagnostics, rules engines, or observability APIs; contributions to open-source infrastructure projects.
Education Requirements
Bachelor's degree in Computer Science or equivalent practical experience.
About the Company
Company: NVIDIA
Headquarters: Santa Clara, California, USA
NVIDIA is a global leader in accelerated computing, renowned for its innovative solutions in AI and digital twins that transform diverse industries. The company specializes in networking technologies, providing end-to-end InfiniBand and Ethernet solutions for servers and storage that optimize performance and scalability. NVIDIA serves sectors such as high-performance computing, enterprise data centers, and cloud computing, constantly reinventing its products and services to stay ahead in the market.
