Principal Architect, AI Inference Networking - NIXL and Dynamo
NVIDIAJob Title
Principal Architect, AI Inference Networking - NIXL and Dynamo
Role Summary
Senior engineering architect responsible for driving adoption and upstream improvements for NIXL (Inference Xfer Library) and integrations with Dynamo in China. The role focuses on profiling and fixing throughput and TTFT bottlenecks in disaggregated inference stacks and defining dynamic data-path APIs for elastic inference deployments.
Work closely with platform teams and core NIXL/Dynamo engineers across China, Israel and the US to move prototypes to production and contribute code and roadmap input upstream.
Experience Level
Senior-level - typically 12+ years building or optimizing large-scale distributed systems and inference or high-performance networking stacks.
Responsibilities
The role combines hands-on engineering, customer-facing profiling, and technical leadership to influence product direction and adoption.
- Drive NIXL adoption in China from architecture through prototype, benchmark, and production rollout, and contribute upstream.
- Profile customer inference stacks and resolve performance bottlenecks affecting throughput and time-to-first-token (KV cache movement, memory registration, transfer scheduling).
- Implement NIXL backends and plugins, extend Dynamo integrations, and contribute performance optimizations upstream (including GPUDirect class data paths).
- Design and specify dynamic data-path APIs to support elastic scaling, worker churn, and runtime rerouting without stalling inference pipelines.
- Represent regional requirements in NVIDIA's roadmap with technical authority to shape priorities.
- Build local community adoption through talks, benchmarks, and reference architectures.
Requirements
Key must-have skills and relevant experience. Candidates should clearly meet most must-have items.
- Must-have: 12+ years building or optimizing large-scale distributed systems (AI inference/training infrastructure, communication libraries, high-performance networking, or HPC runtimes).
- Must-have: Strong systems programming in C++ and Python with shipped performance-critical code.
- Must-have: Hands-on experience with RDMA networking (InfiniBand or RoCE) and GPU memory movement techniques.
- Must-have: Working knowledge of modern LLM serving architectures: prefill/decode disaggregation, KV cache management, tensor/pipeline parallelism, and at least one serving stack (Dynamo, TensorRT-LLM, vLLM, SGLang, Triton).
- Must-have: Strong communication skills and credibility with customer engineering teams and core library maintainers.
- Must-have: Fluent Mandarin and professional English; comfortable collaborating across China, Israel, and US time zones.
Nice-to-have:
- Open-source contributions to NIXL, Dynamo, NCCL, vLLM, SGLang, TensorRT-LLM or similar projects.
- Experience operating inference at genuine scale (thousands of GPUs, multi-tenant clusters, strict SLOs).
- CUDA experience and ability to read kernel/driver-level code when required.
- Track record of building a community or widely-adopted reference projects.
Education Requirements
M.Sc. or Ph.D. in Computer Science, Electrical Engineering, or Computer Engineering, or equivalent depth of experience earned in industry.
About the Company
Company: NVIDIA
Headquarters: Santa Clara, California, USA
NVIDIA is a global leader in accelerated computing, renowned for its innovative solutions in AI and digital twins that transform diverse industries. The company specializes in networking technologies, providing end-to-end InfiniBand and Ethernet solutions for servers and storage that optimize performance and scalability. NVIDIA serves sectors such as high-performance computing, enterprise data centers, and cloud computing, constantly reinventing its products and services to stay ahead in the market.
