Skip to main content
NVIDIA logo

Principal Architect, AI Inference Networking - NIXL and Dynamo

NVIDIA
November 17, 2026
Full-time
On-site
Shanghai, SH, China
EDA Jobs, Level - Senior

Job Title

Principal Architect, AI Inference Networking - NIXL and Dynamo

Role Summary

Senior engineering architect responsible for driving adoption and upstream improvements for NIXL (Inference Xfer Library) and integrations with Dynamo in China. The role focuses on profiling and fixing throughput and TTFT bottlenecks in disaggregated inference stacks and defining dynamic data-path APIs for elastic inference deployments.

Work closely with platform teams and core NIXL/Dynamo engineers across China, Israel and the US to move prototypes to production and contribute code and roadmap input upstream.

Experience Level

Senior-level - typically 12+ years building or optimizing large-scale distributed systems and inference or high-performance networking stacks.

Responsibilities

The role combines hands-on engineering, customer-facing profiling, and technical leadership to influence product direction and adoption.

  • Drive NIXL adoption in China from architecture through prototype, benchmark, and production rollout, and contribute upstream.
  • Profile customer inference stacks and resolve performance bottlenecks affecting throughput and time-to-first-token (KV cache movement, memory registration, transfer scheduling).
  • Implement NIXL backends and plugins, extend Dynamo integrations, and contribute performance optimizations upstream (including GPUDirect class data paths).
  • Design and specify dynamic data-path APIs to support elastic scaling, worker churn, and runtime rerouting without stalling inference pipelines.
  • Represent regional requirements in NVIDIA's roadmap with technical authority to shape priorities.
  • Build local community adoption through talks, benchmarks, and reference architectures.

Requirements

Key must-have skills and relevant experience. Candidates should clearly meet most must-have items.

  • Must-have: 12+ years building or optimizing large-scale distributed systems (AI inference/training infrastructure, communication libraries, high-performance networking, or HPC runtimes).
  • Must-have: Strong systems programming in C++ and Python with shipped performance-critical code.
  • Must-have: Hands-on experience with RDMA networking (InfiniBand or RoCE) and GPU memory movement techniques.
  • Must-have: Working knowledge of modern LLM serving architectures: prefill/decode disaggregation, KV cache management, tensor/pipeline parallelism, and at least one serving stack (Dynamo, TensorRT-LLM, vLLM, SGLang, Triton).
  • Must-have: Strong communication skills and credibility with customer engineering teams and core library maintainers.
  • Must-have: Fluent Mandarin and professional English; comfortable collaborating across China, Israel, and US time zones.

Nice-to-have:

  • Open-source contributions to NIXL, Dynamo, NCCL, vLLM, SGLang, TensorRT-LLM or similar projects.
  • Experience operating inference at genuine scale (thousands of GPUs, multi-tenant clusters, strict SLOs).
  • CUDA experience and ability to read kernel/driver-level code when required.
  • Track record of building a community or widely-adopted reference projects.

Education Requirements

M.Sc. or Ph.D. in Computer Science, Electrical Engineering, or Computer Engineering, or equivalent depth of experience earned in industry.


About the Company

Company: NVIDIA

Headquarters: Santa Clara, California, USA

NVIDIA is a global leader in accelerated computing, renowned for its innovative solutions in AI and digital twins that transform diverse industries. The company specializes in networking technologies, providing end-to-end InfiniBand and Ethernet solutions for servers and storage that optimize performance and scalability. NVIDIA serves sectors such as high-performance computing, enterprise data centers, and cloud computing, constantly reinventing its products and services to stay ahead in the market.

NVIDIA logo

Date Posted: 2026-11-13