Staff Software Engineer, AI/ML Heterogeneous Systems

Job Title

Staff Software Engineer, AI/ML Heterogeneous Systems

Role Summary

Join the AI/ML Software team to build a high-performance software stack for deploying LLMs and generative AI models on RISC-V-based heterogeneous systems. The role spans compiler infrastructure, kernel development, device runtimes, and distributed runtime orchestration.

This position is based in Hsinchu, Taiwan and requires authorization to work in Taiwan.

Experience Level

Senior (Staff level). The posting requests at least 5 years of relevant industry experience.

Responsibilities

Primary responsibilities include designing runtime and compiler interfaces, integrating inference stacks, and collaborating with hardware architects to optimize AI workloads.

• Design and implement core compute programming model and device runtime daemon for heterogeneous systems.
• Define orchestration for asynchronous task execution, managed memory hierarchies, and efficient host-device synchronization.
• Lead deployment and optimization of production inference stacks (e.g., vLLM, SGLang) on RISC-V platforms.
• Optimize memory-bound operations such as PagedAttention and advanced KV-cache management.
• Enable lowering from Triton/MLIR-based compilers to hardware-specific extensions and develop high-performance kernels for RVV and matrix units.
• Collaborate with hardware architects on co-design of future AI accelerators and micro-architectures.

Requirements

Must-have technical skills and experience for immediate contribution; preferred items listed as nice-to-have.

• At least 5 years of relevant industry experience working on AI infrastructure, compilers, runtimes, or heterogeneous systems.
• Strong proficiency in C++ and Python.
• Deep understanding of heterogeneous execution models for compute offload (concepts similar to CUDA, ROCm, or OpenCL runtimes).
• Practical experience with LLM serving internals (examples: vLLM, SGLang, or other serving frameworks) and how frameworks interact with hardware backends.
• Nice-to-have: familiarity with RISC-V Vector (RVV) extensions and the RISC-V software ecosystem; deep CUDA C++ experience; active open-source contributions to vLLM, SGLang, PyTorch, or LLVM/MLIR.

Education Requirements

MS or PhD in Computer Science, Electrical Engineering, or a related technical field (explicitly listed in the posting).

About the Company

Company: SiFive

Headquarters: San Mateo, California, United States

SiFive is a pioneering company in the RISC-V ecosystem, focused on transforming the future of computing by delivering high-performance, data-intensive RISC-V solutions. Their compute platforms empower leading technology firms to innovate across various markets, including AI, machine learning, and automotive sectors. SiFive is recognized for its commitment to ongoing innovation and fostering collaboration among talented teams, impacting lives by enabling advanced chip design.