Runtime Engineer

Job Title

Runtime Engineer

Role Summary

Design and implement the runtime that loads, schedules, and executes compiled programs on Normal's accelerator hardware. The role owns the runtime's architecture and implementation, ensuring a stable interface between compiler, serving, driver, and firmware teams.

Experience Level

Senior - hands-on systems role. No specific years-of-experience stated.

Responsibilities

You will design, implement, and maintain the runtime layer that interfaces software above and hardware below.

• Design runtime components: device management, memory allocation and host-device transfer, executable loading, and command submission.
• Abstract differences between simulators, FPGA prototypes, and future silicon behind a stable interface.
• Manage device-resident state (e.g., resident model weights) across requests so serving can rely on persisted state.
• Minimize runtime overhead on LLM inference critical paths to meet latency and throughput goals.
• Define and maintain cross-team interfaces with compiler, driver, and firmware teams; set technical direction and review dependent designs.

Requirements

Key qualifications and desirable skills.

• Experience building runtimes, drivers, or hardware abstraction layers for accelerators or other non-standard hardware.
• Strong systems programming in C++ or Rust, including memory management, concurrency, and low-level performance work.
• Proven track record of owning a complex systems component end-to-end, from design through production, and driving cross-team technical decisions.
• Understanding of LLM inference workloads and the runtime requirements they impose.
• Comfort building software for hardware that is still evolving, from simulation through bring-up.

Nice-to-have

• Familiarity with open runtime designs (for example IREE's HAL) or runtime internals of CUDA/ROCm.
• Experience with in-memory compute or other non-von-Neumann hardware architectures.
• Experience integrating a runtime with inference servers such as vLLM.
• Contributions to open-source runtime or systems infrastructure.

Education Requirements

Not specified.

About the Company

Company: Normal Computing

Headquarters: New York, NY, USA

Normal Computing develops software and hardware solutions for the semiconductor and AI infrastructure industries, specializing in ASICs for image and video diffusion inference and AI accelerators. The company focuses on architecture and microarchitecture of compute blocks, PE array design, ISA co-design, and FPGA prototyping.