AI Runtime Engineer

Job Title

AI Runtime Engineer

Role Summary

Develop and optimize low-latency, high-performance runtime software to execute deep learning models on EnCharge AI accelerators. Work with hardware, compiler, and AI framework teams to deliver efficient inference and training performance across cloud and edge environments.

Experience Level

Mid-level — typically requires 3+ years of relevant experience in low-level runtime or systems software for AI accelerators, GPUs, or HPC systems.

Responsibilities

Primary responsibilities include building and tuning the runtime execution stack and integrating it with compilers and frameworks.

• Design, develop, and optimize AI runtime software for accelerator hardware.
• Implement task scheduling, concurrency, memory management, and kernel execution strategies.
• Optimize data movement between host and device (PCIe, DMA, shared memory).
• Design and implement high-performance APIs for AI inference frameworks (e.g., ONNX Runtime, OpenVINO, vLLM).
• Apply graph execution optimizations: kernel fusion, pipelining, tensor tiling, and caching.
• Integrate runtime components with AI compilers and toolchains (LLVM, MLIR, XLA, TVM).
• Ensure scalability, reliability, and low-latency/high-throughput behavior for cloud and edge deployments.

Requirements

Must-have technical skills and experience.

• 3+ years building low-level runtime software for AI accelerators, GPUs, or HPC systems.
• Strong proficiency in C/C++ and low-level systems programming.
• Deep understanding of task scheduling, concurrency, and memory hierarchies.
• Experience with hardware-aware optimizations and dataflow architectures.
• Familiarity with deep learning execution frameworks (ONNX Runtime, TensorRT, TVM, OpenVINO).
• Experience with low-latency, high-throughput workload execution and performance tuning.
• Strong debugging and profiling skills for optimizing execution performance.
• Exposure to AI model deployment pipelines (e.g., Triton, TensorFlow Serving).
• Ability to collaborate across hardware, compiler, and framework teams.

Education Requirements

Bachelor's or Master’s degree in Computer Science, Electrical Engineering, or a related technical field (as listed in the original posting).

About the Company

Company: EnCharge AI

EnCharge AI develops advanced AI hardware and software systems for edge-to-cloud computing, focusing on in-memory computing technology to deliver high compute efficiency and density with low power consumption. Founded in 2022, the company targets power-, energy-, and space-constrained applications with integrated hardware architectures and software stacks for scalable, reliable AI deployment.