ML Runtime Engineer

Job Title

ML Runtime Engineer

Role Summary

The ML Runtime Engineer will integrate Fractile's AI accelerators with leading inference frameworks and contribute to a scalable runtime stack focused on transformer model execution. The role sits on the ML Runtime team within the Developer Experience group and involves designing and implementing systems for high-throughput, multi-user inference.

Experience Level

Mid-level. (No explicit years-of-experience stated.)

Responsibilities

You will help design, build and optimise the runtime components that enable fast, multi-user transformer inference on Fractile hardware.

• Integrate Fractile accelerators with inference engines (examples: vLLM, SGLang).
• Research and implement KV-cache management and paged-attention strategies tuned to the hardware.
• Design and implement a scalable reference inference engine and runtime stack.
• Profile and optimise performance for high-throughput, low-latency inference workloads.
• Debug, test and maintain clean, maintainable runtime code and documentation.
• Collaborate with compiler, firmware and systems teams to ensure correct end-to-end behaviour.

Requirements

Must-have technical skills and behaviours for the role.

• Solid experience with ML inference at scale, including multi-user serving.
• Deep understanding of paged attention and transformer inference internals; familiarity with engines such as vLLM.
• Familiarity with key components of the ML software ecosystem (runtimes, memory/cache management, model execution).
• Strong software engineering skills with an emphasis on clean, maintainable systems.
• Experience profiling, debugging and optimising performance-sensitive code.
• Willingness to collaborate across hardware and software teams and document designs.
• May require eligibility checks for UK/US and other export control regulations; willingness to comply with export-control screening.

Nice-to-have:

• Experience with Rust.
• Having built an inference engine from scratch.

Education Requirements

Preferred: A degree in Computer Science or a related field. (Listed as a nice-to-have in the source.)

About the Company

Company: Fractile

Headquarters: London, UK

Fractile is on a mission to revolutionize AI by developing silicon, systems, and software that enable the fastest execution of advanced models at lower costs. Established in 2022, the company has rapidly expanded, fostering a collaborative culture in its London and Bristol offices. With a focus on innovation in ML/AI IC product development, Fractile combines hardware and software to deliver high-impact solutions for data center applications.