Working Student - LLM Inference Performance

Job Title

Working Student - LLM Inference Performance

Role Summary

As a working student on the LLM Inference Performance team, you will implement and optimize low-level kernels, benchmark LLM workloads on real silicon, and build profiling and benchmarking tools to improve inference performance. You will work closely with compiler, runtime, and kernel engineers to translate performance insights into practical improvements.

This role provides hands-on experience with the company's Rust and Python stacks, use of AI coding agents in development workflows, and direct exposure to real hardware measurements and performance tooling.

Experience Level

Entry-level (working student / internship). Intended for current Master's students or early-career candidates; practical experience through coursework or projects is expected.

Responsibilities

Primary responsibilities include:

• Implement, test, and tune low-level kernels for LLM building blocks (attention, MLP, Mixture-of-Experts routing).
• Benchmark performance on simulators and real silicon; build benchmark harnesses.
• Develop profiling scripts, trace visualizations, and analytical/roofline performance models.
• Compare measured performance against theoretical estimates and investigate bottlenecks.
• Collaborate with compiler, runtime, and kernel engineers to drive optimizations informed by measurements.
• Build tooling and workflows that enable AI coding agents to assist with development and benchmarking.

Requirements

Must-have qualifications:

• Strong programming skills in Python plus at least one of C, C++, or Rust.
• Understanding of computer architecture, memory systems, bandwidth, and parallel computing from coursework or projects.
• Interest in low-level programming, hardware performance, and optimization.
• Practical experience using AI coding agents in development workflows and the ability to evaluate/improve their output.
• Analytical mindset: form hypotheses, run measurements, and communicate technical findings clearly.
• Curiosity, initiative, and enthusiasm for learning new technologies.

Nice-to-have:

• GPU or AI accelerator programming experience.
• Embedded systems or bare-metal programming experience.
• Experience with performance profiling, benchmarking, or optimization tools.
• Familiarity with LLM inference fundamentals and architectures.

Education Requirements

Currently enrolled in a Master's degree program (MSc) in Computer Science, Electrical Engineering, Mathematics, or a related technical field. Related technical fields or equivalent practical experience may be considered.

About the Company

Company: Tensordyne

Tensordyne is a startup developing high-performance, low-power AI inference processors and multi-chip systems for generative AI acceleration in data centers. The company focuses on ASIC design, verification, and integration of computational accelerators with third-party SoC IP.