Skip to main content
T

Senior Software Engineer, Inference Performance

Tensordyne
November 17, 2026
Full-time
On-site
Munich, BY, Germany
EDA Jobs, Level - Senior

Job Title

Senior Software Engineer, Inference Performance

Role Summary

Work on the software stack that takes AI models from code to hardware: design abstractions, kernels, libraries, and tooling to maximize inference performance on Tensordyne silicon. You will collaborate with teams across models, compilers, runtime, and hardware to profile, fix, and verify performance regressions on real silicon.

Experience Level

Senior. The posting specifies a senior role; no explicit years-of-experience requirement is stated.

Responsibilities

Deliver production-quality performance software and tooling that lets engineers and agents optimize models on custom hardware.

  • Design APIs, kernel abstractions, and libraries in the Python SDK or lower-level stacks (Rust/C++/C) for performance-critical layers.
  • Profile models (LLMs and multimodal) on real silicon, identify bottlenecks across compiler, runtime, kernels, and parallelism, and implement fixes at the appropriate layer.
  • Compare measured results to analytical models and simulators; explain gaps and convert explanations into fixes or improved models.
  • Build and maintain profilers, traces, counters, and benchmark harnesses to support rapid, agent-driven optimization loops.
  • Define measurement standards, regression tracking, and reporting for performance.
  • Design abstractions that scale to future hardware generations and collaborate with hardware architects to evaluate architectural choices.

Requirements

Core must-have skills and capabilities; followed by desirable (nice-to-have) experience.

  • Must-have: Deep understanding of hardware performance (compute utilization, memory hierarchy, latency hiding, synchronization, multi-chip scaling) and proven experience optimizing AI inference workloads on GPUs or accelerators.
  • Must-have: Strong software engineering background with ownership of substantial systems from design to production; proficiency in Python and at least one systems language (Rust, C++, or C).
  • Must-have: End-to-end LLM inference knowledge (prefill, decode, attention variants, KV caches, MoE, parallelism) and how model behavior maps to compute, memory, and communication.
  • Must-have: Experience with agentic development or automation-driven optimization loops (build profilers/harnesses that enable iterative, measurable improvements).
  • Must-have: Track record of designing APIs, libraries, or performance tools adopted by other engineers; ability to drive problems to resolution across software and hardware boundaries.
  • Nice-to-have: Embedded, bare-metal, or low-level runtime programming experience.
  • Nice-to-have: Experience with compiler frameworks or kernel DSLs (MLIR, Triton, Pallas, NKI, CuTe) or contributions to LLM serving stacks (vLLM, SGLang).
  • Nice-to-have: Demonstrated success taking end-to-end inference workloads to competitive performance on a hardware target.

Education Requirements

Not specified.


About the Company

Company: Tensordyne

Tensordyne is a startup developing high-performance, low-power AI inference processors and multi-chip systems for generative AI acceleration in data centers. The company focuses on ASIC design, verification, and integration of computational accelerators with third-party SoC IP.

Tensordyne logo

Date Posted: 2026-11-12