Software Engineer, LLVM Compiler - MTIA

Job Title

Software Engineer, LLVM Compiler - MTIA

Role Summary

Work on the MTIA GenAI software stack integrating compiler, runtime, and framework layers to enable efficient inference on Meta's in-house AI accelerator ASICs. The role owns a technical area (serving runtime, distributed inference, graph-mode execution/compilation, or PyTorch integration) and drives end-to-end delivery from design through production hardening.

Experience Level

Senior - requires significant domain expertise and leadership; posting specifies 8+ years of professional experience.

Responsibilities

You will set technical direction and deliver production-quality features across framework, compiler, runtime, and silicon boundaries.

• Serve as technical owner and domain expert for a GenAI inference area (serving runtime, distributed inference, graph-mode execution/compilation, or core PyTorch integration).
• Lead ambiguous multi-quarter technical programs: design, execution, CI/test strategy, rollout, and production hardening across teams.
• Design, implement, and ship features from prototype to production for inference frameworks and MTIA integration layers.
• Own performance for production inference workloads: profile across framework, runtime, compiler, and kernel boundaries and drive fixes to the correct layer.
• Build and extend distributed inference substrate: hierarchical KV caching, cross-host transfer, disaggregated prefill/decode, and parallelism strategies.
• Optimize runtime and execution path: graph capture/replay, host-side latency, memory allocation and placement, and throughput under real traffic.
• Validate accuracy vs GPU baselines, close correctness gaps, and maintain accuracy/stability test coverage and regression detection.
• Partner with kernel, compiler, and silicon teams on hardware/software co-design to inform future silicon while designs can change.
• Provide technical leadership: mentor engineers, improve design quality through reviews, and communicate decisions in design documents.

Requirements

Must-have technical skills, delivery experience, and domain expertise; preferred items listed separately.

• 8+ years professional experience in systems software, ML infrastructure, performance engineering, or compiler/framework development.
• 5+ years hands-on experience with generative AI inference or production model-serving systems.
• Proficiency in Python, C++, or Rust for low-level systems and performance-critical code.
• Experience with GenAI inference optimizations: prefill/decode, KV-cache management/compression, batching/scheduling, and latency/throughput tradeoffs.
• Hands-on experience with deep learning framework internals: operator registration/dispatch, eager execution, autograd, or graph capture/compilation paths.
• Experience with runtime-level optimization: graph-mode execution, host-side latency reduction, memory allocation and placement.
• Proven record leading technical design and end-to-end delivery of frameworks or infrastructure projects, including test strategies and CI across teams.
• Experience diagnosing cross-layer issues across framework, compiler, runtime, and hardware, including debugging numerical accuracy problems.

Nice-to-have:

• Experience with LLM serving techniques (continuous batching, chunked attention, speculative decoding, prefix caching, MoE routing).
• Experience with long-context inference and hierarchical/offloaded KV caches.
• Familiarity bringing up AI accelerator backends and performance optimization using profiling/tracing in production.
• Experience with distributed inference/training at scale (tensor/pipeline/expert parallelism, RDMA transports, multi-host serving).
• Knowledge of low-precision numerics and quantization (FP8, INT8/INT4) and related calibration/accuracy analysis.
• Familiarity with the PyTorch compilation stack (TorchDynamo, FX IR, Torch Inductor, Triton, AOT I) and custom operator integration.

Education Requirements

Bachelor's degree in Computer Science, Computer Engineering, or a relevant technical field, or equivalent practical experience.

About the Company

Company: Meta Platforms

Headquarters: Menlo Park, California, United States

American technology company that develops social networking products (Facebook, Instagram, WhatsApp) and invests in virtual/augmented reality hardware and software through Reality Labs, focusing on connectivity, advertising, and immersive computing experiences.