Software Engineer, GenAI Frameworks - MTIA

Job Title

Software Engineer, GenAI Frameworks - MTIA

Role Summary

Join the MTIA GenAI Frameworks team to build and optimize the software stack that runs generative AI models on Meta's in-house MTIA accelerators. The role owns a focused area such as serving runtime, distributed inference, graph-mode execution/compilation, or PyTorch integration.

You will set technical direction, lead cross-team programs from prototype to production, and be accountable for accuracy, stability, performance, and test coverage on custom silicon.

Experience Level

Level: Senior. Guidance: 8+ years professional experience in systems software, ML infrastructure, performance engineering, or compiler/framework development.

Responsibilities

The role drives design, implementation, validation, and production hardening for GenAI inference on MTIA.

• Act as technical owner and domain expert for a key GenAI inference area (serving runtime, distributed inference, graph-mode execution/compilation, or PyTorch integration).
• Lead ambiguous, multi-quarter technical programs end to end: design, execution, CI/test strategy, rollout, and production hardening across teams.
• Design, implement, and ship features in inference frameworks and MTIA integration layers from prototype to production.
• Profile and optimize performance across framework, runtime, compiler, and kernel layers; prioritize fixes at the correct layer.
• Build and extend distributed inference substrate: KV caching, cross-host transfer, disaggregated prefetch/decode, and long-context parallelism strategies.
• Optimize runtime execution: graph capture/replay, host-side latency reduction, memory allocation/placement, and throughput under real traffic.
• Validate model accuracy against GPU baselines and close correctness gaps while distinguishing regressions from stale references.
• Define and maintain quality metrics: accuracy, stability, latency, throughput, and regression detection via tests and CI.
• Collaborate with kernel, compiler, and silicon teams on hardware/software co-design and provide framework-level evidence influencing silicon decisions.
• Provide technical leadership and mentorship; communicate designs via documents and reviews.

Requirements

Must-have technical skills and experience.

• 8+ years professional experience in systems software, ML infrastructure, performance engineering, or compiler/framework development.
• 5+ years hands-on experience with generative AI inference or training frameworks, or production model-serving systems.
• Proficiency in Python, C++, or Rust, including low-level systems and performance-critical code paths.
• Experience with GenAI inference optimization (prefill/decode, KV-cache management/compression, batching, scheduling, latency vs throughput tradeoffs).
• Hands-on experience with deep learning framework internals (operator registration/dispatch, eager execution, autograd, graph capture/compilation paths).
• Experience with runtime-level optimization (graph-mode execution, host-side latency reduction, memory allocation and placement).
• Demonstrated experience leading technical design and delivering frameworks or infrastructure projects, including test strategies and CI across teams.
• Experience diagnosing issues across multiple system layers (framework, compiler, runtime, hardware) and debugging numerical accuracy and stability problems.

Nice-to-have.

• Experience with long-context inference techniques and hierarchical or offloaded KV cache strategies.
• Experience bringing up AI accelerator backends (GPU, TPU, or custom ASIC) in major ML frameworks.
• Familiarity with low-precision numerics and quantization (FP8, INT8/INT4) and calibration/accuracy analysis.
• Experience with distributed inference/training at scale (tensor/pipeline/expert parallelism, RDMA transports, multi-host serving).
• Familiarity with the PyTorch compilation stack (TorchDynamo, FX IR, Torch Inductor, Triton, AOT I) and custom operator integration.
• Track record of production performance optimization using profiling and tracing tools; experience with LLM serving strategies (continuous batching, speculative decoding, disaggregated prefill/decode, MoE routing).
• Experience implementing responsible AI practices (risk assessment, bias mitigation, quality/accuracy reviews).

Education Requirements

Posting specifies a Bachelor's degree in Computer Science, Computer Engineering, or a relevant technical field, or equivalent practical experience. (Equivalent practical experience is explicitly accepted.)

About the Company

Company: Meta Platforms

Headquarters: Menlo Park, California, United States

American technology company that develops social networking products (Facebook, Instagram, WhatsApp) and invests in virtual/augmented reality hardware and software through Reality Labs, focusing on connectivity, advertising, and immersive computing experiences.