AI Framework Engineer

Job Title

AI Framework Engineer

Role Summary

Develop and optimize deep learning and large language model (LLM) frameworks and GPU kernels to improve training and inference performance on AMD GPUs, including multi-GPU and multi-node systems.

Work with internal GPU library teams and open-source maintainers to integrate optimizations into TensorFlow, PyTorch and related toolchains.

Experience Level

Mid-level. The posting expects an experienced engineer with substantial hands-on expertise in LLM frameworks, performance engineering, and production-quality software delivery; no explicit years-of-experience were specified.

Responsibilities

Primary responsibilities include framework and kernel optimization, end-to-end performance work, and collaboration across teams and open-source projects.

• Optimize major DL/LLM frameworks (TensorFlow, PyTorch, vLLM, SGLang) for AMD GPUs and contribute improvements upstream.
• Develop and tune GPU kernels and performance-critical operators to maximize throughput and reduce latency.
• Adapt and optimize LLM model architectures (e.g., Llama, Qwen, DeepSeek) and apply techniques such as FlashAttention, PagedAttention, and quantization.
• Profile systems to identify compute, memory, and communication bottlenecks across multi-GPU and multi-node setups and implement fixes.
• Leverage compiler and graph-compiler technologies to accelerate the end-to-end training and inference pipeline.
• Prototype and integrate advanced inference methods (e.g., speculative decoding, weight-only quantization) into production systems.
• Collaborate with internal teams and open-source maintainers to ensure seamless upstream integration of optimizations.
• Apply software engineering best practices to deliver maintainable, reliable performance-critical code.

Requirements

Must-have technical skills and practical experience required for the role; nice-to-have items are listed separately.

• Must-have: Practical experience with vLLM or SGLang and production-level knowledge of modern LLMs (e.g., DeepSeek, Qwen).
• Must-have: Strong theoretical and practical understanding of Transformer architectures and attention mechanisms (KV cache, MoE) and applied inference optimizations such as FlashAttention, PagedAttention, continuous batching, and quantization (INT8/INT4/GPTQ/AWQ).
• Must-have: Proven ability to profile, diagnose, and optimize compute, memory, and communication bottlenecks across multi-GPU and multi-node environments.
• Must-have: Experience integrating optimized GPU kernels into TensorFlow/PyTorch to accelerate training and inference at scale.
• Must-have: Strong software engineering skills in Python and C++, with effective debugging and testing practices and a track record of delivering maintainable, performance-critical software.
• Nice-to-have: Hands-on GPU kernel development and tuning for AMD GPUs using HIP, CUDA, assembly, or tools like CUTLASS, CK, or Triton; knowledge of GCN/RDNA architectures.
• Nice-to-have: Familiarity with compiler and system-level optimization technologies such as LLVM, ROCm, TVM, or MLIR.
• Nice-to-have: Experience with distributed inference approaches (tensor/ pipeline parallelism) and operating large-scale workloads on heterogeneous clusters.
• Nice-to-have: History of open-source contributions relevant to deep learning frameworks or GPU libraries.

Education Requirements

Bachelor's and/or Master’s degree in Computer Science, Computer Engineering, Electrical Engineering, or a related field (as stated in the posting).

About the Company

Company: Advanced Micro Devices

Headquarters: Sunnyvale, California, USA

Advanced Micro Devices, or AMD, is a global semiconductor company that designs and manufactures microprocessors, graphics processors, and related technologies for a variety of computing devices. Known for pushing the boundaries of innovation, AMD's mission is to deliver high-performance computing solutions for AI, data centers, gaming, and embedded applications. They foster a collaborative, inclusive culture focused on creativity and problem-solving, aiming to drive progress and excellence in technology.