Intern, AI Application Engineer
Samsung SemiconductorJob Title
Intern, AI Application Engineer
Role Summary
This is a hands-on internship on the SoC Architect team focused on evaluating network-attached direct storage (NADS) for LLM inference workloads. You will deploy and tune LLM inference serving, run automated benchmarks across GPU, network and storage, and produce measured storage requirements that inform future SSD/SoC architecture.
Experience Level
Entry-level internship. Intended for current students; no prior industry experience required beyond the technical skills listed below. Must have at least one academic quarter/semester remaining.
Responsibilities
Work with a mentor and the storage architecture team to deploy, benchmark and analyze end-to-end LLM serving with KV cache offload to network-attached storage. Deliver reproducible experiments and present results to the team.
- Deploy and operate an LLM inference service (e.g., vLLM serving Llama-3.3-70B) on a 4Γ NVIDIA H200 GPU server.
- Build an automated benchmark harness that replays long-context, multi-turn and agent-style traces and records latency and throughput (TTFT, TPOT, goodput).
- Configure KV cache offloading (LMCache, NVIDIA Dynamo/NIXL) to NVMe-oF/RDMA-backed network-attached storage, first on BlueField-3 and then on a network-attached SSD prototype.
- Run daily experiment sweeps across request rates, context lengths and cache sizes; track configurations and results in version control.
- Analyze experiment data with Python to identify bottlenecks across GPU, network and SSD; share findings in weekly meetings.
- Collaborate with FPGA, SoC and firmware engineers to validate end-to-end data paths and KV cache correctness.
- Summarize measured storage requirements (bandwidth, IOPS, latency) and present recommendations at the internship closeout.
Requirements
Core technical skills and behaviors required or strongly preferred for success in the role.
- Must-have: Hands-on experience running LLM inference with Python and PyTorch on NVIDIA GPUs under Linux using a serving framework such as vLLM, SGLang, or Hugging Face TGI.
- Must-have: Understanding of LLM serving internals: KV cache, prefix caching, batching and parallelism (tensor/pipeline), and latency metrics (TTFT, TPOT, throughput).
- Must-have: Experience benchmarking and profiling system performance and analyzing results with scripts (e.g., Python, pandas, plotting).
- Preferred: Familiarity with KV cache offloading technologies (LMCache, NVIDIA Dynamo/NIXL), RDMA/RoCE, NVMe/NVMe-oF, and container orchestration (Docker/Kubernetes).
- Behavioral: Collaborative communicator, curious and data-driven, inclusive and adaptable to diverse teams.
Education Requirements
Pursuing a Bachelor's, Master's or PhD in Computer Science, Computer Engineering, Electrical Engineering, or a related field; Master's or PhD preferred. Must have at least one academic quarter/semester remaining during the internship.
About the Company
Company: Samsung Semiconductor
Headquarters: San Diego, California, United States
Samsung Semiconductor is a global leader in semiconductor technology, providing innovative solutions for diverse applications including smartphones, electric vehicles, and data centers. As part of Samsung's Advanced Institute of Technology, the company focuses on cutting-edge research in areas such as CPU microarchitecture, AI, and HPC, fostering a diverse and inclusive work environment for its employees to thrive and innovate.
![]()