Senior Inference-Serving Engineer for Throughput LLMs

Groq, Inc. in New York is seeking a seasoned engineer to accelerate LLM inference in production. You will work on an inference serving stack from trained model to high-throughput token delivery, with focus on performance, reliability, and scale.

You will optimize batching, memory management, quantization, and cross-device distribution across accelerators, collaborating with infrastructure and hardware teams to push models into production.