Distributed LLM Inference Engineer for Scalable AI

Anyscale is hiring a Distributed LLM Inference Engineer to push the performance frontier for inference at scale. You will help navigate batch and online inference at high throughput and low latency, enabling open-source Ray users and customers to scale ML workloads.

You will work across the stack, integrating Ray Data and LLM engines, collaborating with the community on techniques like vLLM, TensorRT-LLM, and related DL frameworks, while adopting best practices from research and industry.