Build and evolve AIConfigurator's core optimization engine for LLM serving, including configuration search, SLA‑aware ranking, efficiency and latency estimation, and Pareto frontier analysis.
Build production‑quality Python/Rust APIs, CLIs, SDK surfaces, and web workflows that help users generate strong deployment configurations for NVIDIA GPU clusters.
Develop configuration generation systems that emit backend‑specific artifacts for Dynamo, Kubernetes, TensorRT‑LLM, vLLM, and SGLang deployments.
Collaborate with inference runtime, performance, benchmarking, and product groups to ensure simulated results correspond with actual deployment performance on H100, H200, B200, GB200, and upcoming NVIDIA platforms.
Improve model, hardware, and backend support by integrating performance databases, profiling data, support matrices, and validation tools.
Drive software quality through maintainable architecture, schema development, tests, documentation, and automation suitable for open‑source and production users.
Convert intricate inference ideas like prefill/decode disaggregation, tensor parallelism, pipeline parallelism, expert parallelism, batching, and KV cache behavior into dependable software abstractions.
Requirements
BS, MS, or PhD in Computer Science, Computer Engineering, Electrical Engineering, Applied Math, or a related field, or equivalent experience.
10+ years of relevant software engineering experience.
Strong Python/Rust engineering skills, including production APIs, CLI tools, packaging, testing, debugging, and maintainable software development.
Experience with GPU computing, distributed systems, ML infrastructure, or high‑performance model serving.
Understanding of LLM inference concepts such as batching, latency, efficiency, memory constraints, parallelism strategies, and serving SLAs.
Experience working with data‑driven performance analysis, benchmarking, simulation, optimization, or managing resource needs.
Ability to collaborate across research, runtime, platform, and customer‑facing engineering teams.
Strong written and verbal communication skills, with the ability to explain sophisticated technical tradeoffs clearly.