Inference Engine EngineerWe are seeking a highly technical Inference Engine Engineer to optimize the performance and efficiency of our core inference engine. In this role, you will focus on designing, implementing, and optimizing GPU kernels and supporting infrastructure for next-generation generative and agentic AI workloads. Your work will directly power the most latency-critical and compute-intensive systems deployed by our customers.We are looking for an exceptional engineer with a strong foundation in GPU programming and compiler infrastructure.
The ideal candidate enjoys pushing performance boundaries and has experience supporting production-scale machine learning applications.Key ResponsibilitiesDesign and optimize custom GPU kernels for AI (e.g., transformer and diffusion) workloadsContribute to the development of FriendliAI's kernel compiler, memory planner, runtime, and other core components.Collaborate with cloud and infrastructure engineers to ensure end-to-end inference performanceAnalyze performance bottlenecks across the software and hardware stack, and implement targeted optimizationsDrive support for new model architectures and tensor compute patternsMaintain production-grade performance infrastructure, including profiling, benchmarking, and validation toolsQualifications5+ years of experience in production or high-impact research environmentsProduction-level expertise in Python and C++Bachelor's or Master's degree in Computer Science, Computer Engineering, Electrical Engineering, or equivalentExperience developing machine learning frameworks or performance-critical runtime systemsHands-on experience writing and optimizing GPU kernelsHands-on experience profiling GPU kernelsExperience working with generative AI models such as transformer and diffusion modelsPreferred ExperienceExperience developing machine learning compilers or code generation systemsFamiliarity with dynamic shape compilation, memory planning, and kernel fusionContributions to inference engines, compilers, or high-performance numerical librariesUnderstanding of multi-GPU and distributed inference strategiesBenefitsFlexible working hoursDaily lunch and dinner provided; unlimited snacks and beveragesSupportive and highly collaborative work environmentHealth check-up support and top-tier equipment/hardware supportA front-row seat to the generative AI infrastructure revolutionCompetitive compensation, startup equity, health insurance, and other benefits.About FriendliAIFriendliAI is building the world's best AI inference platform that makes large language and multi-modal models fast, efficient, and deployable at scale. We power high-throughput, low-latency AI workloads for organizations worldwide and integrate directly with Hugging Face, giving developers instant access to over 500,000 open-source models.We are a small, fast-moving team doing work that matters at one of the most exciting moments in the history of technology. With our world-class inference engine, we are building a platform that the AI industry can actually rely on.