- Design and develop ML evaluation systems for agentic and LLM-based architectures
- Translate research ideas into production-grade ML systems with measurable impact
- Partner with Research, Product, and Platform teams to productize experiments into robust AI solutions
- Stay current with advancements in ML, NLP, and LLM systems and contribute to technical discussions
- Mentor and support other engineers through design reviews, feedback, and knowledge sharing
- Build and evaluate the core intelligence behind agentic AI systems
- Design and own evaluation frameworks ensuring quality, safety, and performance across complex agentic systems
Requirements
- Deep experience in NLP and modern ML systems, including hands-on experience with LLM-based or agentic systems
- Strong architectural skills and proven experience designing complex software systems
- Production experience with Python, AWS, Kubernetes, and/or Docker
- Experience implementing technical solutions and tackling challenging engineering problems
- Bachelor’s Degree in CS or another related field
- Experience building and evaluating agentic systems at scale
- Production experience with LLM-centric services, including inference, orchestration, evaluation, and monitoring
- Familiarity with large-scale ML experimentation, benchmarking, or simulation frameworks
- Knowledge of techniques for optimizing model architectures for faster inference
- Experience with AWS, CI/CD, Kafka, and Athena
- Proficiency in technical mentorship of junior and mid-level engineers
- Willingness to learn, collaborate across teams, and teach others
Core Competencies
Demonstrates expertise in designing and developing ML evaluation systems, particularly for agentic and LLM-based architectures, while effectively collaborating with cross-functional teams to translate research into impactful AI solutions. Proficient in mentoring engineers and ensuring quality and performance in complex systems.
Highest-signal resume keywords
- Deep Experience In NLP
- Production Experience With Python
- Architectural Skills In Complex Software Systems
- Experience With LLM-Centric Services
- Familiarity With Large-Scale ML Experimentation
ATS Optimization Keywords
Hard Skills
- Machine Learning
- Natural Language Processing
- LLM-Based Systems
- Software Architecture
- Model Optimization Techniques
- Technical Solution Implementation
- Evaluation Framework Design
- Benchmarking
- Simulation Frameworks
- Agentic Systems Evaluation
Soft Skills
- Mentorship
- Collaboration
- Knowledge Sharing
- Feedback Provision
- Willingness To Learn
Certifications & Qualifications
- Bachelor’s Degree In Computer Science
Industry Keywords
- AI Solutions
- Production-Grade Systems
- Technical Discussions
- Quality Assurance
- Performance Evaluation
Tools & Technologies
- AWS
- Kubernetes
- Docker
- CI/CD
- Kafka
- Athena