Primarily programming in Python. The company seeks language agnostic engineers who are able and interested to learn new technologies as needed.
Time spent will be roughly 50% building a simulation and evaluation platform for AI agents, and 50% working on the framework used to build the AI agents.
Designing, building, and enhancing a simulation and evaluation platform used to rigorously test, benchmark, and improve AI agents across a wide range of tasks and environments.
Developing scalable evaluation frameworks, tooling, and methodologies to measure agent performance, reliability, decision quality, and failure modes.
Build core frameworks, abstractions, and infrastructure used to develop, orchestrate, and deploy AI agents.
Collaborating closely with other engineers to rapidly iterate on agent architectures, capabilities, and evaluation strategies.
Creating systems that enable reproducible experiments, large-scale simulations, and data-driven optimization of agent behavior.
Improving developer workflows, testing capabilities, and platform reliability to accelerate AI agent development and deployment.
Analyzing evaluation results and agent behaviors to identify weaknesses, guide improvements, and drive overall system performance.
Contributing to technical architecture and engineering best practices across both the agent framework and evaluation platform stack.
Qualifications
5-10 years of engineering experience in good organizations.
Strong software engineering experience building complex backend systems, platforms, frameworks, or developer tooling.
Experience designing or building evaluation, testing, simulation, benchmarking, or experimentation systems for AI/ML products or distributed systems.
Strong programming skills in Python and/or other modern backend languages.
Experience working with LLMs, AI agents, reinforcement learning systems, or other modern AI application architectures.
Strong understanding of software architecture, scalability, reliability, and performance optimization.
Experience building internal platforms, SDKs, APIs, orchestration systems, or infrastructure that enable rapid product development.
Ability to analyze complex system behavior, debug issues, and improve reliability through data-driven approaches.
Comfortable working in fast-moving, ambiguous environments with a high degree of ownership and autonomy.