AI Observability Engineer - GenAI & Agentic AI
We’re looking for an experienced AI Observability Engineer to support the development and production of next-generation GenAI and Agentic AI systems.
This is a great opportunity for someone with a background in Observability, SRE, MLOps, Platform Engineering, AI/ML or Test Automation who wants to work on cutting-edge AI technology.
Key Responsibilities:
- Build telemetry, monitoring, dashboards, alerts, tracing and SLOs for AI/LLM applications and agents
- Develop LLM and AI evaluation frameworks covering response quality, prompts, RAG, latency, reliability and safety
- Analyse production behaviour and identify improvements to AI quality, performance and reliability
- Automate production readiness, reporting and governance processes
- Create runbooks and operational documentation for AI releases
- Work closely with Engineering and Product teams to define monitoring and evaluation requirements
Experience:
- 4+ years' engineering experience across observability, monitoring, platform engineering, SRE, AI/ML or test automation
- Strong hands-on Python experience
- Experience with metrics, logs, traces, dashboards, alerting and SLOs
- Understanding of LLMs, AI agents, RAG or AI evaluation
- Experience supporting production systems
If you're an AI Observability Engineer, GenAI Engineer, SRE, MLOps Engineer, Platform Engineer or AI/ML Engineer looking to work on production Agentic AI systems, we'd love to hear from you.
AI Observability Engineer - GenAI & Agentic AI in Charlotte at GL Global
This position is listed as full time and onsite. It was posted 2 days ago.