Senior Staff Engineer

A global technology client of Insight Global is seeking a Senior Staff Engineer - AI ADC to join their team. This company specializes in high-performance application delivery, network security, and AI-ready infrastructure solutions for enterprises, service providers, and government organizations. This role combines deep domain expertise in Application Delivery Controllers (ADC), Server Load Balancing (SLB), DNS/GSLB, with practical experience in AI/LLM systems, to build intelligent routing, inference‑aware load balancing, and Layer‑8 contextual decision engines for AI workloads. The ideal candidate is a senior staff engineer who can drive architecture, design, prototyping, and execution—working closely with product management, platform teams, and AI strategy groups.

Key Responsibilities Include:

• Lead architecture and development of AI Gateway components for intelligent routing of LLM/AI application traffic.
• Design and implement LLM‑aware load balancing, incorporating semantic, token, latency, and model‑level insights into traffic decisions.
• Develop enhancements for ADC and GSLB platforms to support AI workloads, including: Token‑aware rate limiting Inference latency‑based routing AI model endpoint discovery and health checks Integration with vector databases, model registries, and ML observability systems
• Contribute to advanced features including Layer‑8 context‑aware routing, adaptive traffic shaping, and AI‑driven anomaly detection.
• Drive system‑wide architecture: data plane, control plane, configuration, and distributed state management.

This role is permanent and will be worked hybrid on-site in San Jose, CA. We are looking to pay between $180,000-$195,000 base salary.