MLOps / AI Infrastructure Engineer

Job Title: - MLOps / AI Infrastructure Engineer Location: - Remote (Periodic travel to client sites primarily government/on-prem environments) Duration: - Full-Time / Permanent Work Hours: - Standard business hours (flexible, async-first environment) Job Overview & Description: We are seeking a highly skilled MLOps / AI Infrastructure Engineer to design, deploy, and operate mission-critical AI infrastructure in on-premises and edge environments. This role focuses on building and maintaining GPU-powered compute clusters, Kubernetes-based orchestration systems, and scalable MLOps pipelines for AI training and inference workloads. The ideal candidate will have deep expertise in GPU infrastructure (NVIDIA H200/A100), Kubernetes on bare-metal, high-performance networking, and software-defined storage, along with hands-on experience in deploying secure, compliant systems in air-gapped and government-regulated environments. You will collaborate closely with AI/ML engineers, architects, and client technical teams to ensure high availability, performance, and compliance of AI platforms operating outside traditional cloud environments. Roles & Responsibilities: GPU Compute & Infrastructure • Deploy, configure, and maintain GPU servers (NVIDIA H200, A100) • Manage CUDA, drivers, firmware, NVLink/NVSwitch topology • Implement NVIDIA tools (DCGM, MIG, NVIDIA Container Toolkit) • Monitor hardware health, utilization, and performance • Automate bare-metal provisioning (PXE/iPXE, MAAS, Foreman) Kubernetes & Container Orchestration • Build and manage Kubernetes clusters (kubeadm / Rancher RKE2) • Configure GPU node pools using NVIDIA GPU Operator • Implement CNI solutions (Calico, Cilium, SR-IOV) • Manage ingress, load balancing (MetalLB), and service mesh (Istio/Linkerd) • Enforce cluster security (RBAC, network policies, secrets management) MLOps & AI Workloads • Deploy ML platforms (MLflow, Kubeflow) • Manage model serving using NVIDIA Triton Inference Server • Build CI/CD pipelines (ArgoCD, Flux GitOps approach) • Optimize GPU utilization for training and inference • Manage model storage (Ceph, MinIO) Networking & Storage • Design high-bandwidth networking (InfiniBand, RoCE v2, Ethernet) • Configure RDMA for distributed AI workloads • Deploy software-defined storage (Ceph, Rook, MinIO) • Implement VLANs, firewall policies, and secure connectivity (VPN) Security & Compliance • Implement controls aligned with NIST SP 800-171 / CMMC • Maintain OS hardening (RHEL/Rocky Linux, Ubuntu CIS benchmarks) • Automate compliance checks (OpenSCAP) • Document infrastructure (SSP, diagrams, DR plans) • Support audits and penetration testing remediation Required Qualifications: • 6+ years of infrastructure engineering experience • 3+ years managing GPU clusters or HPC environments • Strong expertise in NVIDIA GPU stack (CUDA, DCGM, MIG, NVLink) • Hands-on Kubernetes (bare-metal deployment & operations) • Strong networking knowledge (BGP, VLANs, RDMA, load balancing) • Experience with storage solutions (Ceph, MinIO, Rook) • MLOps experience (MLflow, Kubeflow, Triton, GitOps pipelines) • Knowledge of NIST SP 800-171 compliance • Experience with Terraform or Ansible • Strong Linux administration skills • Excellent documentation and communication skills Certifications (Preferred / Required): • Certified Kubernetes Administrator (CKA) / Certified Kubernetes Security Specialist (CKS) • NVIDIA Certifications (GPU / AI Infrastructure) • RHCSA / RHCE (Red Hat Certified System Administrator/Engineer) • Security or compliance certifications (preferred)