Software Developer / Engineer (Mid-Level)Location: Philadelphia, PA Work Mode: Hybrid (Minimum 3 days onsite) Employment Type: ContractInterview ProcessRound 1: 1-hour In-Person InterviewRound 2: 1-hour In-Person InterviewRole OverviewWe are seeking a mid-level Software Developer/Engineer with hands-on experience in deploying on-premise LLM solutions and vector databases. The ideal candidate will have strong expertise in Python, RAG pipelines, and enterprise-grade AI system implementation within secure environments.Key ResponsibilitiesDeploy and manage open-source LLMs (e.g., Llama 3, Mistral/Mixtral) in on-prem or private environmentsDevelop and optimize LLM inference workflows using PythonImplement Retrieval-Augmented Generation (RAG) pipelinesDesign and integrate vector database solutions for efficient semantic searchPerform model quantization and performance tuning for CPU-based inferenceEnsure data privacy, security, and governance compliance in enterprise environmentsImplement access controls, logging, and monitoring mechanismsDeliver reference architecture, prototypes, and technical documentationCollaborate with internal teams for knowledge transfer and system adoptionRequired Skills & QualificationsStrong experience with Python for AI/ML and backend developmentHands-on experience with open-source LLM deployment (Llama 3, Mistral, Mixtral)Experience with CPU-based inference and optimization techniquesPractical experience with vector databases (Qdrant, Chroma, Milvus, pgvector)Proven experience building RAG pipelinesKnowledge of embeddings, similarity search, and metadata filteringUnderstanding of enterprise security, data privacy, and air-gapped environmentsPreferred QualificationsExperience with LangChain or llamaIndexFamiliarity with Docker and KubernetesExposure to Rust, Go, or C++ for high-performance systemsExperience with LLM inference frameworks (vLLM, llama.