Lead Data Engineer (Databricks / AI Solutions) – Washington, D.C. (Hybrid)Randstad is seeking a highly skilled Lead Data Engineer to architect and deliver next-generation data and AI solutions for a premier client in the transportation sector. In this hybrid, hands-on leadership role, you will drive the design and deployment of enterprise-scale Databricks-based data platforms while mentoring a team of engineers.
You will act as the critical link between executive vision and technical execution—ensuring scalable, secure, and production-ready AI systems that align with enterprise governance and modernization goals.Key ResponsibilitiesLead the end-to-end architecture, design, and implementation of enterprise data and AI solutions on DatabricksDefine best practices for scalable data engineering, AI workflows, and cloud-native architecturesBuild and optimize complex data pipelines using Python, Scala, or JavaDesign and implement robust ETL/ELT frameworks and real-time/batch processing systemsOversee model deployment pipelines and MLOps practices from development through productionEnsure seamless integration of machine learning models into enterprise systemsProvide technical guidance through code reviews, pair programming, and coachingFoster a high-performance engineering culture focused on quality and innovationPartner with architects, product owners, and governance teams to align solutions with business strategy and compliance requirementsTranslate business needs into scalable technical solutionsImprove platform performance, reliability, observability, and data qualityEnsure adherence to enterprise data governance, security, and compliance standardsRequired QualificationsExperience: 6–8 years in data engineering, AI systems, or cloud data platform development with increasing leadership responsibilityTechnical Skills: Advanced proficiency in SQL and Python (or Scala/Java); strong experience with Databricks ecosystem; expertise in building scalable data pipelines and distributed systemsArchitecture: Proven experience designing enterprise-scale data integration and AI-driven solutionsMLOps: Hands-on experience with model deployment, CI/CD pipelines, and production AI systemsMethodologies: Strong understanding of Agile development practices and data governance frameworksEducation: Bachelor's degree in Computer Science, Information Systems, or related field (or equivalent experience)Preferred SkillsExperience in cloud platforms (Azure, AWS, or GCP)Knowledge of streaming architectures (Kafka, Spark Streaming)Exposure to data governance tools and frameworksExperience working in transportation, logistics, or large-scale enterprise environmentsSoft SkillsStrong analytical and problem-solving abilitiesExcellent communication skills with both technical and non-technical stakeholdersAbility to balance hands-on engineering with strategic leadershipProven ability to thrive in fast-paced, complex environments