Data EngineerLocation: Washington, DC - Remote position but candidate may have to travel onsite to DC occasionally based on need.Duration: 6 monthsSkills: 8-10+ years experience requiredMySQLDatabricksPysparkDesign and develop scalable ETL/ELT pipelines using Databricks and PySpark.Build batch and real-time streaming ingestion frameworks.Develop reusable ingestion and transformation frameworks.Implement Medallion architecture (Bronze, Silver, Gold layers).Develop incremental and CDC-based ingestion pipelinesDesign and implement real-time streaming pipelines using Kafka and Structured Streaming.Optimize Spark jobs, SQL queries, and streaming pipelines.Implement Delta Lake-based ingestion and transformation frameworks.Tune partitioning, caching, and Spark execution strategies.Strong SQL and data modeling skills.Experience with cloud platforms and distributed systems.Familiarity with CI/CD pipelines and DevOps practices.