Data EngineerLocation: McLean VA (Onsite Hybrid Model)Duration: 9-12 Months || Extension PossibleJob ResponsibilitiesMust haves include Pyspark, Hadoop, Snowflake.Must have programming experience in Python, Spark for data processing and analysis.Must have strong SQL experience.Experience with AWS Cloud is preferred.Cleanse, manipulate and analyze large datasets (structured and unstructured data – XMLs, JSONs, PDFs) using Hadoop platform.Experience in processing large volumes and variety of data (structured and unstructured data, writing code for parallel processing, XMLs, JSONs, PDFs)Experience – using Hadoop platform and performing analysis.Develop Python, PySpark, Spark scripts to filter/cleanse/map/aggregate data.Manage and implement data processes (Data Quality reports)Develop data profiling, deduping logic, matching logic for analysisProgramming languages experience in Python, PySpark and Spark for data ingestionProgramming experience in Big Data platform using Hadoop platformPresent ideas and recommendations on Hadoop and other technologies best use to managementFamiliarity with Hadoop cluster environment and configurations for resource management for analysis workExperience with containerization and orchestration.Good to have hands on experience with AWS, Kubernetes, Kubeflow, Docker etc.