Data EngineerThe Data Engineer works within the ETL and Operations team to build high quality data pipelines driving analytic solutions from diverse, disparate sources of data.This role requires a comprehensive understanding of data architecture, data engineering, and data analysis. The ideal candidate is a skilled data engineer with experience creating data products supporting analytic solutions. They are able to identify and implement solutions in a highly technical environment and work as part of a technical, cross-functional team.
Strong problem-solving and troubleshooting skills are a must.This role focuses on designing, developing, and maintaining datasets within AWS. Day-to-day responsibilities include:Coordinating, building, and managing new data ingestsArchitecting updates, fixes, and optimizations across suite of production jobsProviding feedback on and enacting changes for improvements across the team including both technical and process updatesLearn and assess new technologies for implementation through proof-of-concept projects and testingMentoring junior developersPartnering with data analysts and data scientists to design, build, and deploy aggregation processesTop Skills:Hadoop/Hive experienceVery Strong SQLAWS experience (EMR, Lambda, Glue, Step Functions)Experience working with large data sets (billions of records per day)Write and review codePython scripting (PySpark, control scripts)Architecting/designing data pipelinesQuery and data pipeline optimizationStrong communication and collaboration skillsAbility to help others and mentor junior resourcesNice to have:SparkScalaGitShell scriptingLinux