AWS Data Engineer

Long term contract position with a financial firm.

Hybrid 3 days/week in Newark, NJ.

1. Design and development of batch and real-time data pipelines using AWS Data & Analytics Services such as AWS Glue, Athena, Lambda, Redshift etc.,

1. Experience in building complex data processing and ETL jobs using Pyspark and SQL

1. Experience in ingesting and processing structured and semi-structured data

1. Exposure to data warehousing, data lake and lake house architectures

1. Good understanding of different data modeling techniques

1. Ability to optimize cost and improve performance of data pipelines

1. Exposure to CI/CD process

1. Knowledge of best practices in securing data pipelines and data assets in AWS.

1. Knowledge of data quality and data governance processes is a plus.