We are seeking an experienced AWS Data Engineer to design, build, and maintain scalable data pipelines and architectures on the AWS cloud platform.
This role involves developing ETL/ELT processes, integrating data from multiple sources, ensuring data quality and security, and collaborating with stakeholders to deliver optimized data solutions for analytics and business intelligence.
Key Responsibilities
- Design and build data pipelines and ETL/ELT processes using AWS services such as Glue, EMR, and Redshift.
- Develop and maintain efficient data models for storage and analysis.
- Integrate data from various sources and ensure data quality, security, and compliance.
- Implement validation checks and security best practices across data workflows.
- Monitor and optimize data processing jobs and databases for performance.
- Collaborate with stakeholders to understand data requirements and deliver solutions.
- Maintain and operationalize existing data solutions and support modernization initiatives.
Required Qualifications
- Minimum 6+ years of experience in designing and developing enterprise-wide big data solutions.
- Strong experience with AWS services: Glue, Lambda, S3, EMR, SNS/SQS, CloudWatch, Redshift.
- Proficiency in Scala and Python for application development and automation.
- Strong SQL experience, preferably with Redshift; experience with relational databases (Oracle, MySQL, PostgreSQL).
- Hands‑on experience with ETL/ELT processes and frameworks.
- Experience with Big Data technologies such as Hadoop and Spark.
- Familiarity with shell scripting in Linux/Unix environments.
Preferred Qualifications
- Experience with Snowflake.
- Knowledge of CI/CD pipelines and automation tools.
- Exposure to machine learning models, regression, and validation techniques.