Summary : Lead the design and delivery of scalable data engineering solutions on AWS, supporting enterprise data onboarding and domain-oriented data products. The role will provide hands-on technical leadership across AWS-native data services and work with Prudential teams to build trusted, reusable and analytics/AI-ready data assets.
The full job description covers all associated skills, previous experience, and any qualifications that applicants are expected to have.
Key Responsibilities
Technical Leadership & Architecture
- Lead the design and implementation of data ingestion, transformation and processing solutions using AWS native services including S3, Glue, EMR, Redshift and Athena.
- Define appropriate engineering patterns for batch and large-scale data processing using Python and PySpark.
- Guide implementation of modern lake/lakehouse patterns, including S3 and Apache Iceberg, where applicable.
- Provide technical direction across design, development, testing, deployment and production support.
Data Engineering & Data Products
- Lead onboarding of data from legacy and enterprise source systems into AWS.
- Drive development of reusable pipelines and engineering components for specific enterprise data domains.
- Support the ADS / Authoritative Data Source approach, where domain data is packaged as trusted data products for enterprise consumption.
- Ensure datasets produced by the engineering team are consumable for downstream analytics and AI use cases.
Engineering Quality
- Conduct design and code reviews for Python/PySpark and AWS data-engineering components.
- Establish development standards, reusable patterns and engineering best practices.
- Ensure appropriate data validation and reconciliation are incorporated into pipelines.
- Troubleshoot performance and scalability issues across data pipelines and processing workloads.
Team Leadership & Collaboration
- Provide day-to-day technical guidance to Data Engineers.
- Break functional/data requirements into engineering tasks and coordinate delivery.
- Work closely with Prudential's architecture, data, technology and business-aligned teams.
- Communicate technical risks, dependencies and design decisions to stakeholders. xhqgsiq
- Mentor engineers and build the team's AWS/Python/PySpark engineering capability.
AWS Lead Data Engineer in newark at Unknown Company
This position is listed as full time and onsite.