Position: Data Engineer
Location:Manhattan NY (Hybrid 3 days a week to office)
Long Term
Key Skills
SQL | Python | Spark | Airflow / ADF | Cloud data services | CI/CD
Required Qualifications
- 8-10+ years in data engineering
- Strong SQL and Python (or Scala) skills
- Experience with orchestration tools such as Airflow or Azure Data Factory
- Hands-on experience with cloud data services and big data frameworks such as Spark
- Understanding of data warehousing and lakehouse concepts
- Familiarity with Git, CI/CD and infrastructure-as-code
- Experience supporting ML/AI data needs is a plus
Key Responsibilities
- Design, build and optimize batch and streaming data pipelines
- Implement data models and transformations defined by the Data Architect
- Ingest and integrate data from multiple source systems and APIs
- Implement data quality checks, monitoring and alerting
- Optimize performance and cost of data workloads
- Collaborate with AI Developers and analysts to provision datasets
- Maintain documentation, version control and CI/CD for data code
- Work with offshore data engineers on knowledge transfer and delivery
Data Engineer (Manhattan) in manhattan at Unknown Company
This position is listed as full time and hybrid.