Data Engineer

Title: Data Engineer
Duration: asap - 4 Months
Position: Remote (The person can be remote, though local to Boston.)
Details:
Client is seeking a Data Engineer to design, develop, implement discrete data pipelines in support of RAG-based retrieval, inferencing, and reasoning. Self-starter and collaborative mentality is a critical part of this fast-moving team. Experienced-based proficiency with Microsoft Azure environment is necessary.
Data Engineering Enablement

• Build batch/streaming ingestion pipelines (ADF/Synapse/Databricks Workflows, Event Hubs/Kafka).
• Implement change data capture (CDC) patterns and schema evolution handling with quarantine controls.
• Support automation of data reliability assessment.

Operations and Reliability

• Establish continuous integration/continuous deployment (CI/CD) pipelines, along with infrastructure-as-a-code orchestration (Terraform or Bicep), and segregated environment strategy (dev/test/prod).
• Define service level agreements (SLAs) and service level objectives (SLOs) for data reliability; configure monitoring & alerting.
• Deliver cost optimization, performance tuning (OPTIMIZE, Z-ORDER), Backup strategy, and Disaster Recovery strategy.

Delivery and Handover

• Document architecture, patterns, and runbooks; conduct training for data engineers.
• Collaborate with security, networking, and BI teams; drive stakeholder alignment.

Required Skills & Experience

• 5-8 years in data engineering and data pipeline development
• Proficient in Python and SQL; experience with CI/CD and IaC (Terraform or Bicep).
• Excellent communication, documentation, and stakeholder management.
• Expertise in engagement with business users to align business objectives with technical considerations within Azure.