Data Platform Engineer

Our healthcare client is looking for a Data Engineer with 7+ years experience to own the multi-stage data pipeline layer that ingests from external sources (BigQuery direct queries, vendor file feeds, APIs) into a governed lake/warehouse. Deliver scaled event-driven integrations; managing ETL for downstream integrations, an event-bus for user engagement services, and API / query access for billions of data points.

What you'll do in this role:

Ship a production-ready, like-for-like ETL replacement with zero data loss and validated parity against legacy outputs.

Establish lake → warehouse pipelines with schemas, contracts, and idempotent restatement for corrections.

Implement rules evaluation and event context enrichment to support downstream filtering and event-response integrations.

Stand up observability for pipelines and jobs (SLOs, error/retry rates, latency dashboards, lineage) and on-call runbooks.

Provision reproducible infrastructure with Terraform; define CI/CD and cost/performance guardrails across environments.

Develop and support agentic workflows (data access, context management, & modern AI development practices in your own work) across the organization.

Key Responsibilities

Design and implement ingestion/connectors (BigQuery direct, CSV→JSON, REST) and normalization into standardized data models.

Build event-driven jobs and services (Python/Node) that enrich, dedupe, and apply rules; ensure idempotency and safe replays.

Define data contracts, schema evolution, backfill strategy, and cutover plans; partner with stakeholders on acceptance criteria.

Operate AWS workloads (EC2, Lambda, AppRunner, RDS, Redshift) with Terraform; secure secrets, roles, and least-privilege access.

Optimize SQL for MPP systems (Redshift, Snowflake, or similar); profile queries, partition/cluster, and tune materializations.

Implement observability (logs, metrics, tracing, lineage) and incident response; drive postmortems and remediation.

Maintain concise documentation of architecture, workflows, standards, and governance.

Qualifications

7–10+ years backend/data engineering with production ownership of large event-driven data systems.

Proficient in Python, Node, and similar toolsets for ETL job implementation; strong testing and reliability toolset.

Deep AWS experience and Terraform-based IaC; CI/CD for data and application deployments.

Expert SQL and performance tuning on Redshift, Snowflake, or equivalent.

Experience delivering idempotent pipelines, restaging/reconciliation, and parity validation against legacy systems.

Experience in highly-governed environments (HIPAA, GLBA, PCI, etc.).

Solid security and compliance practices, supporting internal and external audits (SOC 2 ISO 27001, etc.) and remediation.

Preferred

Orchestration experience (Airflow or similar), streaming (Kinesis/Kafka/SQS), and dbt or warehouse-centric modeling.

Data quality frameworks, lineage/metadata tooling, and SLA/SLO design.

Exposure to real-time enrichment and rules engines; familiarity with warehouse-native features (tasks/streams).

Things you should be able to do:

Diagram a multi-stage lake → warehouse pipeline with cutover, backfill, restaging for corrections, and SLOs.

Discuss agentic workflow support from a data/platform perspective.

Demonstrate experience (positive and lessons-learned moments) for at-scale data/platform implementations.