ETL/Pipeline EngineerInsight Global is seeking an experienced ETL/Pipeline Engineer to support a large-scale initiative focused on modernizing and scaling data pipeline capabilities across our client's retail technology ecosystem. This individual will provide hands-on engineering support across data ingestion, transformation, validation, orchestration, and downstream analytics workflows that support critical business operations and high-volume data processing. Day-to-day responsibilities include translating business requirements, source-to-target mappings, data contracts, and schema requirements into scalable ETL logic; building and optimizing Spark-based processing jobs using Python, PySpark, Spark SQL, and SQL; migrating and validating existing data pipelines; performing data quality checks, reconciliation, functional testing, integration testing, regression testing, and performance testing; and supporting job scheduling, dependency management, monitoring, troubleshooting, and production readiness activities.
This person will partner closely with data engineering, QA, DevOps, architecture, product, analytics, and business stakeholders to manage technical risks, resolve blockers, maintain documentation, support deployment readiness, and ensure reliable delivery of production-ready data solutions. The ideal candidate will have a strong background in ETL development and data pipeline engineering, experience working in complex enterprise environments, and the ability to stay hands-on while driving pipelines through development, testing, release readiness, production support, and operational handoff.