Technical Lead

Job Description

We are seeking a Quality Assurance Lead (Contractor) to drive end-to-end testing strategy and quality engineering practices for large-scale data platforms and distributed systems. This role will focus on ensuring data integrity, regulatory compliance, and system reliability, particularly within high-volume batch and streaming environments. The ideal candidate brings strong expertise in big data testing, automation frameworks, and data validation, along with experience in workflow orchestration and DevOps-driven QA processes.

Responsibilities

• Lead and define the QA strategy, test planning, and execution across complex data and application platforms.
• Design and implement automated testing frameworks using Python and PySpark for large-scale data validation.
• Validate data pipelines across Hadoop ecosystem (HDFS, Hive, Impala, Sqoop) ensuring completeness, accuracy, and consistency.
• Develop end-to-end test scenarios covering batch processing, data ingestion, transformation, and downstream consumption.
• Perform database validation and reconciliation using Oracle PL/SQL and SQL-based techniques.
• Validate machine learning pipelines and models, including experiment tracking via MLflow.
• Monitor and validate workflows orchestrated through Control-M / CA-7 schedulers.
• Collaborate with development teams to embed quality gates into CI/CD pipelines using Jenkins and Git.
• Drive test automation within DevOps pipelines, ensuring seamless integration with build and release processes.
• Perform Unix-based scripting for test automation, data checks, and batch validation.
• Lead defect triage, root cause analysis, and resolution tracking.
• Ensure adherence to regulatory and audit requirements, especially in data-sensitive environments.
• Mentor QA engineers and promote best practices in test automation and quality engineering.

Required Skills & Experience

• 8+ years of experience in Quality Assurance / Test Engineering, with leadership responsibilities.
• Strong proficiency in Python and PySpark for automation and data validation.
• Solid experience with big data platforms (Hadoop, Hive, Impala, Sqoop).
• Hands‑on expertise in Oracle PL/SQL and relational database validation.
• Experience with workflow schedulers such as Control‑M or CA‑7.
• Strong Unix/Linux scripting experience.
• Experience integrating QA into CI/CD pipelines using Jenkins and Git.
• Familiarity with DevOps practices and automated pipeline testing.
• Experience with container platforms (Docker / OpenShift).
• Strong analytical and debugging skills, especially in data-related issues.

Nice‑to‑Have Skills

• Experience with Actimize or OFSAA FCCM platforms (financial crime / AML domain).
• Exposure to cloud platforms (AWS – Glue, Athena, S3, SageMaker).
• Experience testing machine learning models and data science workflows.
• Knowledge of data governance, lineage, and regulatory reporting frameworks.

Salary & Benefits

Salary range: $100,000 – $110,000 a year.

• Discretionary Annual Incentive.
• Comprehensive Medical Coverage: Medical & Health, Dental & Vision, Disability Planning & Insurance, Pet Insurance Plans.
• Family Support: Maternal & Parental Leaves.
• Insurance Options: Auto & Home Insurance, Identity Theft Protection.
• Convenience & Professional Growth: Commuter Benefits & Certification & Training Reimbursement.
• Time Off: Vacation, Time Off, Sick Leave & Holidays.
• Legal & Financial Assistance: Legal Assistance, 401K Plan, Performance Bonus, College Fund, Student Loan Refinancing.

Education

Bachelor of Computer Science.