Data Architect with GCP

• Define and guide implementation of a metadata-driven generic ETL framework supporting multiple source types including databases, files, APIs, MongoDB, Splunk, VDS, and Active Directory.


• Architect reusable metadata models using MD_ static metadata tables and OPS_ operational/runtime tables for source configuration, pipeline definition, run tracking, audit logging, checkpointing, schema drift, and notifications.


• Provide architecture direction for MVC-based orchestration, including repository layer, service layer, wrapper, controller, execution plan models, and standardized request/response contracts.


• Design scalable ingestion patterns for full load, incremental load, watermark-based processing, CDC-style patterns, batch ingestion, API ingestion, and file-based ingestion.


• Provide technical leadership for GCP / BigQuery architecture, including S3 to BigQuery integration, schema mapping, dataset design, partitioning, clustering, performance optimization, and secure connectivity.


• Review and guide CI/CD automation using GitHub Actions, JFrog, Harness, Liquibase, Helm, OCP deployments, and environment-specific configuration management.


• Provide technical guidance to team on Python, PySpark, SQL, BigQuery, PostgreSQL, data modeling, performance tuning, and production ready framework design.


• Support UAT and production readiness by defining validation scenarios, reconciliation strategy, performance testing approach, security validation, runbooks, and sign-off criteria.


• Partner with client stakeholders to finalize architecture decisions, design documents, Confluence documentation, implementation standards, and technical roadmaps.


• Ensure the solution is secure, scalable, maintainable, metadata-driven, auditable, and operationalized.

Data Lake Architecture & Storage

Data Ingestion & Orchestration

Data Processing & Transformation

DevOps / CI-CD

• Experience implementing CI/CD pipelines for data and analytics workloads


• Familiarity with Git-based source control, build automation, and deployment strategies