Unknown Company

GCP Spanner Data Engineer

dallas, tx • Posted 3 days ago
Onsite Contract General

GCP Spanner Data EngineerLocations: Dallas, TX / Irving, TX / Basking Ridge, NJ Employment Type: Contract / C2C Experience Required: 12+ Years Project Duration: 12 monthsWork Arrangement: Work from Client Office – 3 days a weekInterview: Three rounds of Video interviewTelecom domain experience is a plus.Must have:Minimum 2 years of GraphQL schema work exp. Minimum 2 years of GCP Spanner, Python and BigQuery work exp.Job Description:Key Responsibilities:Design and manage the GraphQL schemaBuild highly optimized resolver functions that bridge the GraphQL schema directly to data warehousesImplement GraphQL Subscriptions to stream live data, event changes, or real-time metrics using message brokers like Apache KafkaDesign and implement scalable data pipelines using GCP-native services (Dataflow, Dataproc, Pub/Sub, Cloud Composer/Airflow)Architect and optimize BigQuery datasets, tables, and queries for analytical workloads at scaleDesign and manage Cloud Spanner schemas for globally distributed, strongly consistent transactional dataBuild and maintain data models, transformations, and orchestration workflows using Cloud Workflows and related toolsDevelop backend data services and ETL/ELT scripts in PythonIntegrate and manage Firestore for real-time, document-oriented data use casesImplement data governance, lineage, and quality frameworks using tools like Dataplex or Data CatalogCollaborate on infrastructure-as-code using Terraform for GCP resource provisioningMonitor pipeline health, optimize costs, and troubleshoot production issuesRedesign and optimize existing data pipelines and architectures as needed to ensure high performance and scalability.Oversee the end-to-end data delivery process, from ingestion to transformation and reporting.Perform code reviews, enforce best practices, and ensure the quality and consistency of the codebase.Manage workflows and scheduling with tools like Apache Airflow to ensure smooth execution of pipelines.Use GCP technologies like Dataflow, Apache Beam, BigQuery, Dataproc, and other services for data transformation, storage, and analysis.Develop scalable solutions with Apache Spark, Hadoop, and other distributed systems on GCP.Monitor and optimize the performance of data pipelines and processes.Collaborate with DevOps and Cloud teams for CI/CD integration and infrastructure optimization.Provide mentorship to junior engineers and manage the team's performance effectively.Required Skills and Experience:12+ years of experience in data engineering with at least 5+ years on Google Cloud Platform (GCP).Must have minimum 2 years of GraphQL schema work exp.Must have minimum 2 years of GCP Spanner, Python and BigQuery work exp.Solid understanding of network/telecom domains, including relevant data types and use cases.Expertise in GCP tools, including: BigQuery for schema design, partitioning, clustering, query optimization, cost governance, data warehousing and analytics; Dataproc for managing Apache Spark and Hadoop clusters; Airflow for orchestration of workflows and pipelines; Data streaming with Python.Strong experience in Apache Spark and Hadoop ecosystems.Production experience with Cloud Spanner — schema design, interleaving, transaction patterns, and performance tuningSolid understanding of GCP data services: Dataflow, Pub/Sub, Cloud Storage, Dataproc, Cloud ComposerExperience with Cloud Workflows for serverless orchestrationHands-on experience with Firestore (Native mode preferred) for NoSQL/document storage patternsStrong SQL skills and understanding of data warehousing conceptsExperience with CI/CD pipelines (Cloud Build, GitHub Actions) and version control (Git)Ability to design, develop, and optimize ETL/ELT pipelines for large-scale data.Hands-on experience in data modeling and data architecture design.Strong problem-solving skills and ability to redesign existing solutions if needed.Excellent communication and interpersonal skills to collaborate with both technical and business teams.Bachelor's Degree in Computer Science, Engineering, or a related field; OR equivalent combination of education and relevant experience.Preferred Skills:Familiarity with real-time data streaming technologies.Experience with dbt for transformation layer on BigQueryFamiliarity with streaming architectures (exactly once semantics, late data handling)Knowledge of data mesh or data Lakehouse patternsExposure to Vertex AI or ML pipelines for MLOps workflowsGCP Professional Data Engineer certification

Back to Job Search