We are seeking an experienced Databricks Architect with proven hands-on experience leading or executing Greenplum-to-Databricks migration projects . The ideal candidate will have strong expertise in Databricks architecture, data migration, data warehouse modernization, and large-scale data platform transformations.
The candidate will be responsible for architecting and supporting the migration of data, workloads, ETL processes, and data pipelines from Greenplum to Databricks , ensuring a smooth, scalable, and efficient transition.
Key Responsibilities
- Lead the architecture and technical execution of Greenplum-to-Databricks migration initiatives .
- Assess existing Greenplum databases, schemas, tables, workloads, stored procedures, and ETL processes.
- Develop migration strategies and roadmaps for transitioning data and workloads from Greenplum to Databricks .
- Design scalable Databricks Lakehouse architecture to support migrated workloads.
- Migrate and optimize data pipelines, SQL queries, ETL/ELT processes, and data models for the Databricks environment.
- Identify migration challenges, dependencies, and performance considerations during the transition.
- Ensure data accuracy, integrity, security, and performance throughout the migration process.
- Work closely with data engineers, architects, developers, and business stakeholders to plan and execute migration activities.
- Provide technical leadership and guidance on Databricks best practices and migration methodologies.
- Troubleshoot and resolve migration-related issues and optimize migrated workloads for performance and scalability.
- Develop technical documentation, migration plans, architecture diagrams, and implementation guidelines.
Required Skills & Qualifications
- Mandatory: Proven experience migrating data and workloads from Greenplum to Databricks.
- Strong experience as a Databricks Architect, Data Architect, or Senior Data Engineer .
- Hands-on expertise with Databricks and Lakehouse architecture .
- Strong knowledge of Apache Spark, PySpark, and SQL .
- Experience with data warehouse modernization and large-scale data migration.
- Strong understanding of ETL/ELT pipelines, data modeling, and data integration.
- Experience analyzing and migrating Greenplum databases and workloads .
- Experience with cloud platforms such as Azure, AWS, or GCP .
- Strong problem-solving and communication skills.
Preferred Skills
- Experience with Delta Lake and Unity Catalog .
- Experience migrating Greenplum SQL, stored procedures, and ETL workloads to Databricks.
- Knowledge of Databricks Workflows and data pipeline orchestration.
- Experience with cloud data migration and modernization programs.
- Experience with performance tuning and optimization of migrated workloads.