- Lead the end-to-end architecture and solution design for enterprise data platforms on Azure, focusing on Databricks Lakehouse, Delta Lake, and scalable cloud-native data ecosystems
- Define target-state data architecture, ingestion patterns, transformation frameworks, and serving layers for reporting, advanced analytics, ML, and business-critical decisioning
- Design and implement ETL/ELT pipelines using PySpark, SQL, Databricks Workflows, Auto Loader, and Delta Live Tables
- Own architecture standards for data modeling, medallion design, reusable engineering patterns, CI/CD, code quality, environment strategy, and release management
- Drive platform governance and security using Unity Catalog, RBAC/ABAC controls, lineage, auditability, and Azure Purview integration
- Optimize Spark workloads, cluster policies, partitioning, file sizing, caching, and compute costs
- Collaborate with stakeholders, product owners, analysts, architects, and downstream consumers to translate requirements into scalable technical designs
- Provide technical leadership through design reviews, implementation guidance, architectural issue resolution, and best-practice establishment
- Evaluate Databricks capabilities including Photon, serverless compute, Lakehouse Federation, and streaming patterns
- Ensure delivery governance through estimation, technical planning, dependency management, risk mitigation, and Agile execution
Requirements
- 10–15 years of experience in data engineering, cloud data platform design, or enterprise data architecture
- At least 5+ years of hands-on experience with Databricks and Azure
- Bachelor’s degree in computer science, Information Technology, Engineering, or a related discipline; master’s degree preferred
- Expertise in Lakehouse, medallion architecture, data modeling, data warehousing, and scalable ingestion and transformation frameworks
- Proficiency in Databricks, PySpark, Python, SQL, Delta Lake, Databricks Workflows, Auto Loader, and Delta Live Tables
- Experience with Azure Data Factory, Azure Data Lake Storage, Azure Key Vault, Azure DevOps, and enterprise cloud ecosystem integration
- Experience defining architecture standards, reusable design patterns, CI/CD strategy, environment management, and delivery best practices
- Experience with Unity Catalog, role-based access controls, data governance, lineage, security frameworks, and Azure Purview
- Ability to optimize large-scale Spark and Databricks workloads, including performance tuning, cluster sizing, workload management, and cost optimization
- Experience translating business requirements into scalable solution designs and implementation roadmaps
- Strong communication, leadership, and problem-solving skills; ability to mentor teams, review technical designs, and drive architecture decisions
- Insurance domain knowledge is preferred
Core Competencies
Demonstrates expertise in designing and implementing enterprise data architectures on Azure, with a focus on Databricks Lakehouse and Delta Lake. Proficient in optimizing data ingestion, transformation frameworks, and ensuring data governance and security.
Highest-signal resume keywords
- Azure Data Platform Design
- Databricks Lakehouse Architecture
- ETL/ELT Pipeline Development
- Data Governance and Security
- Cloud-Native Data Ecosystems
ATS Optimization Keywords
Hard Skills
- Databricks
- PySpark
- SQL
- Delta Lake
- Data Modeling
- Data Warehousing
- CI/CD
- Performance Tuning
- Agile Execution
- Architecture Standards
Soft Skills
- Leadership
- Communication
- Problem-Solving
- Mentoring
Certifications & Qualifications
- Bachelor’s Degree in Computer Science
- Master’s Degree Preferred
Industry Keywords
- Insurance Domain Knowledge
- Data Governance
- Medallion Architecture
- Scalable Ingestion Frameworks
Tools & Technologies
- Azure Data Factory
- Azure Data Lake Storage
- Azure Key Vault
- Azure DevOps
- Unity Catalog
- Azure Purview