Azure Databricks Architect

Location: Seattle, WA (Hybrid – 3 Days Onsite)

Duration : Full-Time

Job Summary

We are seeking an experienced Azure Databricks Architect to lead the design, implementation, and optimization of enterprise-scale data platforms on Azure. The ideal candidate will have deep expertise in Azure Databricks, Spark, Delta Lake, and modern Lakehouse architecture, with strong experience designing secure, scalable, and high-performance data solutions. Knowledge of Unity Catalog, Medallion Architecture, and Infrastructure as Code is highly preferred. These are all considered current best practices for enterprise Azure Databricks environments.

Required Skills

• 12+ years of IT experience with 5+ years as a Data/Cloud Architect.
• Strong hands-on experience with Azure Databricks .
• Expertise in PySpark, Spark SQL, Python, and SQL .
• Strong knowledge of Delta Lake , Lakehouse Architecture , and Medallion (Bronze/Silver/Gold) Architecture .
• Experience with Unity Catalog , data governance, RBAC, and data lineage.
• Hands-on experience with Azure Data Factory (ADF) .
• Strong experience with Azure Data Lake Storage (ADLS Gen2) .
• Experience with Azure Synapse Analytics or Microsoft Fabric is a plus.
• Experience with CI/CD using Azure DevOps , Git, and Infrastructure as Code (Terraform/Bicep/ARM).
• Strong understanding of performance tuning, cluster optimization, partitioning, caching, and cost optimization.
• Experience implementing data quality, monitoring, logging, and observability.
• Knowledge of streaming technologies (Structured Streaming/Event Hubs/Kafka) is preferred.
• Excellent communication and stakeholder management skills.

Responsibilities

• Design and architect enterprise-scale Azure Lakehouse solutions.
• Define end-to-end data architecture, ingestion, transformation, and serving layers.
• Design secure data governance using Unity Catalog and role-based access controls.
• Optimize Spark workloads for performance and cost.
• Implement reusable frameworks, coding standards, and CI/CD pipelines.
• Work closely with business, engineering, and cloud infrastructure teams.
• Provide architectural guidance, code reviews, and technical leadership.
• Ensure scalability, reliability, security, and disaster recovery across the platform.

Nice to Have

• Databricks Certified Professional/Architect certification.
• Experience with MLflow, MLOps, or AI/GenAI workloads on Databricks.