Databricks Solution Architect

Job Title: Databricks Solution Architect/ RSA

Location: Remote

Employment Type: W2 Contract (No C2C and H1B candidates)

Rate: $75-90/hr

About the Role

We are seeking an experienced Databricks Solution Architect (RSA) with 10–12+ years of experience to lead the architecture, design, and delivery of enterprise data and Lakehouse solutions across multiple client engagements.

This role is suited for an architect who can operate effectively in complex enterprise environments, work directly with technical and business stakeholders, and adapt to evolving project requirements. You will provide architectural direction while remaining close to implementation, helping client teams build scalable, secure, governed, and production-ready solutions on the Databricks platform.

Required Experience

• 10–12+ years of experience across Data Engineering, Cloud Engineering, Data Platforms, or Solution Architecture.
• 5+ years of hands-on experience designing and delivering solutions on the Databricks Lakehouse Platform.
• Proven experience architecting and leading enterprise-scale cloud data platform implementations.
• Strong client-facing experience working with engineering teams, architects, business stakeholders, and technical leadership.
• Experience leading solution design sessions, architecture reviews, and technical decision-making.
• Demonstrated ability to mentor engineers, provide architectural guidance, and establish engineering best practices.
• Experience taking solutions from architecture and design through implementation, production rollout, and optimization.

Key Responsibilities

• Lead technical priorities across multiple enterprise client engagements, balancing immediate delivery needs with scalable architecture and long-term platform strategy.
• Architect and maintain CI/CD pipelines using GitHub Actions, Azure DevOps, or Jenkins to support automated testing, version control, and reliable deployment of Databricks workloads and platform assets.
• Serve as a primary technical advisor to client delivery teams, translating business and technical requirements into scalable, production-ready solutions while guiding architectural decisions.
• Architect, implement, and optimize enterprise Databricks Lakehouse platforms using Medallion Architecture patterns to support multiple business domains and large-scale production workloads.
• Own the end-to-end solution lifecycle, including technical design, ETL/ELT development, orchestration, deployment, monitoring, performance tuning, and production support.
• Design and implement governance frameworks using Unity Catalog, RBAC, ABAC, lineage, and compliance controls while enabling secure and efficient development practices.
• Design integration patterns across AWS, Azure, and GCP services and enterprise systems including CRM, ERP, APIs, Kafka, Kinesis, and other streaming or batch data sources.
• Lead architecture reviews and technical design discussions, mentor engineering teams, and establish reusable platform standards and engineering best practices.
• Drive performance and cost optimization across Databricks compute, SQL Warehouses, Spark workloads, storage, and supporting cloud infrastructure.

Technical Qualifications

• Strong hands-on experience with Unity Catalog, Lakeflow Pipelines (formerly Delta Live Tables), Photon, Lakehouse Federation, and enterprise governance patterns including RBAC, ABAC, lineage, and data access controls.
• Experience designing and operating Lakeflow Jobs, Databricks Workflows, SQL Warehouses, and serverless compute for production data workloads.
• Demonstrated experience designing CI/CD and release processes for Databricks using GitHub Actions, Azure DevOps, Jenkins, or similar tooling.
• Hands-on experience with Terraform, Infrastructure as Code, Databricks CLI, and Declarative Automation Bundles (formerly Databricks Asset Bundles) for repeatable platform and workload deployments.
• Strong experience with Apache Spark, PySpark, Spark SQL, and Scala, including query tuning, workload optimization, partitioning, data layout, and large-scale pipeline design.
• Experience with orchestration and transformation frameworks such as Airflow and dbt, including dependency management, testing, and production scheduling.
• Strong cloud architecture experience across AWS, Azure, or GCP, including networking, IAM, storage, compute, security, and private connectivity patterns.
• Experience designing batch, streaming, and near-real-time architectures using Kafka, Kinesis, Pub/Sub, Auto Loader, and Structured Streaming.
• Experience implementing enterprise data integration patterns across relational databases, APIs, streaming systems, SaaS platforms, and external data platforms.
• Familiarity with Python-based APIs and data services using frameworks such as FastAPI or Flask.

Interview & Placement Process

Our hiring process is designed to move efficiently while ensuring strong alignment between your experience and the needs of each client engagement. Most candidates complete the process within one to a few weeks, depending on project availability and client scheduling.

1. Initial Screening: A brief conversation with the CloudTech Innovations team to review your background, experience, technical strengths, and overall alignment with the role.
2. Client Introduction: An introductory discussion with the client or delivery team to review the engagement, understand high-level expectations, and confirm mutual alignment.
3. Technical Deep-Dive: A focused technical interview conducted by our implementation partner. This discussion evaluates your architecture experience, Databricks knowledge, problem-solving approach, and ability to work across different enterprise environments. Your strengths are then aligned with the end-client opportunities that best match your background.

Project Flexibility: If your experience is strong but the initial engagement is not the right match, we may consider you for other active client opportunities that better align with your technical background and experience.

1. Final Client Interview: A final discussion with the end client to confirm technical and project fit. Depending on the engagement, additional interviews may be required based on client-specific needs.

Ideal Candidate Profile

• Adaptable Technical Leader: Comfortable stepping into unfamiliar environments, codebases, and delivery models, quickly understanding the landscape and contributing meaningful technical direction.
• Delivery-Oriented Architect: Able to balance near-term delivery needs with long-term architectural quality, scalability, maintainability, and platform stability.
• Trusted Client Advisor: Strong communication and stakeholder-management skills, with the ability to explain technical trade-offs, guide decisions, manage expectations, and build trust across technical and business teams.
• Strong Ownership Mindset: Takes accountability for solution quality, implementation outcomes, production stability, and the overall success of the engagement.
• Hands-On Leadership: Willing to move beyond high-level architecture and work closely with engineering teams on implementation, troubleshooting, design decisions, and delivery execution.
• Certifications: Databricks certifications and cloud architecture certifications across AWS, Azure, or GCP are preferred. TOGAF or similar enterprise architecture certifications are a plus.