DataBricks Solution Architect

Required Skills:

• Understanding how application consume data
• Need someone who can come in look at the architecture and code and offer optimization solution
• Focus on performance, cost, security and governance
• Deliver an optimization roadmap

·       DataBricks Solutions architect on the application side.

·       Someone who has experience with consumption of data

Responsibilities:

1. Architecture assessment

• Review the existing Databricks architecture
• Understand how applications consume Databricks data
• Review data flows, integrations, APIs, SQL Warehouses, clusters, Delta tables, etc.
• Identify architectural bottlenecks and unnecessary complexity

2. Code review
This is critical. They should be comfortable getting into the actual:

• PySpark code
• Spark SQL
• Databricks notebooks
• SQL queries
• ETL/ELT pipelines
• Delta Lake implementation
• Streaming code,