The Databricks Data Engineer will play a critical role in accelerating data modernization efforts . Their expertise will support timely report conversion, improve data pipeline efficiency , and help ensure data integrity and accessibility across the organization . Databricks platform expertise (workspace management, clusters, job scheduling). Spark (PySpark/Scala) proficiency. SQL optimization and advanced querying skills. Azure Data Factory (ADF) or equivalent ETL pipeline experience Data warehousing concepts and architecture (Delta Lake) . Python scripting for data manipulation and automation . Familiarity with Azure Cloud environment (Blob Storage, ADLS) . Knowledge of data governance, data security, and compliance .
Experience Required:
- 3+ years hands-on experience with Databricks, including cluster management and optimization.
- 3+ years of Delta Lake architecture and implementation. 5+ years background in PySpark or Scala for data engineering tasks.
- 5+ years of demonstrated success building efficient ETL pipelines, using Azure Data Factory (ADF) or similar.
- 5+ years of migrating legacy reports to modern cloud analytics platforms.
- 5+ years of strong SQL skills with expertise in data modeling and performance tuning.
- 5+ years of BI tools such as Power BI or Tableau.
Education:
- Bachelor’s degree in Computer Science, Information Systems, Data Science, or related technical field.
- Relevant industry certifications in Databricks, Azure Cloud, Data Engineering, or similar fields.
- Databricks Certified Data Engineer Associate (No more than 12 months expired)
Seniority level
Seniority level
Mid-Senior level
Employment type
Employment type
Contract
Job function
Job function
Information TechnologyIndustries
Government Administration
Referrals increase your chances of interviewing at HireTalent - Staffing & Recruiting Firm by 2x
San Francisco, CA $110,000 - $170,000 3 weeks ago
Irvine, CA $140,000 - $170,000 3 weeks ago
We’re unlocking community knowledge in a new way. Experts add insights directly into each article, started with the help of AI.
#J-18808-Ljbffr