- Design and implement end-to-end data architecture using Databricks Lakehouse platform
- Define data engineering best practices and build scalable ETL/ELT pipelines
- Architect solutions leveraging Apache Spark (PySpark/Scala) and Databricks workflows
- Implement Delta Lake for data reliability, performance, and governance
- Collaborate with stakeholders to understand business requirements and translate them into technical solutions
- Optimize performance, cost, and scalability of data pipelines
- Provide technical leadership , mentoring data engineers and developers
- Ensure data security, governance, and compliance standards are met
- Integrate Databricks with cloud services (Azure, AWS, or GCP) and other enterprise systems
- Support real-time and batch data processing use cases
Required Skills & Qualifications
Technical Skills
- Strong hands-on experience with Databricks Platform
- Expertise in Apache Spark (PySpark / Scala / SQL)
- Experience with Delta Lake, Unity Catalog, and Lakehouse architecture
- Proficiency in data pipelines, ETL/ELT frameworks
- Strong experience in SQL and data modeling
- Hands-on experience with cloud platforms:
- Azure (ADF, ADLS, Synapse) OR
- AWS (S3, Glue, EMR) OR
#J-18808-Ljbffr