Azure Databricks DeveloperDesign, develop, and maintain cloud native data engineering solutions using Azure Databricks.Build and manage PySpark notebooks to process large scale structured and semi structured datasets.Design, create, and maintain Delta Lake tables, ensuring data reliability, ACID transactions, and schema enforcement.Develop scalable data workflows and pipelines using Databricks notebooks and orchestration patterns.Optimize performance of Spark jobs, including tuning partitions, memory usage, caching strategies, and query execution.Work extensively with PySpark and Spark SQL, choosing the appropriate approach based on use case and performance needs.Support cloud data migration initiatives, migrating data pipelines from on prem or legacy platforms to Azure Databricks.Integrate Databricks with upstream and downstream systems (e.g., data sources, storage layers, reporting tools).Ensure data pipelines are robust, reusable, and maintainable, following enterprise data engineering best practices.Implement error handling, logging, monitoring, and recovery strategies for production grade data pipelines.Collaborate with data architects, analysts, and downstream consumers to understand data requirements.Perform debugging and root cause analysis for data quality, performance, or pipeline failures.Support testing, validation, and reconciliation of data during development, migration, and production phases.Follow security, governance, and compliance standards applicable to cloud data platforms.Actively participate in Agile/Scrum delivery, owning data engineering stories from development through deployment.Maintain documentation for notebooks, workflows, data models, and migration approaches.