Job TitleThe role requires an individual to work 100% of the time at the Client's site in Tallahassee, FL. The candidate must have experience in ETL development and data engineering, specifically with already defined ADF, Azure Databricks, Azure Synapse Analytics, and Azure Purview. They should have experience in building and optimizing large-scale ETL pipelines for high-performance, high availability environments. Expertise in Spark, Python, and/or Scala for large-scale data transformations is required.
Strong SQL proficiency and experience working with complex data structures are necessary. The candidate should also have experience with data governance, security protocols, and role-based access control (RBAC) within the Azure ecosystem. Additionally, experience in designing ETL processes that are resilient, efficient, and fully compliant with regulatory standards is needed.Preferred qualifications include Microsoft Certified: Azure Data Engineer Associate; Microsoft Certified: Azure Solutions Architect Expert; Microsoft Certified: Azure Fundamentals; and Databricks Certified: Data Engineer Associate.ResponsibilitiesResponsibilities include but are not limited to the following: Lead the creation of ETL processes to integrate data across different sources, ensuring high performance and reliability. Build and maintain ETL workflows for analytics and reporting, focusing on data accuracy and security.
Work with Azure Data Factory, Databricks, Synapse, and Purview to handle large datasets and deliver scalable data solutions. Continuously improving ETL processes to reduce latency and increase data processing speed, meeting performance metrics. Implement data security and governance practices, including data lineage tracking and role-based access, using Azure Purview. Engage with data engineers, analysts, and business stakeholders to design solutions that meet departmental goals.
Create and maintain detailed documentation for ETL processes, enforcing best practices for data handling and security.