Data Engineer-Azure Data Bricks, JavaLocation: Eden Prairie, MN (Onsite)Duration: 6+ MonthsJob Description:Azure Databricks and Java are the mandate skills other can be secondary8+ years hands-on experience with Azure Data Bricks. Must be well versed with the platformDesign and implement scalable and efficient data pipelines to ensure smooth data movement from multiple sources using Azure DatabricksDeveloping scalable and re-usable frameworks for ingesting of data setsEnsure data quality and integrity throughout the entire data pipeline, implementing robust data validation and cleansing mechanismsWorking with event based / streaming technologies to ingest and process dataDevelop and maintain data pipelines on Snowflake and Azure platforms. Assist in deploying applications with Kubernetes and Dockers.
Contribute to event streaming and processing using KAFKAMust have 3+ years of relevant experience Java development background with strong Azure knowledge, develop and maintain Java applications using Springboot, Jboss, and other technologiesAzure Cloud Experience – hands-on experience with Storage Format, provisioning options and single tenant and multi-tenant conceptsBigdata Experience: Proficient in ETL development with Talend, Big Data processing and storage, MDM implementation for data consistencyPython/Spark/Scala development > 5+ years coding experienceLead the design and implementation of cloud-based data solutionsThe candidate should be very well versed with Cloud technology and Java is a must