Data EngineerLocation: Houston, USABand: U3Year of Experience: 5 to 7 YearsDuration: 6 Months (Will be extended for 12 months)No Of Positions: 15-20 (3 positions is needed in a month time. in one Quarter time.)High Level ResponsibilitiesConnect and Collect data exploration and preparation.Transform and Enrich data representation and transformation.Publish and Serve Publish data.Monitor data pipelines and solutionsDetailed ResponsibilitiesDeveloping, building, maintaining, and managing data pipelines work with large dataset, databases and software used to analyze them. Fluent with Azure data tools and component.
Primary focus to ensure that data flows smoothly from its source to its destination efficiently and securely.Demonstrate experience in programming scripting (e.g. Python, SQL).Demonstrate experience Analytics/data product solution architecture on Azure (Azure data Factory, Data bricks).Handson experience in data extract, load and transformation techniques and tools including orchestration of needed azure resources.Ensuring the accuracy of data and promoting data quality.Building, testing, and maintaining database pipeline Architecture.Demonstrated experience in Data and Software Engineering tools and Agile methodologies (e.g. Agile, ADO,Ansible,G/T)Demonstrated experience in Analytica Data framework and Azure Analysis ServicesSkillsModern Data WarehousingData flow transformationsImplement and design data engineering and integration patterns: messaging, shared databases, file, service based.Data security design - authorization, sharding, allocations, authenticationDatabase systems and large-scale processing systemsConsume and develop data-driven APIs.Technologies: Azure Data Factory Azure Synapse Azure Data Lake Data bricks Database (SQL, NoSQL, Cosmos DB) Data Flow Transformations via SQL, Python Business Intelligence (Power BI, Spotfire, Power Apps) SAP BWDuties ExamplesBuild scalable data pipelines to enable data-driven work.Design and architect data integration patterns and styles.Establish fit-for-purpose guardrails for data ingestions, transformations.Design and develop reusable data engineering patterns from simple ETL to complex modern data warehousing involving multiple endpoints and data sources.Leverage existing frameworks and accelerators.