Sr. Hadoop Data EngineerHalo Group is a premier provider of IT talent. We place technology experts within the teams of the world's leading companies to help them build innovative businesses that keep them one step closer to their customers and one step ahead of the competition.
We offer a meaningful work environment for employees, attractive and interesting engagements for consultants, and cutting-edge digital innovation for our customers. We delight in helping our customers execute their digital vision. Big projects or small, Halo Group knows that by combining the highest quality talent with our unwavering support, we will become an invaluable extension of the team.
Halo Group's experienced consultants in Detroit, Atlanta and Dallas specialize in all areas of product/project governance, UX/UI, multi-platform applications, quality assurance/testing, cloud computing, and data analytics. Since its inception, Halo Group has been recognized for numerous awards, including: INC 5000, Future 50, 101 Best and Brightest, Michigan 50 Companies to Watch, Goldline Research, "Most Dependable Companies", Ernst & Young, "Entrepreneur of the Year" Finalist.Job DescriptionPreferred:Selecting and integrating Big Data tools and frameworks required to provide requested capabilitiesImplementing Data ingestion and ETL processes on HadoopMonitoring performance and advising any necessary infrastructure changesDefining data retention policiesDesign and build data processing pipelines for structured and unstructured data using tools and frameworks in the Hadoop ecosystemDevelop applications that are scalable to handle millions of events/recordsDesign and launch scalable, reliable and efficient processes to move, transform and report on large amounts of dataParticipate in meetings with business (account/product management, data scientists) to obtain new requirementsFollow our Agile software development process with daily scrums and monthly SprintsAbility to work collaboratively on a cross-functional team with a wide range of experience levelsQualificationsBachelor's degree and 8+ years relevant experience or Master's degree and 6+ years of relevant experience4+ years in industry implementing big data solutions on HadoopProficient understanding of distributed computing principlesProficiency with Hadoop v2, MapReduce, HDFSExperience with building stream-processing systems, using solutions such as Storm or Kafka and Spark-StreamingGood knowledge of Big Data querying tools, such as Pig, Hive, PhoenixExperience with SparkExperience with integration of data from multiple data sourcesExperience with 1 or 2 NoSQL/Graph databases, such as HBase, Cassandra, MongoDB, Neo4jProficiency in a programming languages like SCALA, Java, PythonExperience with Linux OS, shell scriptingExperience with relational databases (SQL)Experience in working with real-time data feedsExperience in working with unstructured dataExperience in implementing Scoop Jobs to Import/Export data from HadoopKnowledge of various ETL techniques and frameworks, such as Pig, Hive, or FlumeExperience with various messaging systems, such as Kafka or RabbitMQExperience with Big Data ML toolkits, such as Mahout, SparkML, or H2OGood understanding of Lambda Architecture, along with its advantages and drawbacksExperience with Hortonworks Hadoop Data Platform (HDP)Experience with all or some of the following supporting Hadoop administration and security frameworks: HCatalog, Drill, NiFi, Oozie, Falcon, Ranger, Ambari, Zeplin.Additional Information** U.S. Citizens and those who are authorized to work independently in the United States are encouraged to apply.
We are unable to sponsor at this time. This is a Full-Time / Permanent job opportunity. Only for US Citizen and Green Card Holder ** All your information will be kept confidential according to EEO guidelines.By clicking the link above or any third-party link within this posting, you are leaving this site and going to a third-party website where the third-party website's terms and privacy policy apply