Big Data EngineerResponsible for designing and building big data platforms, tools, and solutions that help manage, secure, and generate value from data. Mostly involved in creating scalable and reusable solutions for gathering, collecting, storing, processing, and serving data at scales. On a day-to-day basis his responsibilities include:Gathering requirements for the business use case and design an implementation plan for the application.Provide different prototypes for the Business use case using Bigdata tools and let client's Analysts team to confirm on best model for Data Analysis.Implement the use case with Bigdata open source tools and technologies like SQOOP, HADOOP, HIVE, Unix Scripting, SPARK, SCALA, PYTHON, PYSPARK, SCALA, AWS and EMR to achieve the requirements.Create the Hive databases and store the processed data into Hive tables and then perform the aggregate operations to generate the new datasets and analyze the data.Provide Big Data Solutions in Cloudera Environment and act as a technical expert addressing problems related to system and application design, Data Lake, Data warehouse, performance optimization, code integration, Automation workflows, etc.Develop an Enterprise Data Lake Application using Bigdata tools like Hive, Spark Streaming, Kafka, Flume, Sqoop, Oozie, Spark, Shell Scripting, Python, SQL, HDFS, Streamsets.Develop the automation scripts for data visualization reports out of processed data to present the research for the end clients.Integrate and deploy the code into lower environments to check for the merge conflicts of the code.Involve in research on the latest open source big data tools to implement the client's requirements at low cost and better performance.Implementing the code and integrating the code into the production servers.Contribute for knowledge share and transfer with the team members for better outputs and best implementation solutions.Provide automation solutions for the jobs which are run manually to generate frequent reports which are utilized by the business team for data analysis.Work Locations: Various unanticipated work locations throughout the United States; relocation may be required.
Must be willing to relocate.Minimum Qualifications Education: Bachelor – Electronics Engineering or TelecommunicationsExperience: NoneJob Order No: ***This position is eligible for a referral bonus through our employee referral program***