DevOps for Spark/HadoopManage, deploy, maintain, and finetune software applications on private/public clouds in both batch and realtime worlds. Work closely with development, release engineering teams closely to propose and implement optimizations. Make physical deployment recommendations on both public and private clouds.
Maintain Spark/Hadoop infrastructure and monitor data services and solutions.Requirements:7 or more years relevant work experienceExperience with Linux operating systemExperience working in production environmentExperience with Spark (Hadoop) ecosystemExperience with Bash or PythonExperience with monitoring tools like Nagios/SplunkExperience in managing ETL jobs in production and reporting failuresCI CD Expertise (Screwdriver/Jenkins)Good to Have:At least 7 years experience in Linux Administration (Redhat and CentOS distributions preferred).At least 5 years experience in managing Spark/Hadoop clusters, including providing monitoring and administration support ofSpark, HBase/HiveKafkaExperience with configuration management tools, deployment pipelines, and orchestration servicesAnsible or ChefJenkinsExperience managing cloud native applications and infrastructure.Experience troubleshooting and actively debugging live production systems.Ability to identify improvements, and work with the team to design solutions that will be deployed into production.Ability to work independently and as a member of collaborative teams.Provide on-site Linux System Administration support for Hadoop Cluster and related software stackWork directly with software developers to simplify processes, enhance services and automate application delivery facilitating the release of production-ready software.Proactively escalate and own issues to resolution before they become problems.Document maintenance and support procedures for routine and complex tasks.