Unknown Company

Big Data Engineer // Norfolk Southern Railway

atlanta, ga • Posted 1 weeks ago
Remote Contract General

Job TitleVisa: USC and GC only Atlanta, GA (Remote 100%) 2+ years contractSkills and Experience:Required:9+ years of overall IT experience4+ years of experience with high-velocity high-volume stream processing: Apache Kafka and Spark StreamingExperience with real-time data processing and streaming techniques using Spark structured streaming and KafkaDeep knowledge of troubleshooting and tuning Spark applications4+ years of experience with data ingestion from Message Queues (Tibco, IBM, etc.) and different file formats across different platforms like JSON, XML, CSV4+ years of experience with Big Data tools/technologies like Hadoop, Spark, Spark SQL, Kafka, Sqoop, Hive, S3, HDFS, or Cloud platforms e.g. AWS, GCP, etc.4+ years of experience building, testing, and optimizing 'Big Data' data ingestion pipelines, architectures and data sets2+ years of experience with Kudu and Impala2+ years of experience with Scala (and/or Python) and PySpark/Scala-Spark2+ years of experience with NoSQL databases, including HBASE and/or CassandraKnowledge of Unix/Linux platform and shell scripting is a mustStrong analytical and problem solving skillsPreferred (Not Required):Experience with Cloudera/Hortonworks HDP and HDF platformsExperience with NIFI, Schema Registry, NIFI RegistryStrong SQL skills with ability to write intermediate complexity queriesStrong understanding of Relational & Dimensional modelingExperience with GIT code versioning softwareExperience with REST API and Web ServicesGood business analyst and requirements gathering/writing skillsEducation Bachelor's Degree required. Preferably in Information Systems, Computer Science, Computer Information Systems or related field

Back to Job Search