Data Engineer - 100% Remote - Contract thru EOYInterviews Set: 5/18, 5/19, 5/20Interview Process:30 minute phone screen1 hour technical interviewTop 3 Requirements:Ability to work independently and figure things out on their ownSQL – Senior levelApache Spark (with Scala) – Could look at Pyspark, but need to be senior level in either oneSnowflake – Just need knowledge, don’t need to be an expertNice To Have:SnowflakeRally/Agile experienceTeam: 2 Data Engineers, 2 BA’s, 1 ArchitectProject: Transformation of Data / Ingestion. Taking old system that is decommissioned, and moving data to a new system (new schema, new design, etc.). Will ingest data, transform it, and make it match the reports.
3 options that they are considering: Flat file (from new system), direct connect to new system, or Kafka streaming.Responsibilities:Research, proof of concepts, architecture, design, development, test, deployment and maintenanceDoes not have a lot of documentation for a path forward, so there will be a lot of discovery workWalk back data and figure it (may not get requirements)Writing logic for data ingestion and ETLRally for Agile – Know how to do user stories