Data Engineer
We need an experienced Data Engineer at the National Guard Bureau (NGB). The NGB is the link between the Army and Air National Guard and the Department of War (DoW). You will support operations on the Minuteman project team, which manages the NGB's enterprise data environment within the DoW's central analytics platform, and plays a crucial role in designing, developing, and maintaining the Minuteman Community Space's data infrastructure and systems. In this role, you will design and implement end-to-end data pipelines using ETL processes and technologies including Databricks, Python, Spark, Scala, JavaScript/JSON, and SQL, working closely with cross-functional teams to understand data requirements and design optimal data models and architectures that ensure efficient data processing and analysis. This is a full-time opportunity. We can offer a competitive salary, and a comprehensive benefits package. RESPONSIBILITIES
We need an experienced Data Engineer at the National Guard Bureau (NGB). The NGB is the link between the Army and Air National Guard and the Department of War (DoW). You will support operations on the Minuteman project team, which manages the NGB's enterprise data environment within the DoW's central analytics platform, and plays a crucial role in designing, developing, and maintaining the Minuteman Community Space's data infrastructure and systems. In this role, you will design and implement end-to-end data pipelines using ETL processes and technologies including Databricks, Python, Spark, Scala, JavaScript/JSON, and SQL, working closely with cross-functional teams to understand data requirements and design optimal data models and architectures that ensure efficient data processing and analysis. This is a full-time opportunity. We can offer a competitive salary, and a comprehensive benefits package. RESPONSIBILITIES
- Designs, develops, and implements end to end data pipelines utilizing ETL processes and technologies such as Databricks, Python, Spark, Scala, JavaScript/JSON, and SQL.
- Works closely with cross functional teams to understand data requirements and design optimal data models and architectures.
- Creates and optimizes data pipelines from scratch, ensuring scalability, reliability, and high performance processing.
- Performs data cleansing, data integration, and data quality assurance activities to maintain the accuracy and integrity of large datasets.
- Leverages big data technologies to efficiently process and analyze large datasets.
- Collaborates with data scientists, analysts, and stakeholders to provide timely and accurate data insights and support decision making processes.
- Troubleshoots data related problems and provides innovative solutions to complex data challenges.
- Implements and enforces data governance policies and procedures, ensuring compliance with regulatory requirements and industry best practices.
- Three (3) to ten (10) years of directly related experience
- 3+ years of experience as a Data Engineer
- Proficiency in ETL processes and firsthand experience with technologies such as Databricks, Python, Spark, Scala, JavaScript/JSON, SQL, and Jupyter Notebooks
- Experience working with large datasets and leveraging big data technologies to process and analyze data efficiently
- Understanding of data modeling/visualization, database design principles, and data governance practices
- Experience with the Department of War (DoW) mission and associated datasets
- Previous experience with Advana or the War Data Platform
- Experience with APIs
- Experience with Machine Learning models and/or Generative AI
- Experience building within Palantir Foundry applications, including Workshop, Ontology Manager, and Gaia
- B.A./B.S. or M.A./M.S. in a related field
- Remote
- Active Secret security clearance
- U.S. citizenship is required since this role supports the U.S. Federal government
- OUSD Personnel & Readiness (P&R), Minuteman project team
- 40 hours per week. 8 hours per day
- Employment Classification Eligibility — W2
- Salary range: $125,000 - $135,000
- Benefits: Benefits package includes options for health, dental, and vision insurance coverage; 401k contribution options
Data Engineer (Remote) in yorktown at Unknown Company
This position is listed as full time and able to be worked remotely.