AI Data Engineer

Data EngineerMust Have Technical/Functional SkillsStrong hands-on experience in Python, SQL and data engineering.Experience designing and developing ETL/ELT pipelines for large-scale data processing.Strong knowledge of PySpark / Spark for data transformation and processing.Experience with AWS data services such as Glue, S3, Redshift and related cloud services.Hands-on experience with Apache Airflow or equivalent orchestration tools.Strong understanding of data transformation, cleansing, validation, reconciliation and data-quality frameworks.Experience working with structured, semi-structured and unstructured data.Knowledge of batch and streaming data pipelines, preferably Kafka.Experience building data pipelines supporting AI/ML, GenAI, LLM and RAG use cases.Experience with CI/CD, Git, Docker and Kubernetes/EKS for production deployments.Good understanding of data modeling, performance optimization and production support.Roles & ResponsibilitiesDesign, develop and maintain scalable end-to-end ETL/ELT and data pipelines.Build data processing and transformation solutions using Python, SQL and PySpark.Develop batch and streaming pipelines to integrate data from databases, APIs, files and enterprise source systems.Implement data-quality checks, validation, reconciliation, deduplication and exception handling.Optimize data pipelines, SQL queries and Spark workloads for high-volume processing.Prepare and transform datasets required for AI/ML and Generative AI applications.Develop data ingestion and preparation pipelines supporting LLM/RAG solutions and vector databases.Collaborate with AI/ML engineers, application teams and business stakeholders on downstream data requirements.Implement automated deployment and monitoring through CI/CD and DevOps practices.Support production deployments, monitoring, troubleshooting and root-cause analysis for data-related issues.Salary Range: $90,000 to $115,000 per year