Associate Principal - Cloud Engineering

Job Title: DATABRICKS Architect Work LocationIrving TXJob Description:Role Overview We are seeking a highly skilled Databricks Engineer with AWS expertise to support Data Services portfolioThe role will be responsible for designing developing optimizing and supporting enterprise scale data platforms data pipelines and analytics solutions leveraging Databricks Lakehouse AWS Cloud and modern data engineering practices The candidate will work closely with data architects’ application teams business stakeholders and platform engineering teams to deliver scalable secure and high-performance data solutions Key Responsibilities Design develop and maintain scalable data pipelines using Databricks PySpark Spark SQL and Delta LakeBuild and support enterprise data lake and Lakehouse solutions on AWSDevelop batch and Realtime ingestion frameworks from multiple source systemsImplement data transformation enrichment cleansing and validation processesOptimize Spark workloads for performance scalability and cost efficiencyDesign and implement data models supporting reporting analytics and downstream applicationsIntegrate Databricks with AWS services including S3 Glue Lambda Redshift EventBridge and IAMSupport data migration and modernization initiatives from traditional data warehouses to Databricks LakehouseImplement CICD pipelines and Infrastructure as Code IaC using Terraform and Gitbased repositoriesCollaborate with security teams to ensure compliance with client governance risk and data security standardsTroubleshoot production incidents and provide L3 operational supportParticipate in architecture reviews code reviews and technical governance activitiesMandatory Technical Skills DatabricksDatabricks Lakehouse ArchitectureDatabricks WorkflowsDelta LakeUnity CatalogDatabricks SQLAuto LoaderStructured StreamingDatabricks Performance TuningMedallion ArchitectureAWSAmazon S3AWS GlueAWS LambdaAWS IAMAmazon RedshiftAWS CloudWatchSNS SQSEventBridgeKMSSecrets ManagerData EngineeringPySparkApache SparkPythonSQLData Warehousing ConceptsETLELT DesignData ModelingData Quality FrameworksBatch RealTime Processing Preferred Skills SnowflakeMicrosoft FabricGCP BigQueryKafkaAirflowdbtTerraformDockerKubernetes OpenShiftInformaticaCollibraData Vault ModelingDimensional Data ModelingBanking Domain ExperienceStrong experience within Banking and Financial ServicesUnderstanding of enterprise data governance regulatory reporting risk compliance and audit requirementsExperience working in large global organizations with stringent security and operational standards Qualifications Bachelors or Masters degree in Computer Science Engineering Information Systems or related disciplineDatabricks Certified Data Engineer PreferredAWS Certified Data Analytics or AWS Solution Architect Preferred