Data Engineer

Key Responsibilities Data Engineering & Integration• Design, build, and optimize ETL/ELT pipelines using Apache Spark, PySpark, Databricks, Azure Synapse, or equivalent platforms. • Develop scalable batch and real-time data processing solutions. • Integrate data from Core Banking, Payments, Treasury, Trade Finance, CRM, Compliance, and Risk systems. • Develop and maintain enterprise data models including 3NF, Dimensional Modeling, and Data Vault 2.0. Streaming & Modern Data Platforms• Build and operationalize real-time streaming pipelines using Kafka, Confluent, or Azure Event Hubs. • Support data platform modernization initiatives, including migration from legacy platforms (e.g., Teradata, DB2) to cloud-native environments such as Snowflake, Databricks, or Azure Synapse. • Implement scalable cloud-based data lake and data warehouse architectures. Data Quality & Governance• Implement data quality, validation, lineage, and observability frameworks using tools such as Great Expectations, Deequ, or dbt. • Collaborate with Governance and Security teams to ensure compliance with enterprise data standards. • Support metadata management, cataloging, and lineage initiatives using Azure Purview, Apache Atlas, or Collibra. Regulatory & Compliance Support• Support regulatory reporting and risk data flows including:MAS 610MAS 649Basel III / Basel IVIFRS 9 / IFRS 17BCBS 239 • Ensure data security controls including encryption, tokenization, masking, RBAC, and audit logging are implemented. DevOps & MLOps• Develop CI/CD pipelines using Azure DevOps, GitHub Actions, or Terraform. • Collaborate with Data Scientists and AI teams to deploy ML feature stores and model-serving pipelines. • Support automation and Infrastructure-as-Code (IaC) initiatives. Required Technical SkillsProgramming Languages• Python • PySpark • SQL • Scala Data Platforms• Azure Data Lake • Azure Synapse Analytics • Databricks • Snowflake Data Orchestration• Apache Airflow • Azure Data Factory (ADF) • dbt Streaming Technologies• Apache Kafka • Confluent Platform • Azure Event Hubs Data Governance• Azure Purview • Apache Atlas • Collibra Security & Compliance• Encryption • Tokenization • Role-Based Access Control (RBAC) • Audit Logging DevOps & Infrastructure• Terraform • Azure DevOps • GitHub Actions Qualifications• Bachelor's or Master's Degree in Computer Science, Data Engineering, Information Technology, or related discipline. • 6–10 years of Data Engineering experience. • Minimum 3 years of experience within Banking, Financial Services, Insurance, or Capital Markets environments. • Strong experience designing and implementing cloud-based data platforms on Azure and/or AWS. • Hands-on experience with batch and real-time data processing frameworks. • Understanding of regulatory reporting and risk data management frameworks.