Senior Data Engineer (PySpark)

Job Title: Data Engineer (PySpark)

We are seeking a highly skilled Senior Data Engineer with strong expertise in PySpark and data modelling to join our Digital Platforms delivery team. The ideal candidate will design, build, and optimize scalable data pipelines and contribute to enterprise-grade data architecture supporting analytics, reporting, and digital banking initiatives.

Key Responsibilities

• Design, develop, and maintain scalable data pipelines using PySpark and distributed data processing frameworks
• Build and optimize ETL/ELT workflows for large-scale structured and unstructured datasets
• Implement robust data models (dimensional, relational, and data vault) aligned with business requirements
• Work closely with stakeholders, data analysts, and business teams to translate requirements into technical solutions
• Ensure data quality, governance, and consistency across multiple data sources
• Optimize performance of Spark jobs and handle large datasets efficiently
• Collaborate with DevOps teams for deployment, CI/CD, and monitoring of data pipelines
• Troubleshoot data issues and ensure high availability and reliability of data systems
• Contribute to data platform architecture and best practices within the Data Engineering chapter

Required Skills & Experience

• 5–8+ years of experience in data engineering or related roles
• Strong hands-on experience with PySpark / Apache Spark
• Expertise in data modelling (Star Schema, Snowflake, Data Vault)
• Experience with big data ecosystems (Hadoop, Hive, Spark, Kafka – preferred)
• Proficiency in Python and SQL
• Experience with cloud platforms (AWS / Azure / GCP – any one preferred)
• Strong understanding of ETL frameworks and data warehousing concepts
• Experience working in agile environments

Good to Have

• Experience in banking or financial services domain
• Knowledge of real-time data processing / streaming (Kafka, Spark Streaming)
• Exposure to data governance and security practices
• Familiarity with tools like Airflow, Databricks, or similar platforms