نبذة عني
Senior Data Engineer with nearly 8 years of experience delivering scalable, production-grade data solutions across banking,
insurance, healthcare, and financial services. Specialized in architecting end‑to‑end ETL/ELT pi…
Senior Data Engineer with nearly 8 years of experience delivering scalable, production-grade data solutions across banking,
insurance, healthcare, and financial services. Specialized in architecting end‑to‑end ETL/ELT pipelines, large-scale data
migration, and distributed data processing using PySpark, Spark SQL, Scala, Azure Databricks, Data Lake, ADF, HDFS,
and MongoDB. Proven expertise in modernizing legacy systems through Oracle‑to‑MongoDB migration, schema
transformation, and reconciliation frameworks. Adept at integrating complex data sources, optimizing compute costs, and
enabling analytics teams with reliable, high-quality datasets. Recognized for a structured engineering approach, strong
ownership, and the ability to lead data initiatives from design to production.
الخبرة
Data Engineer - Data Migration
Led the end-to-end migration of legacy Oracle datasets to MongoDB for the BusinessOnline corporate banking platform, modernizing critical data flows.
Designed PySpark pipelines using HDFS staging + Hive curated layers to transform relational schemas into nested MongoDB documents based on business mapping specifications.
Built a multi-stage reconciliation framework (Oracle → Hive staging → Hive curated → MongoDB) ensuring data completeness, accuracy, and rule adherence.
Delivered automated pre- and post-migration business reports with pass/ fail counts, exception summaries, and validation insights for business sign-off and audit compliance.
Optimized multiple critical PySpark migration job from 1 hour 40 minutes down to under 5 minutes by redesigning transformations, improving partitioning strategy, and eliminating unnecessary shuffles — resulting in massive performance gains and reduced compute cost.
Collaborated with solution architects, DBAs, and MongoDB specialists to define document modeling standards, indexing strategies, and performance guidelines.
Implemented data validation, anomaly detection, and exception handling to ensure zero‑loss migration for high-value corporate banking data.
Optimized PySpark jobs and HDFS storage strategies (partitioning, compression, file formats), improving throughput and reducing cluster load.
Supported business UAT cycles by providing traceability reports, reconciliation summaries, and defect analysis, ensuring smooth migration readiness.
Played a key role in establishing migration best practices, reusable PySpark templates, and documentation for future modernization projects.
Data Engineer - Data Migration (Contract)
Led the end-to-end migration of legacy Oracle datasets to MongoDB for the BusinessOnline corporate banking platform, modernizing critical data flows., Designed PySpark pipelines using HDFS staging + Hive curated layers to transform relational schemas into nested MongoDB documents based on business mapping specifications., Built a multi-stage reconciliation framework (Oracle → Hive staging → Hive curated → MongoDB) ensuring data completeness, accuracy, and rule adherence., Delivered automated pre- and post-migration business reports with pass/ fail counts, exception summaries, and validation insights for business sign-off and audit compliance., Optimized multiple critical PySpark migration job from 1 hour 40 minutes down to under 5 minutes by redesigning transformations, improving partitioning strategy, and eliminating unnecessary shuffles — resulting in massive performance gains and reduced compute cost., Collaborated with solution architects, DBAs, and MongoDB specialists to define document modeling standards, indexing strategies, and performance guidelines., Implemented data validation, anomaly detection, and exception handling to ensure zero‑loss migration for high-value corporate banking data., Optimized PySpark jobs and HDFS storage strategies (partitioning, compression, file formats), improving throughput and reducing cluster load., Supported business UAT cycles by providing traceability reports, reconciliation summaries, and defect analysis, ensuring smooth migration readiness., Played a key role in establishing migration best practices, reusable PySpark templates, and documentation for future modernization projects.
Senior Data Engineer - Analytics
Architected and delivered enterprise-grade ETL/ ELT pipelines in Azure Databricks using PySpark, Spark SQL, and Scala, powering analytics for 12+ high‑visibility dashboards.
Engineered a unified ingestion framework integrating Firebase, Couchbase, DPAS, Google Console, Postgres, and Apple Store into a consistent analytics-ready model.
Designed KPI computation layers for funnels, cohorts, retention, churn, DAU/MAU, and user journeys, enabling data-driven product decisions.
Worked closely with the Chief Data Officer to define KPI frameworks, align engineering outputs with business strategy, and deliver insight-ready datasets for executive reporting.
Identified and analyzed fake app registration patterns, preventing revenue leakage and saving the client over 100,000$ SGD through anomaly detection and data-driven validation.
Built modular, parameterized Databricks workflows supporting incremental loads, schema evolution, and reusable transformation logic.
Optimized Spark workloads through partitioning, caching, broadcast joins, and cluster tuning—achieving significant reductions in compute cost and runtime.
Implemented robust data quality checks, validation rules, and anomaly detection to ensure reliability of analytics datasets.
Managed deployments across DEV/UAT/PROD, establishing stable job orchestration, monitoring, and alerting.
Mentored junior engineers on Spark best practices, Databricks development standards, and scalable data engineering patterns.
Senior Data Engineer - Analytics
Architected and delivered enterprise-grade ETL/ ELT pipelines in Azure Databricks using PySpark, Spark SQL, and Scala, powering analytics for 12+ high‑visibility dashboards., Engineered a unified ingestion framework integrating Firebase, Couchbase, DPAS, Google Console, Postgres, and Apple Store into a consistent analytics-ready model., Designed KPI computation layers for funnels, cohorts, retention, churn, DAU/MAU, and user journeys, enabling data-driven product decisions., Worked closely with the Chief Data Officer to define KPI frameworks, align engineering outputs with business strategy, and deliver insight-ready datasets for executive reporting., Identified and analyzed fake app registration patterns, preventing revenue leakage and saving the client over 100,000$ SGD through anomaly detection and data-driven validation., Built modular, parameterized Databricks workflows supporting incremental loads, schema evolution, and reusable transformation logic., Optimized Spark workloads through partitioning, caching, broadcast joins, and cluster tuning—achieving significant reductions in compute cost and runtime., Implemented robust data quality checks, validation rules, and anomaly detection to ensure reliability of analytics datasets., Managed deployments across DEV/UAT/PROD, establishing stable job orchestration, monitoring, and alerting., Mentored junior engineers on Spark best practices, Databricks development standards, and scalable data engineering patterns.
Data Engineer - BIU
Developed high-performance data pipelines using Spark DataFrame API, implementing analytical and window functions for large-scale banking datasets.
Translated complex SQL logic into optimized PySpark/Scala workflows within Databricks, improving maintainability and performance.
Led SQL and Spark performance tuning initiatives, improving execution efficiency and reducing processing time across critical BI workloads.
Delivered business-aligned analytical datasets and reports for risk, operations, credit, and customer analytics teams.
Designed and maintained traditional database components (DDL, DML, stored procedures, triggers, indexing strategies).
Collaborated with business teams to convert ambiguous requirements into clear data models, ensuring accurate KPI and metric definitions.
Implemented data quality checks, validation rules, and reconciliation logic to ensure accuracy of regulatory and operational reports.
Automated recurring data processes, reducing manual effort and improving reliability of daily/ weekly reporting cycles.
Data Engineer - BIU
Developed high-performance data pipelines using Spark DataFrame API, implementing analytical and window functions for large-scale banking datasets., Translated complex SQL logic into optimized PySpark/Scala workflows within Databricks, improving maintainability and performance., Led SQL and Spark performance tuning initiatives, improving execution efficiency and reducing processing time across critical BI workloads., Delivered business-aligned analytical datasets and reports for risk, operations, credit, and customer analytics teams., Designed and maintained traditional database components (DDL, DML, stored procedures, triggers, indexing strategies)., Collaborated with business teams to convert ambiguous requirements into clear data models, ensuring accurate KPI and metric definitions., Implemented data quality checks, validation rules, and reconciliation logic to ensure accuracy of regulatory and operational reports., Automated recurring data processes, reducing manual effort and improving reliability of daily/ weekly reporting cycles.
المشاريع
Migration
Led the end-to-end migration of legacy Oracle datasets to MongoDB for theBusinessOnline corporate banking platform, modernizing critical data flows.• Designed PySpark pipelines using HDFS staging + Hive curated layers to transformrelational schemas into nested MongoDB documents based on business mappingspecifications.• Built a multi-stage reconciliation framework (Oracle → Hive staging → Hive curated →MongoDB) ensuring data completeness, accuracy, and rule adherence.• Delivered automated pre- and post-migration business reports with pass/ fail counts,exception summaries, and validation insights for business sign-off and audit compliance.• Optimized multiple critical PySpark migration job from 1 hour 40 minutes down tounder 5 minutes by redesigning transformations, improving partitioning strategy, andeliminating unnecessary shuffles — resulting in massive performance gains and reducedcompute cost.• Collaborated with solution architects, DBAs, and MongoDB specialists to definedocument modeling standards, indexing strategies, and performance guidelines.• Implemented data validation, anomaly detection, and exception handling to ensurezero‑loss migration for high-value corporate banking data.• Optimized PySpark jobs and HDFS storage strategies (partitioning, compression, fileformats), improving throughput and reducing cluster load.• Supported business UAT cycles by providing traceability reports, reconciliationsummaries, and defect analysis, ensuring smooth migration readiness.• Played a key role in establishing migration best pra