Process-driven Data Engineer with 3 years of experience in designing, migrating, and optimizing scalable data pipelines...
Engineered a financial data lakehouse on AWS S3 using Apache Iceberg and Trino, enabling time-travel queries, ACID transactions, and schema evolution—reducing reporting latency by 40%.
Migrated terabytes of historical trading and order data from Hive tables to Iceberg, eliminating AWS Glue crawler dependencies and cutting ETL costs by 30%.
Designed and orchestrated data pipelines in Apache Airflow to ingest and normalize 1.5B+ records of daily market, order, and execution data from multiple OMS and client sources (Succession, VTrader, VFlux), improving data reliability by 95%.
Developed low-latency Spark Structured Streaming pipelines on Apache Kafka to process real-time trade and order events, leveraging ScyllaDB for high-throughput storage to generate Consolidated Audit Trail (CAT) and regulatory reports, cutting report turnaround time by 40%.
Collaborated with compliance and analytics teams to ensure data accuracy, lineage, and FINRA reporting standards, establishing automated monitoring for critical market feeds and regulatory submissions.
Assisted in building batch pipelines in SQL Server for financial transactions, processing 2M+ daily rows with optimized indexing strategies.
Developed Python ETL scripts to load customer & sales data into PostgreSQL, reducing manual workload by 70%.
Supported migration of workflows to Spark-based processing, cutting runtime from 2 hours to 40 minutes.
- Engineered a financial data lakehouse on AWS S3 using Apache Iceberg and Trino, enabling time-travel queries,
- Migrated terabytes of historical trading and order data from Hive tables to Iceberg, eliminating AWS Glue crawler
dependencies and cutting ETL costs by 30%.
- Designed and orchestrated data pipelines in Apache Airflow to ingest and normalize 1.5B+ records of daily
market, order, and execution data from multiple OMS and client sources (Succession, VTrader, VFlux),
improving data reliability by 95%.
- Developed low-latency Spark Structured Streaming pipelines on Apache Kafka to process real-time trade and
order events, leveraging ScyllaDB for high-throughput storage to generate Consolidated Audit Trail (CAT) and
regulatory reports, cutting report turnaround time by 40%.
- Collaborated with compliance and analytics teams to ensure data accuracy, lineage, and FINRA reporting stan-
dards, establishing automated monitoring for critical market feeds and regulatory submissions.
- Assisted in building batch pipelines in SQL Server for financial transactions, processing 2M+ daily rows with
optimized indexing strategies.
- Developed Python ETL scripts to load customer & sales data into PostgreSQL, reducing manual workload by 70%.
- Supported migration of workflows to Spark-based processing, cutting runtime from 2 hours to 40 minutes.
نظام التوظيف التشغيلي المعتمد على الذكاء الاصطناعي أولاً، الذي يوحّد ويؤتمت دورة التوظيف بالكامل للمؤسسات الحديثة حول العالم.
د.جوب منطقة حرة ذ.م.م. 2026 © جميع الحقوق محفوظة
