Senior Data Engineer with 6.5+ years architecting governed, high-performance data platforms in regulated environments (Banking, Fintech, Enterprise Analytics). Specialized in Lakehouse/Medallion architectures, cloud migr…
Senior Data Engineer with 6.5+ years architecting governed, high-performance data platforms in regulated environments (Banking, Fintech, Enterprise Analytics). Specialized in Lakehouse/Medallion architectures, cloud migrations, and metadata-driven pipelines across AWS, Azure, and GCP. Expert in Spark performance tuning (AQE, Partitioning) and proactive observability via Datadog/CloudWatch. Proven track record delivering audit-ready governance frameworks (Collibra, Purview) under strict compliance constraints. Processing 5-10 TB/day with 30% cloud cost reduction and 35% throughput improvement.
Architected and standardized a governed AWS Lakehouse (Medallion architecture) utilizing S3, Glue, and Athena to support regulatory, risk, and analytics use cases for 10+ internal teams, ensuring PII compliance and full audit readiness.
Owned end-to-end design and operation of 30+ production pipelines processing 5-10 TB/day, implementing metadata-driven ingestion patterns ensuring consistency across complex financial datasets.
Optimized Spark workloads implementing Adaptive Query Execution (AQE), Broadcast Joins, and Partitioning strategies, increasing data throughput by 35% and significantly reducing cloud compute costs.
Integrated AWS Glue Data Catalog with Collibra, establishing enterprise-wide metadata visibility and automated data lineage critical for regulatory reviews and internal audits.
Built proactive monitoring dashboards using Datadog and AWS CloudWatch for SLA tracking, reducing MTTR and increasing platform uptime by 25%.
Partnered with governance and security stakeholders to translate complex regulatory requirements into scalable, production-ready technical blueprints ensuring 100% adherence to bank-wide data standards.
Architected and standardized a governed AWS Lakehouse (Medallion architecture) utilizing S3, Glue, and Athena to support regulatory, risk, and analytics use cases for 10+ internal teams, ensuring PII compliance and full audit readiness., Owned end-to-end design and operation of 30+ production pipelines processing 5-10 TB/day, implementing metadata-driven ingestion patterns ensuring consistency across complex financial datasets., Optimized Spark workloads implementing Adaptive Query Execution (AQE), Broadcast Joins, and Partitioning strategies, increasing data throughput by 35% and significantly reducing cloud compute costs., Integrated AWS Glue Data Catalog with Collibra, establishing enterprise-wide metadata visibility and automated data lineage critical for regulatory reviews and internal audits., Built proactive monitoring dashboards using Datadog and AWS CloudWatch for SLA tracking, reducing MTTR and increasing platform uptime by 25%., Partnered with governance and security stakeholders to translate complex regulatory requirements into scalable, production-ready technical blueprints ensuring 100% adherence to bank-wide data standards.
Successfully migrated on-premise legacy Hadoop data to a GCP Lakehouse architecture utilizing BigQuery and GCS, modernizing the analytics environment for enterprise reporting and reducing infrastructure costs by 30%.
Built a metadata-driven ingestion framework using Cloud Functions and Cloud Composer (Airflow), standardizing data onboarding and reducing manual coding requirements by 50%.
Improved BigQuery query performance via Table Partitioning and Clustering, reducing execution time for high-concurrency executive dashboards and lowering slot consumption.
Developed batch and stream ELT pipelines using Dataflow (Apache Beam), ensuring reliable data delivery into the BigQuery Gold layer for research insights.
Streamlined deployments using Terraform and Cloud Build, moving the team to full Infrastructure as Code (IaC) with consistent environment parity.
Configured IAM least-privilege policies and VPC Service Controls ensuring newly migrated pipelines met baseline enterprise audit standards.
Successfully migrated on-premise legacy Hadoop data to a GCP Lakehouse architecture utilizing BigQuery and GCS, modernizing the analytics environment for enterprise reporting and reducing infrastructure costs by 30%., Built a metadata-driven ingestion framework using Cloud Functions and Cloud Composer (Airflow), standardizing data onboarding and reducing manual coding requirements by 50%., Improved BigQuery query performance via Table Partitioning and Clustering, reducing execution time for high-concurrency executive dashboards and lowering slot consumption., Developed batch and stream ELT pipelines using Dataflow (Apache Beam), ensuring reliable data delivery into the BigQuery Gold layer for research insights., Streamlined deployments using Terraform and Cloud Build, moving the team to full Infrastructure as Code (IaC) with consistent environment parity., Configured IAM least-privilege policies and VPC Service Controls ensuring newly migrated pipelines met baseline enterprise audit standards.
Developed and maintained end-to-end ELT pipelines using Azure Data Factory and Databricks, automating ingestion of high-volume financial data into ADLS Gen2.
Built robust PySpark data processing modules implementing Medallion (Bronze/Silver/Gold) architecture to ensure data quality and readiness for risk and credit analysis teams.
Optimized SQL query performance and indexing within Azure Synapse Analytics, reducing report generation time for critical fintech dashboards by 20%.
Implemented automated data validation checks within pipelines to catch PII anomalies and ensure compliance with financial data privacy regulations.
Implemented Microsoft Purview for automated data discovery and lineage, supporting compliance audits and governance initiatives.
Configured CI/CD release pipelines using Azure DevOps, transitioning team from manual deployments to automated version-controlled releases reducing deployment effort by 45%.
Developed and maintained end-to-end ELT pipelines using Azure Data Factory and Databricks, automating ingestion of high-volume financial data into ADLS Gen2., Built robust PySpark data processing modules implementing Medallion (Bronze/Silver/Gold) architecture to ensure data quality and readiness for risk and credit analysis teams., Optimized SQL query performance and indexing within Azure Synapse Analytics, reducing report generation time for critical fintech dashboards by 20%., Implemented automated data validation checks within pipelines to catch PII anomalies and ensure compliance with financial data privacy regulations., Implemented Microsoft Purview for automated data discovery and lineage, supporting compliance audits and governance initiatives., Configured CI/CD release pipelines using Azure DevOps, transitioning team from manual deployments to automated version-controlled releases reducing deployment effort by 45%.
Developed automated Power BI dashboards using SQL providing leadership with real-time insights into viewership trends and advertising performance.
Built PySpark-based ETL pipelines on Hadoop (Hive, HBase) supporting large-scale viewership and content analytics.
Designed logging and monitoring frameworks that reduced MTTR and improved production stability.
Supported early-stage migration of legacy systems to Azure Data Lake, improving maintainability and scalability.
Developed automated Power BI dashboards using SQL providing leadership with real-time insights into viewership trends and advertising performance., Built PySpark-based ETL pipelines on Hadoop (Hive, HBase) supporting large-scale viewership and content analytics., Designed logging and monitoring frameworks that reduced MTTR and improved production stability., Supported early-stage migration of legacy systems to Azure Data Lake, improving maintainability and scalability.
The AI-first Recruitment Operating System that unifies and automates the entire hiring lifecycle for modern enterprises worldwide.
Dr Job FZ LLC. 2026 © All Rights Reserved
