Results-driven Senior Data Engineer with 5+ years of experience designing and delivering production-grade, cloud-native data platforms across Financial Services and Healthcare verticals. Proven track record building end-…
Results-driven Senior Data Engineer with 5+ years of experience designing and delivering production-grade, cloud-native data platforms across Financial Services and Healthcare verticals. Proven track record building end-to-end ETL/ELT pipelines, lakehouse architectures, and real-time streaming systems on AWS and Azure using Apache Spark, Databricks, Snowflake, and Kafka. Adept at translating complex business requirements into scalable, cost-efficient data solutions with measurable impact on reporting latency, data quality, and operational efficiency.
Architected and deployed a cloud-native financial analytics platform on AWS, integrating EMR logs, S3, and third-party APIs to automate data ingestion and eliminate manual pipeline tasks, increasing data availability for reporting and QA by 30%., Engineered Star Schema data models for claims and provider analytics in Snowflake, reducing ad-hoc query runtimes by 40% and accelerating executive dashboard refresh cycles across multiple business units., Tuned complex SQL queries and Snowflake materialized views, achieving a 40% reduction in data latency and enabling near-real-time visibility into patient activity, pharmacy operations, and transaction data., Developed Python-based data validation and anomaly-detection scripts embedded in ETL pipelines, enabling early-stage detection of schema mismatches and data quality issues across source, staging, and target layers., Built real-time ingestion pipelines from external REST APIs into Amazon Redshift using AWS-native services (Kinesis, Lambda, Glue), enabling sub-minute data freshness for operational dashboards., Designed metadata-driven, reusable pipeline templates that reduced boilerplate code by ~50%, standardizing ingestion architecture across domains and accelerating team onboarding., Delivered Power BI and AWS CloudWatch monitoring dashboards to track ETL health, job runtimes, and SLA breach alerts, enabling proactive pipeline incident response., Collaborated with analysts to build Tableau dashboards on pharmacy adherence and population health trends, directly supporting clinical KPI tracking and strategic reporting., Produced field-level transformation documentation, data flow diagrams, and ETL logic specs to support audit readiness, compliance reviews, and knowledge transfer., Designed and implemented an enterprise-scale AWS data lake for credit risk scoring, fraud detection, and financial analytics, processing over 2 TB/day of structured and semi-structured financial data., Built high-throughput ETL/ELT pipelines using AWS Glue and Apache Spark, automating ingestion of daily transaction feeds and third-party risk data with file validation, control totals, and reconciliation checks., Developed real-time event-driven streaming pipelines using AWS Kinesis to capture and deliver transactional data to downstream analytics systems with low latency., Implemented Databricks Delta Lake medallion architecture (Bronze → Silver → Gold), applying deduplication logic, late-arriving record handling, and incremental MERGE strategies to ensure data accuracy during month-end financial close cycles., Designed and maintained Snowflake schemas for risk analytics and financial reporting, separating heavy transformation workloads from BI-facing marts and applying clustering on policy_id and transaction_date to improve dashboard performance., Added comprehensive data-quality controls including row-count validation, freshness monitoring, and balance reconciliation checks; automated alerting on load failures prevented data quality issues from reaching regulatory and executive dashboards., Tuned Spark workloads by optimizing partition strategies, resolving small-file issues via compaction, and scheduling compute-heavy jobs during off-peak windows, reducing cluster costs by 20% without impacting reporting SLAs., Partnered with credit risk analysts and data scientists to align data models with analytical requirements, enabling faster iteration on risk scoring models and customer behavior analysis., Developed and maintained batch data ingestion pipelines using Azure Data Factory to load policy, billing, and customer data from AWS S3 and Azure sources into ADLS Gen2, supporting enterprise reporting for millions of records daily., Built PySpark transformation jobs in Databricks to cleanse and standardize multi-source datasets, reducing downstream reconciliation issues by 20% and improving overall data quality., Integrated cross-cloud data workflows consuming S3 feeds and AWS Glue-managed schemas into Azure-based transformation and analytics pipelines, ensuring seamless interoperability., Implemented incremental loading, watermarking, and fault-handling logic within existing pipelines, reducing full data refreshes and improving daily job stability and runtime predictability., Supported near-real-time ingestion pipelines using Apache Kafka, validating event schemas and ensuring timely delivery of transactional data to downstream analytics systems., Tuned Spark workloads by adjusting partitioning, join strategies, and memory configurations, resulting in fewer execution failures and more consistent pipeline runtimes., Designed and implemented batch ETL pipelines on AWS to move structured and semi-structured data into S3 and Redshift, enabling timely data access for reporting and analytics teams., Built PySpark transformations for data cleansing, standardization, and aggregation, ensuring downstream analytics received consistent and accurate datasets., Implemented SQL-based validation and reconciliation checks between source and target tables, reducing reporting errors by 15–20% and improving data trustworthiness., Supported Redshift schema design and optimized data loading strategies using COPY commands and distribution keys, improving query performance for analytics dashboards.
The AI-first Recruitment Operating System that unifies and automates the entire hiring lifecycle for modern enterprises worldwide.
Dr Job FZ LLC. 2026 © All Rights Reserved
