نبذة عني
Having 5+ years of IT experience in Azure Data Engineer, specializing in Azure Data Factory (ADF), Azure Data Lake Storage, Databricks, Synapse Analytics, PySpark, and Spark SQL, delivering end-to-end data solutions in c…
Having 5+ years of IT experience in Azure Data Engineer, specializing in Azure Data Factory (ADF), Azure Data Lake Storage, Databricks, Synapse Analytics, PySpark, and Spark SQL, delivering end-to-end data solutions in cloud environments. Having hands-on experience as a SQL Developer, working on database design, performance optimization, and complex query development across platforms like SQL Server, Azure SQL, and SSIS. Good hands-on experience in creating Pipelines in ADF using Linked Services, Datasets, Pipeline and Dataflow to Extract, Transform and load data from different sources like Azure SQL, Blob storage. Extensive experience in Extract Transform and Load data from Source Systems to Azure Data Storage services using a combination of Azure Data Factory, Spark SQL Data Ingestion to one or more Azure Services - (Azure Data Lake, Azure Storage, Azure SQL). Used Control Flow Tasks like For Each Activity, Switch Activity, Get Metadata Activity, If Activity, Lookup Activity and also Copy Activity Web Activity, and Execute Pipeline Activity. Experience in Dataflow Components, Control Flow Elements, Connection Managers, and Triggers. Integrated Git Repository to automate and improve CI/CD process. Good experience in Data extraction (extract, Schemas, corrupt record handling), transformations and loads (user - defined functions, join optimizations) and Production (optimize and automate Extract, Transform and Load). Good hands-on experience in Databricks by applying transformations using PySpark and Spark SQL Azure Data Factory (ADF), Integration Run Time (IR), File System Data Ingestion, Relational Data Ingestion Experience in programming languages such as Python, with Databricks PySpark, designing and building data engineering workloads to leverage cloud computing and frameworks for the building of data pipelines.
الخبرة
Azure Data Engineer
Designed and developed ETL/ELT pipelines to ingest structured, semi-structured, and unstructured data from multiple on-premises and cloud sources into Azure Data Lake and Azure Synapse.
Implemented data workflows, triggers, and parameterized pipelines for dynamic data movement.
Integrated ADF with Key Vault, Event Grid, and Logic Apps for secure and event-driven orchestration.
Developed big data processing solutions using Databricks with PySpark for data transformation, cleansing, and enrichment.
Implemented Delta Lake for handling slowly changing dimensions (SCD) and maintaining data quality.
Optimized Spark jobs for scalability, partitioning, caching, and performance tuning.
Integrated Databricks with ADF, Synapse, and Power BI for end-to-end analytics workflows.
Designed and developed data models, fact/dimension tables, and star schema for reporting and analytics.
Created and optimized SQL queries, stored procedures, and views for performance improvement.
Developed scalable ETL pipelines in PySpark for processing large volumes of structured and unstructured data.
Applied data cleansing, deduplication, aggregations, and transformations using PySpark DataFrames and RDDs.
Designed and maintained relational databases, schemas, and tables for transactional and analytical workloads.
Developed complex SQL queries, stored procedures, functions, triggers, and views for business requirements.
Implemented data migration and integration between SQL Server and Azure platforms (ADF/Databricks).
Performed query optimization and indexing strategies for better performance.
Worked on SQL Server Integration Services (SSIS) for legacy ETL solutions and migration to ADF.
Developed and managed end-to-end data solutions using Microsoft Fabric, including Data Factory, Lakehouse, Data Warehouse, and Power BI integration for enterprise analytics and reporting.
Designed and optimized Snowflake data warehouse architectures, implementing scalable ELT pipelines, performance tuning, and secure data sharing for business-critical analytics workloads.
Azure Data Engineer
Designed and developed ETL/ELT pipelines to ingest structured, semi-structured, and unstructured data from multiple on-premises and cloud sources into Azure Data Lake and Azure Synapse., Implemented data workflows, triggers, and parameterized pipelines for dynamic data movement., Integrated ADF with Key Vault, Event Grid, and Logic Apps for secure and event-driven orchestration., Developed big data processing solutions using Databricks with PySpark for data transformation, cleansing, and enrichment., Implemented Delta Lake for handling slowly changing dimensions (SCD) and maintaining data quality., Optimized Spark jobs for scalability, partitioning, caching, and performance tuning., Integrated Databricks with ADF, Synapse, and Power BI for end-to-end analytics workflows., Designed and developed data models, fact/dimension tables, and star schema for reporting and analytics., Created and optimized SQL queries, stored procedures, and views for performance improvement., Developed scalable ETL pipelines in PySpark for processing large volumes of structured and unstructured data., Applied data cleansing, deduplication, aggregations, and transformations using PySpark DataFrames and RDDs., Designed and maintained relational databases, schemas, and tables for transactional and analytical workloads., Developed complex SQL queries, stored procedures, functions, triggers, and views for business requirements., Implemented data migration and integration between SQL Server and Azure platforms (ADF/Databricks)., Performed query optimization and indexing strategies for better performance., Worked on SQL Server Integration Services (SSIS) for legacy ETL solutions and migration to ADF., Developed and managed end-to-end data solutions using Microsoft Fabric, including Data Factory, Lakehouse, Data Warehouse, and Power BI integration for enterprise analytics and reporting., Designed and optimized Snowflake data warehouse architectures, implementing scalable ELT pipelines, performance tuning, and secure data sharing for business-critical analytics workloads.
Software Engineer
Designing Data Pipeline job in ADF.
Worked on reading and writing multiple data formats like JSON, Parquet etc. using PySpark.
Designing Azure Data Pipeline job to extract data through xml, Azure blob storage, Flat File source.
Using JIRA tools and Git for committing code.
Implementing Test Driven Development.
Creating and maintaining a scalable data pipeline and building out new integration.
Developing API and integrated with data streaming job, deploying them on pivoted cloud foundry (PCF) on scaling mode.
Create and maintain optimal Azure data pipeline architecture and Optimization of PySpark code in Azure Databricks using best practices and right parameters.
Created different Hive RAW and Standardize table for data validation and Analysis with Partition and bucket.
Helping team for solving critical issues, also optimization of spark code.
Responsible for creating a data streaming pipeline.
Developing Proof of Concept and implementing in a project to get the desired result.
Writing Scripting for interacting with an internal server with Azure Cloud Storage for migrating the data.
Interacting with clients and understanding the requirement and developing the code.
Creating Architecture flow diagram for Project.
Responsible for delivery support and any code related issue.
Software Engineer
Designing Data Pipeline job in ADF. Worked on reading and writing multiple data formats like JSON, Parquet etc. using PySpark., Designing Azure Data Pipeline job to extract data through xml, Azure blob storage, Flat File source., Using JIRA tools and Git for committing code., Implementing Test Driven Development., Creating and maintaining a scalable data pipeline and building out new integration., Developing API and integrated with data streaming job, deploying them on pivoted cloud foundry (PCF) on scaling mode., Create and maintain optimal Azure data pipeline architecture and Optimization of PySpark code in Azure Databricks using best practices and right parameters., Created different Hive RAW and Standardize table for data validation and Analysis with Partition and bucket., Helping team for solving critical issues, also optimization of spark code., Responsible for creating a data streaming pipeline., Developing Proof of Concept and implementing in a project to get the desired result., Writing Scripting for interacting with an internal server with Azure Cloud Storage for migrating the data., Interacting with clients and understanding the requirement and developing the code., Creating Architecture flow diagram for Project., Responsible for delivery support and any code related issue.
Software Engineer
Developed and maintained Azure Data Factory (ADF) pipelines to ingest policy, claims, agent, and customer data from multiple internal and third-party insurance systems.
Designed SQL Server tables, views, stored procedures, and performance-optimized queries to support actuarial, underwriting, and claims analytics.
Built automated ADF ETL/ELT workflows to process daily, weekly, and monthly insurance datasets with high data accuracy and consistency.
Integrated structured and semi-structured data from policy administration systems, claims systems, and agent performance platforms into centralized SQL databases.
Implemented data validation, cleansing, error-handling, and audit frameworks to ensure high-quality insurance data for reporting.
Created interactive Power BI dashboards for claims trends, policy renewals, customer behavior, premium analytics, and agent performance insights.
Designed data models and DAX measures in Power BI to support analytics requirements for Aflac’s business teams.
Performed query tuning, indexing, and optimization on SQL Server to support high-volume insurance data processing.
Collaborated with actuarial, underwriting, compliance, and business stakeholders to gather requirements and translate them into scalable data solutions.
Ensured PHI/PII data security by applying RBAC, encryption, ADF Managed Identity, and Aflac’s data governance policies.
Automated refresh schedules for Power BI reports and ADF pipelines to support real-time and near-real-time reporting needs.
Documented pipeline logic, SQL transformations, data mapping sheets, and dashboard specifications for operational support and cross-team collaboration.
Software Engineer
Developed and maintained Azure Data Factory (ADF) pipelines to ingest policy, claims, agent, and customer data from multiple internal and third-party insurance systems., Designed SQL Server tables, views, stored procedures, and performance-optimized queries to support actuarial, underwriting, and claims analytics., Built automated ADF ETL/ELT workflows to process daily, weekly, and monthly insurance datasets with high data accuracy and consistency., Integrated structured and semi-structured data from policy administration systems, claims systems, and agent performance platforms into centralized SQL databases., Implemented data validation, cleansing, error-handling, and audit frameworks to ensure high-quality insurance data for reporting., Created interactive Power BI dashboards for claims trends, policy renewals, customer behavior, premium analytics, and agent performance insights., Designed data models and DAX measures in Power BI to support analytics requirements for Aflac’s business teams., Performed query tuning, indexing, and optimization on SQL Server to support high-volume insurance data processing., Collaborated with actuarial, underwriting, compliance, and business stakeholders to gather requirements and translate them into scalable data solutions., Ensured PHI/PII data security by applying RBAC, encryption, ADF Managed Identity, and Aflac’s data governance policies., Automated refresh schedules for Power BI reports and ADF pipelines to support real-time and near-real-time reporting needs., Documented pipeline logic, SQL transformations, data mapping sheets, and dashboard specifications for operational support and cross-team collaboration.
Software Engineer
Designed and developed complex T-SQL stored procedures, functions, and queries to process sales, inventory, product, and customer data across multiple H&M retail channels.
Built SSIS ETL packages to extract data from POS systems, e-commerce portals, vendor feeds, ERP systems, and warehouse management systems into SQL Server.
Created and maintained SSRS reports for sales performance, stock availability, replenishment cycles, customer trends, and store operational insights.
Performed query optimization, indexing, and performance tuning to ensure efficient processing of large retail datasets with high transaction volumes.
Developed automated ETL workflows to support daily, weekly, and monthly reporting cycles for merchandising, finance, supply chain, and marketing teams.
Designed and maintained data models, including fact and dimension tables, for analytics and reporting needs such as product hierarchy and customer segmentation.
Integrated data from multiple global H&M systems to support unified reporting and consistent business insights across regions.
Collaborated with store operations, merchandising, and BI teams to gather requirements and convert them into efficient SQL-based solutions.
Ensured data governance, security, and compliance by implementing role-based access, encryption, and regular data integrity audits.
Supported production environments by conducting root cause analysis, troubleshooting ETL failures, and ensuring timely data availability.
Documented ETL processes, SQL logic, report structures, data flow diagrams, and operational procedures for team collaboration and future maintenance.
Software Engineer
Designed and developed complex T-SQL stored procedures, functions, and queries to process sales, inventory, product, and customer data across multiple H&M retail channels., Built SSIS ETL packages to extract data from POS systems, e-commerce portals, vendor feeds, ERP systems, and warehouse management systems into SQL Server., Created and maintained SSRS reports for sales performance, stock availability, replenishment cycles, customer trends, and store operational insights., Performed query optimization, indexing, and performance tuning to ensure efficient processing of large retail datasets with high transaction volumes., Developed automated ETL workflows to support daily, weekly, and monthly reporting cycles for merchandising, finance, supply chain, and marketing teams., Designed and maintained data models, including fact and dimension tables, for analytics and reporting needs such as product hierarchy and customer segmentation., Integrated data from multiple global H&M systems to support unified reporting and consistent business insights across regions., Collaborated with store operations, merchandising, and BI teams to gather requirements and convert them into efficient SQL-based solutions., Ensured data governance, security, and compliance by implementing role-based access, encryption, and regular data integrity audits., Supported production environments by conducting root cause analysis, troubleshooting ETL failures, and ensuring timely data availability., Documented ETL processes, SQL logic, report structures, data flow diagrams, and operational procedures for team collaboration and future maintenance.