Diligent and results-oriented Data Engineer with a master’s in computer science and 4 years of professional experience in data engineering. Proficient in diverse programming languages including SQL, Python, PySpark. Hand…
Diligent and results-oriented Data Engineer with a master’s in computer science and 4 years of professional experience in data engineering. Proficient in diverse programming languages including SQL, Python, PySpark. Hands-on experience with Azure Data Factory, Azure Data Pipelines, Power BI, and MongoDB. Skilled in designing and executing end-to-end ETL workflows, contributing significantly to the success of data-driven projects. Experienced utilizing Power BI to create interactive dashboards, visualizations, and reports. Proficient in ERD data models creation, SQL query performance tuning using tools like DB Visualizer Designed and developed cloud migration solutions for migrating On-Premises databases and ETL using Azure migration tools. Developed and optimized CI/CD DevOps pipelines using Azure DevOps, and other related tools like GitHub, and Unix Shell Scripts. Built scalable Data pipelines leveraging AWS serverless computing services like Lambda, Glue, SNS etc. Demonstrated expertise in utilizing various AWS services, including S3, API Gateway, IAM, CloudWatch, SES, and DynamoDB for comprehensive and secure application implementations. Adept with Agile/Scrum, and SDLC methodologies.
● Created Databricks notebooks, and Delta Tables following Lakehouse architecture: Bronze, Silver, and Gold.
● Orchestrated ETL data pipelines using Azure Data Factory, Data Flows, and Databricks notebooks to extract, transform, and load data from various sources into a centralized data lake/delta lake.
● Disparate data sources in various formats like Text, CSV, JSON, Parquet, and EDW were ingested into Data Lake/ Delta Lake (ADLS) hosted in Azure Cloud using Databricks ETL processes/notebooks written in SQL and Python utilizing Spark Core and Spark SQL libraries.
● Developed CI/CD pipelines using Azure DevOps and Shell Scripts for deploying the artifacts to various environments: test, stage, and production.
● Worked closely with data scientists to understand data requirements and implemented solutions for advanced analytics and machine learning.
● Leveraged JIRA for Scrum, GitHub for Source Control, and Azure DevOps for (CI/CD).
● Contributed to the documentation (Confluence) of data engineering processes, ensuring knowledge transfer and team alignment.
Created Databricks notebooks and Delta Tables following Lakehouse architecture: Bronze, Silver, and Gold.
Orchestrated ETL data pipelines using Azure Data Factory, Data Flows, and Databricks notebooks to extract, transform, and load data from various sources into a centralized data lake/delta lake.
Ingested disparate data sources in formats like Text, CSV, JSON, Parquet, and EDW into Data Lake/Delta Lake (ADLS) hosted in Azure Cloud using Databricks ETL processes/notebooks written in SQL and Python utilizing Spark Core and Spark SQL libraries.
Developed CI/CD pipelines using Azure DevOps and Shell Scripts for deploying artifacts to test, stage, and production environments.
Worked closely with data scientists to understand data requirements and implemented solutions for advanced analytics and machine learning.
Leveraged JIRA for Scrum, GitHub for source control, and Azure DevOps for CI/CD.
Contributed to documentation (Confluence) of data engineering processes, ensuring knowledge transfer and team alignment.
Designed and implemented a real-time data pipeline to process semi-structured data by integrating raw records from various data sources to Azure Synapse Analytics.
Ingested streaming data into Azure Synapse Analytics from Event Hubs and IoT Hub.
Used Synapse Analytics capabilities to perform transformations and aggregations on the data.
Implemented and actively monitored programs to ensure precision and efficacy.
Implemented alerts and notifications for timely response to failures.
Developed data pipelines using Azure Data Factory to extract, transform, and load diverse datasets into Azure Data Lake Storage and SQL DW/Synapse Analytics for visualization purposes.
Built Data Flows in Azure Data Factory for ETL by applying complex data transformations and writing to Azure SQL DW using activities such as Source, Lookup, filters, aggregations, updates, pivots, and Sinks.
Used linked services such as ADLS, Azure SQL Database, Azure Blob Storage, Key Vaults, Azure SQL Data Warehouse, and API Service.
Automated pipelines using scheduled event-based tumbling window triggers in Azure Data Factory.
Engineered end-to-end ETL workflows, contributing significantly to the success of data-driven projects.
Utilized Power BI for creating interactive dashboards and transforming raw data into visually compelling insights.
Performed a POC on migration of an app hosted on Azure App Service to Kubernetes.
Implemented a Deployment (Stateless Set) for the frontend app and web API using Docker file.
Used a Stateful Set for the database, ensuring stable identities and effective management of persistent data.
Built basic ETL that ingested transactional and event data from a web app to Azure Data Warehouse.
Worked with clients to understand business needs and translate those business needs into actionable reports in Tableau.
Used Python in Spark to distribute data processing on large datasets in Azure Data Factory using HDInsight.
Supported implementation and active monitoring of pipelines if failures occurred.
The AI-first Recruitment Operating System that unifies and automates the entire hiring lifecycle for modern enterprises worldwide.
Dr Job FZ LLC. 2026 © All Rights Reserved
