نبذة عني
Detail-oriented and results-driven Data Engineer with 3 years of experience in designing, building, and maintaining scalable data pipelines and ETL processes. Proficient in using Azure Data Factory (ADF), PySpark, Python…
Detail-oriented and results-driven Data Engineer with 3 years of experience in designing, building, and maintaining scalable data pipelines and ETL processes. Proficient in using Azure Data Factory (ADF), PySpark, Python, and Databricks for large-scale data processing. Strong expertise in writing optimized SQL queries and managing data storage solutions using Azure Data Lake Storage (ADLS). Adept at working in cloud-based big data environments, ensuring data accuracy, integrity, and performance across the pipeline lifecycle.
الخبرة
System Engineer
Developed code in Azure Databricks using PySpark to ingest and transform data, Designed, developed and maintained data pipelines on Azure Data Factory to ingest, transform and load data from various sources into Azure Data Lake Storage., Developed streamline data ingestion to Azure Data Lake using Auto loader, Bug fixes and Performance Improvements, Worked with CI/CD pipelines in Azure DevOps, Handled the Security Deployment Lead role at the Relationship Level
System Engineer
Engineered code in Azure Databricks utilizing PySpark for efficient data ingestion and transformation.
Orchestrated the design, development, and maintenance of robust data pipelines in Azure Data Factory to seamlessly ingest, transform, and load data from diverse sources into Azure Data Lake Storage.
Configured Linked Services, Datasets, and Triggers to automate data movement and transformation at scale.
Revamped data ingestion processes to Azure Data Lake by implementing Auto Loader, enhancing efficiency.
Instituted bug fixes and performance optimizations to improve system reliability and speed.
Facilitated the operations of CI/CD pipelines within Azure DevOps to ensure streamlined deployment workflows.
Championed the Security Deployment Lead role at the Relationship Level, overseeing security measures and compliance.