نبذة عني
Skilled Azure Data Engineer with around 3 years of expertise in designing, developing, and maintaining data solutions along with Azure data services, tools, and technologies. Versatile in leveraging Azure Databricks for …
Skilled Azure Data Engineer with around 3 years of expertise in designing, developing, and maintaining data solutions along with Azure data services, tools, and technologies. Versatile in leveraging Azure Databricks for big data analytics, data engineering, and machine learning, utilizing PySpark for data processing and transformation. Experienced in building ETL pipelines using Azure Data Factory, working with Azure Synapse Analytics, Azure SQL Database, Azure Cosmos DB, Azure Data Lake Storage, Azure Blob Storage, Azure Stream Analytics, Azure Monitor, Python, PowerShell, SQL, Power BI, Jira, and Confluence.
الخبرة
Azure Data Engineer
Designed, developed, and maintained data pipelines using Azure Data Factory or Azure Data Pipelines to move and transform data from various sources to target destinations
Performed ETL (Extract, Transform, Load) processes using tools like Azure DataBricks, PySpark, and SQL for cleaning, transformed the data up to 98%, and prepared data for analytical purposes
Integrated data from diverse sources such as Azure SQL Database, Azure Cosmos DB, Azure Data Lake Storage, and Azure Blob Storage into a unified data platform
Managed Azure data warehousing solutions like Azure Synapse Analytics (formerly SQL Data Warehouse) for storing and querying large datasets
Monitored and optimized the performance of data pipelines, databases, and data processing workflows to ensure efficient data processing up to 95%
Collected real-time data processing using Azure Stream Analytics for handling streaming data sources.
Created and maintained data visualizations and dashboards using tools like Power BI for data reporting and analysis
Used Python, PowerShell, or other scripting languages to automate data-related tasks and workflows
Collaborated with data scientists, analysts, and other stakeholders to understand data requirements and delivered data solutions
Set up monitoring and alerting systems using Azure Monitor to proactively identify and address issues in data pipelines and infrastructure
Azure Data Engineer
Designed, developed, and maintained data pipelines using Azure Data Factory or Azure Data Pipelines to move and transform data from various sources to target destinations
Performed ETL processes using tools like Azure DataBricks, PySpark, and SQL for cleaning, transformed the data up to 98%, and prepared data for analytical purposes
Integrated data from diverse sources such as Azure SQL Database, Azure Cosmos DB, Azure Data Lake Storage, and Azure Blob Storage into a unified data platform
Managed Azure data warehousing solutions like Azure Synapse Analytics (formerly SQL Data Warehouse) for storing and querying large datasets
Monitored and optimized the performance of data pipelines, databases, and data processing workflows to ensure efficient data processing up to 95%
Collected real-time data processing using Azure Stream Analytics for handling streaming data sources
Created and maintained data visualizations and dashboards using tools like Power BI for data reporting and analysis
Used Python, PowerShell, or other scripting languages to automate data-related tasks and workflows
Collaborated with data scientists, analysts, and other stakeholders to understand data requirements and delivered data solutions
Set up monitoring and alerting systems using Azure Monitor to proactively identify and address issues in data pipelines and infrastructure
ETL Developer
Assured that data is cleansed, mapped, transformed, and otherwise optimized for storage and use according to business and technical requirements
Developed and maintained stored procedures, views, and functions in SQL server to optimize data extract, transform, and load (ETL) processes by 35%
Knowledge in Big Data, architecture, and engineering including data storage and management, distributed processing, data pipelines, data mining, analyses, visualization, data integration, and high-performance algorithms
Created and maintained data pipelines, which are the systems that move data from its source to its destination
Able to automate tasks and deploy production standard code with unit testing, continuous integration, versioning
Loaded transformed data into storage and reporting structures in destinations including data warehouses, high-speed indexes, real-time reporting systems, and analytics applications
Built data pipelines to collectively bring together data and other responsibilities include extracting data, troubleshooting, and maintaining the data warehouse
Creative in building and optimizing big data data pipelines, architectures, and data sets
Generated maintained and analyzed dashboards, reports, and trends, minimizing customer pain points by 20%
Confident and concise communication skills, with the ability to drive alignment, collaboration, and efficiency within teams