نبذة عني
Experienced Data Engineer skilled in creating efficient ETL/ELT data pipelines using AWS and GCP. Specialized in leading large-scale data projects, emphasizing performance and cost optimization which involves building, m…
Experienced Data Engineer skilled in creating efficient ETL/ELT data pipelines using AWS and GCP. Specialized in leading large-scale data projects, emphasizing performance and cost optimization which involves building, maintaining and automating data pipelines to serve various business needs. Proficient in designing and managing data architecture, applying data modeling for various organizations' data lakes and warehouses. Expertise includes AWS, GCP, Python, Redshift, BigQuery, SQL, ETL/ELT, Airflow, Terraform, DBT, Data Structures, and Reporting/BI tools.
الخبرة
Data Engineer
• Developed and implemented data-driven solutions leveraging Airflow, BigQuery, Python
and DBT to analyze marketing data, providing key insights for retention strategies.
• Led and built a global service platform to centralize attributes from diverse teams,
streamlining data integration processes and significantly improving operational
efficiency, resulting in a remarkable 50% reduction in onboarding time.
• Designed and implemented a geo-location-based mapping solution for users and
vendors, optimizing targeting accuracy and enhancing customer outreach.
• Led cloud cost control initiatives by identifying and implementing strategies to optimize
resource utilization using automated solutions that effectively helped to slash the
pipeline costs by 30% showcasing a keen focus on optimization and cost efficiency.
• Successfully led migration projects, ensuring a smooth transition and minimizing
disruptions to business operations.
Data Engineer
Developed and implemented data-driven solutions leveraging Airflow, BigQuery, Python and DBT to analyze marketing data, providing key insights for retention strategies.
Led and built a global service platform to centralize attributes from diverse teams, streamlining data integration processes and significantly improving operational efficiency, resulting in a 50% reduction in onboarding time.
Designed and implemented a geo-location-based mapping solution for users and vendors, optimizing targeting accuracy and enhancing customer outreach.
Led cloud cost control initiatives by identifying and implementing strategies to optimize resource utilization using automated solutions that effectively helped to slash the pipeline costs by 30% showcasing a keen focus on optimization and cost efficiency.
Successfully led migration projects, ensuring a smooth transition and minimizing disruptions to business operations.
Data Engineer
Developed and implemented an automated maintenance solution for Redshift cluster tables, contributing to enhanced performance.
Built an automated throughput monitor for Redshift cluster to monitor to effectively identify performance degradation in its node performance.
Implemented a billing data pipeline using Python and Redshift which calculates the total resource consumption for any given process and maps to its owner.
Led successful migration of Redshift clusters resulting in improved cluster performance and operational efficiency.
Data Engineer
Built an access review application on AWS cloud platform using AWS Glue, S3 and Lambda services to review/revoke the access based on business compliance requirement and the data provided.
Built robust data pipelines and implemented a data warehousing model to collect, integrate and transform data from different sources into a single unit which can be utilized in different applications.
Led the automation of existing ETLs through different event-triggered mechanisms, eliminating reliance on manual workforce interventions.
Collaborated with data analytics team to analyze and automate data related services and deliver actionable Business Intelligence reporting insights in an agile environment Python and AWS cloud.
Successfully led projects to migrate from Talend environment to AWS cloud.
Associate
Designed and developed a model of an Incentive Compensation (IC) Plan based tool for pharmaceuticals domain on SQL, Talend, Python and Redshift.
Worked on a data warehousing application that integrated data from different sources of varying nature and provided analytical abilities for the end users.
Worked on a reporting model to build reports dynamically as per requirements using master data model.
Worked in the Product Development team to deliver solutions to analyse and automate data related services and to provide Business Intelligence reporting insights in an agile environment.
Mentored junior analysts and interns and supervised them on data modelling and programming deliverables of their respective projects, also kept a track of their progress.
Software Intern
Worked on a project that offers traffic flow to a product website from different affiliates, search results and other internal sites and created ETL packages and SQL stored procedures for different business scenarios in SSIS and SSMS.
Worked on a reporting model which helped to identify the important KPIs using SSAS cubes, data marts and dimensions.
Worked on Adobe Analytics tools to schedule jobs to get data from upstream.
Followed Agile methodology and used Visual Studio Online (VSO) for collaborative development and task management.
Cleared MCSA 70-762 SQL certification.