About Me
I am a Data Engineer with experience in processing and analyzing data to derive meaningful information. I have proficiency in languages such as Python & SQL, and hands-on experience with cloud platforms like GCP (includi…
I am a Data Engineer with experience in processing and analyzing data to derive meaningful information. I have proficiency in languages such as Python & SQL, and hands-on experience with cloud platforms like GCP (including BigQuery, Cloud Run, Cloud Functions, Airflow and more). I have particularly used Airflow to create ETL processes (using SQL and Python) that insert data into Snowflake, Fivetran for EL, and MonteCarlo for monitoring Data Quality and Lineage. I love team collaboration, innovation and continuous learning.
Experience
Data Engineer II
Developed and maintained Airflow DAGs in Python for ETL processes.
Integrated APIs to gather data from various systems.
Connected to Snowflake for data storage and retrieval.
Applied software engineering best practices to deliver clean, high quality, transformed data to stakeholder from multiple teams in the company.
Managed and granted access to resources using Snowflake and Terraform.
Ensured the proper functioning of ETL processes by constantly monitoring them.
Managed alerts when ETL processes failed or exhibited unexpected behavior.
Data Engineer
Implemented a data quality engine for BigQuery tables using user-defined SQL rules and schedules.
Ensured reliable data processes with real-time Slack notifications.
Utilized Cloud Scheduler, Cloud Run, AI Platform Jobs, BigQuery, and Cloud Storage.
Developed a Slack bot with Python and the Slack SDK to enhance team communication.
The bot gathered daily insights from team members on mood, tasks, and challenges.
Posted summaries to a shared Slack channel.
Integrated Slack, Cloud Functions, BigQuery, and Cloud Scheduler jobs.
Created a Feature Store in BigQuery using Table Valued Functions (TVFs) to centralize features for data models.
Managed the store in a Bitbucket repository with Terraform modules and SQL files.
Data Analyst Intern
Applied a simple Fault Tree Analysis algorithm in Python, enhancing analytical processes and system reliability.
Contributed to data cleansing procedures by systematically addressing null values, duplicates, noisy data, and outliers.
Optimized data quality and integrity.
Interpreted and analyzed data from industrial equipment using Python.
Utilized NumPy, Pandas, and Matplotlib for effective data visualization.