Hardworking and reliable Data Engineer with strong ability in designing pipeline and analytics with 2.5 years of experience.
GCP Data Engineer | I have experience in designing pipelines and monitoring with GCP services like Bigquery, pubsub, cloud storage, and cloud function. I have worked on code enhancement and impact analysis work as well. I have implemented automation scripts in Python.
Designed and implemented a robust data pipeline utilizing Apache Airflow and Google Cloud Dataflow, seamlessly ingesting Pub/Sub messages into BigQuery tables, enhancing real-time data processing and storage efficiency.
Engineered a comprehensive solution for transforming BigQuery tables as per ML model results and publishing details to Pub/Sub topics, demonstrating proficiency in end-to-end data processing, orchestration, and message communication.
Demonstrated expertise in GCP's data ecosystem by orchestrating the entire workflow, ensuring smooth integration of Pub/Sub, BigQuery, and Dataflow.
Sound knowledge on implementing pipeline with Apache Beam Python SDK.
Collaborated with the development team to write Hive queries and its enhancement on Hadoop with YARN cluster manager.
Worked on on-premises to GCP migration and conducted root cause analysis for failing queries, ensuring seamless data processing and analysis.
Utilized PySpark scripting to automate the creation of 100+ source tables, increasing 50% speed of QA and DEV initial efforts.
Implemented a PySpark script to automate HDFS Commands and Hive query to perform comprehensive verification of 1:1 and transformation logic with report generation saved time required for quality check.
Developed a Python Machine Learning - Time Series model (ARIMA) to accurately predict the duration of job execution, resulting in improved efficiency and optimized resource allocation.
Implemented a robust script that parses JIRA exports and calculates defect counts for each entity, reducing manual effort and ensuring quick and accurate defect analysis.
The AI-first Recruitment Operating System that unifies and automates the entire hiring lifecycle for modern enterprises worldwide.
Dr Job FZ LLC. 2026 © All Rights Reserved
