GCP Data Engineer at Tata Consultancy Services with hands-on experience in migrating Spark Scala data pipelines from on-prem Hadoop environments to Google Cloud Platform. Skilled in building and supporting batch and real…
GCP Data Engineer at Tata Consultancy Services with hands-on experience in migrating Spark Scala data pipelines from on-prem Hadoop environments to Google Cloud Platform. Skilled in building and supporting batch and real-time data pipelines using Dataproc Serverless, BigQuery, GCS, and Kafka. Experienced in large-scale application migration across multiple business sectors, with a strong focus on data reliability, performance, and cloud-native solutions.
Migrated Spark Scala data pipelines from on-prem Hadoop environments to Google Cloud Platform (GCP)., Transitioned storage from HDFS to Google Cloud Storage (GCS) and compute from Hadoop clusters to Dataproc Serverless., Migrated batch pipelines scheduled via Control-M, handling file ingestion from FTPIXIB to GCS raw layers using Cloud Run jobs and shell scripts., Processed batch data using Dataproc Spark jobs, writing refined data in Parquet/ CSV formats and loading into BigQuery and SQL databases., Built and migrated real-time pipelines consuming data from Kafka, processing with Spark, and writing to BigQuery., Worked on end-to-end migration and support for 8 business sector applications, ensuring data quality and reliable processing.
The AI-first Recruitment Operating System that unifies and automates the entire hiring lifecycle for modern enterprises worldwide.
Dr Job FZ LLC. 2026 © All Rights Reserved
