Data engineer with around 5 years of experience designing and implementing ETL pipelines using SQL and Python Proficient in extracting data from a variety of sources including databases, and flat files, and loading data …
Data engineer with around 5 years of experience designing and implementing ETL pipelines using SQL and Python Proficient in extracting data from a variety of sources including databases, and flat files, and loading data into data warehouses using SQL and loading frameworks. Skilled in data transformation and cleansing using Python and ETL frameworks. Experienced in optimizing data pipeline performance and scalability.
Designed and implemented data migration strategies, ensuring seamless transition of data from on-premises databases to BQ, minimizing downtime and ensuring data consistency.
Leveraged GCP services such as Composer (Cloud Composer), Cloud Shell, and Airflow to automate and orchestrate complex data workflows, streamlining data migration tasks and improving operational efficiency.
Utilized GitHub Actions to establish robust CICD pipelines, promoting collaboration and ensuring consistency across development, testing, and production environments.
Developed and optimized complex SQL queries, stored procedures, and functions to extract, transform, and load (ETL) data from diverse sources into relational databases, including Teradata, Oracle, MySQL, and BigQuery.
Developed and maintained data pipelines using Python and SQL, ensuring efficient extraction, transformation, and loading (ETL) of data across heterogeneous data sources.
Extracting, transforming, and loading (ETL) data into BigQuery from various sources, showcasing your ability to handle large volumes of data and ensure data quality.
• Designed and developed Security Framework to provide fine grained access to objects in AWS S3 using AWS Lambda, DynamoDB.
• Integrated Apache Airflow with AWS to monitor multi-stage ML workflows with the tasks running on Amazon Sage Maker.
• Designed and maintained OBIEE (Oracle Business Intelligence Enterprise Edition) and OAS (Oracle Analytics Server) dashboards and reports.
• Utilized Snowflake as a cloud data platform to build scalable and flexible data warehouse solutions, optimizing performance and cost-effectiveness. Experienced in developing Web Services with Python programming language.
• Implemented a CI/CD pipeline using Jenkins, Airflow for Containers from Docker, and Kubernetes.
Experienced in working with Azure BLOB and Data Lake storage and loading data into Azure SQL Synapse Analytics (DW).
Planning and design the solutions using Azure Files Share.
Worked on Big Data Integration & Analytics based on Hadoop, Spark, Kafka clusters.
Developed ETL processes using PL/SQL, including creating data quality checks and profiling scripts.
Composed python scripts to parse XML and JSON reports and load the information in database.
Written code in Python and PySpark for data manipulation, statistical analysis, modelling, and data validation.
Designed and maintained scheduled data refresh schedules and automated report distribution using Power BI Service.
Experienced in managing Tableau Server infrastructure.
Connecting Tableau to various data sources, such as databases, data warehouses, and APIs.
Hands - on experience in Azure Cloud Services (PaaS & IaaS), Azure Synapse Analytics, SQL Azure, Data Factory, Azure Analysis services, Key Vault, Azure Data Lake.
Developed ETL pipelines using Cloud Functions, Cloud Data Fusion, and Cloud Dataproc to de-identify personal and sensitive data in both real-time and batch modes, loading the de-identified data into BigQuery and Cloud Bigtable for analysis.
Implemented ETL processes with Cloud Composer (Apache Airflow) and Cloud Dataproc to handle sensitive data in batch mode before it is loaded into Cloud Bigtable for querying and analysis.
Supported and maintained data pipelines and ETL processes in production, troubleshooting and resolving data issues using tools like CloudWatch.
Strong knowledge of SQL with experience in writing complex queries for data analysis, alongside continuous improvement of Power BI skills for data visualization and analysis.
The AI-first Recruitment Operating System that unifies and automates the entire hiring lifecycle for modern enterprises worldwide.
Dr Job FZ LLC. 2026 © All Rights Reserved
