Seeking to work in a challenging environment that demands my skills in developing applications using Big Data Eco system including Hadoop framework components.
And contribute myself to develop the organization with my im…
Seeking to work in a challenging environment that demands my skills in developing applications using Big Data Eco system including Hadoop framework components.
And contribute myself to develop the organization with my impressive performance.
• Developed data ingestion pipelines from various data sources and APIs, building ETL pipelines to transform, aggregate, and process data using Azure Databricks.
• Reduced job completion time by 66.6% (from 63 minutes to 21 minutes) at no additional cost by performing a thorough Time vs. Cost analysis.
• Utilized Avro, Parquet, and JSON formats for efficient compression and improved query performance.
• Created Databricks notebooks using SQL and Python, and automated notebooks using jobs.
• Possess a strong understanding of Spark architecture and Delta Lake concepts, with substantial experience in implementing best practices for data storage, loading, retrieval from ADLS, and storing data in Delta Lake.
• Leveraged Azure Data Factory (ADF) to ingest data from various source systems into Azure Data Lake Storage.
• Conducted unit testing, integration testing, debugging, and resolved issues reported by QA.
• Built a CI/CD pipeline in Azure DevOps.
Led the seamless migration of an Audit Reporting Solution from Hadoop to an in-house cloud, achieving 50% faster data retrieval and a 21% increase in processing efficiency.
Transformed business concepts into actionable queries, extracting and refining data from over 5 sources to enhance decision-making processes.
Orchestrated the implementation of a data warehouse, managing datasets of approximately 100GB for Risk & Audit reporting products used by more than 10 stakeholders, aggregating data from multiple sources.
Developed analytical solutions using Spark and SQL, significantly reducing processing and data retrieval time by 66%.
Collaborated with business teams to understand their problems, designing seamless analytical solutions and reusable assets.
Designed test cases to ensure thorough regression and unit testing.
Developed data ingestion pipelines from various data sources and APIs, building ETL pipelines to transform, aggregate, and process data using Azure Databricks.
Reduced job completion time by 66.6% (from 63 minutes to 21 minutes) at no additional cost by performing a thorough Time vs. Cost analysis.
Utilized Avro, Parquet, and JSON formats for efficient compression and improved query performance.
Created Databricks notebooks using SQL and Python, and automated notebooks using jobs.
Possess a strong understanding of Spark architecture and Delta Lake concepts, with substantial experience in implementing best practices for data storage, loading, retrieval from ADLS, and storing data in Delta Lake.
Leveraged Azure Data Factory (ADF) to ingest data from various source systems into Azure Data Lake Storage.
Conducted unit testing, integration testing, debugging, and resolved issues reported by QA.
Built a CI/CD pipeline in Azure DevOps.
Understood business requirements for RBC investment banking transactions and developed code accordingly.
Developed Hive scripts to create source, staging, and final tables in Hive.
Created Sqoop import jobs to transfer static tables from Sybase IQ to Hadoop.
Conducted unit testing, integration testing, debugging, and resolved issues reported by QA.
Used Sqoop ingestion to extract data files and tables to the Hadoop raw layer.
Developed Spark code using Python, applied SCD logic, and implemented business logic to create multiple dimension and fact tables in the Hadoop refined layer.
Developed unit test cases for performance testing on the developed components, defining various test cases and performing data quality checks on the output.
نظام التوظيف التشغيلي المعتمد على الذكاء الاصطناعي أولاً، الذي يوحّد ويؤتمت دورة التوظيف بالكامل للمؤسسات الحديثة حول العالم.
د.جوب منطقة حرة ذ.م.م. 2026 © جميع الحقوق محفوظة
