About Me
Objective:
Seeking a challenging work environment where I can leverage my technical expertise to develop
and implement innovative solutions, driving organizational growth. As a Data Engineer, I aim to
contribute my sk…
Objective:
Seeking a challenging work environment where I can leverage my technical expertise to develop
and implement innovative solutions, driving organizational growth. As a Data Engineer, I aim to
contribute my skills in data processing, integration, and analysis to enhance decision-making
capabilities and propel the organization forward.
Experience
Data Engineer
• Designed and implemented scalable ETL pipelines using Talend Studio to extract,
transform, and load customer information, loan data, and interest calculations from
heterogeneous data sources, ensuring seamless integration across banking systems.
• Reduced data loading times by 40% by optimizing Talend jobs for parallel execution and
data partitioning, resulting in improved data processing efficiency and faster loan
processing times.
• Implemented data quality checks and validation routines within Talend jobs, ensuring
data accuracy and consistency, leading to a 30% decrease in data discrepancies and
improved decision-making for loan approvals.
• Implemented custom transformation logic using Glue's built-in transformations or
custom Python scripts to standardize data formats and resolve data quality issues.
• Developed and optimized PySpark scripts for processing and analyzing large volumes of
financial data.
• Developed and optimized PySpark scripts for processing and analyzing large volumes of
financial data, enabling timely decision-making and risk assessment.
• Optimized query performance and cost efficiency in AWS Athena by partitioning data
and optimizing query execution plans, resulting in a 40% reduction in query response
times and improved ad-hoc analysis capabilities for financial reporting.
• Achieved seamless data loading and transformation workflows within Redshift by
integrating Talend jobs with Redshift's COPY and UNLOAD commands, ensuring highthroughput data processing and timely insights generation for investment decisions.
• Actively participated in Agile ceremonies and cross-functional collaboration sessions,
contributing to user story refinement and sprint planning to drive continuous
improvement in data management processes
Data Engineer
Designed and implemented scalable ETL pipelines using Talend Studio to extract, transform, and load customer information, loan data, and interest calculations from heterogeneous data sources, ensuring seamless integration across banking systems.
Reduced data loading times by 40% by optimizing Talend jobs for parallel execution and data partitioning, resulting in improved data processing efficiency and faster loan processing times.
Implemented data quality checks and validation routines within Talend jobs, ensuring data accuracy and consistency, leading to a 30% decrease in data discrepancies and improved decision-making for loan approvals.
Implemented custom transformation logic using Glue's built-in transformations or custom Python scripts to standardize data formats and resolve data quality issues.
Developed and optimized PySpark scripts for processing and analyzing large volumes of financial data.
Developed and optimized PySpark scripts for processing and analyzing large volumes of financial data, enabling timely decision-making and risk assessment.
Optimized query performance and cost efficiency in AWS Athena by partitioning data and optimizing query execution plans, resulting in a 40% reduction in query response times and improved ad-hoc analysis capabilities for financial reporting.
Achieved seamless data loading and transformation workflows within Redshift by integrating Talend jobs with Redshift's COPY and UNLOAD commands, ensuring high-throughput data processing and timely insights generation for investment decisions.
Actively participated in Agile ceremonies and cross-functional collaboration sessions, contributing to user story refinement and sprint planning to drive continuous improvement in data management processes.
Data Engineer
Designed and implemented ETL pipelines using Talend Studio to process depositor data for FDIC-related transactions.
Implemented data validation and quality checks within Talend jobs, ensuring compliance with regulatory requirements and data integrity.
Reduced data processing time by 50% through optimization techniques, enhancing operational efficiency and facilitating timely regulatory reporting.
Leveraged PySpark for data processing tasks, improving scalability and performance in handling Parquet files, and enhancing data processing efficiency.
Processed and analyzed large volumes of depositor data using AWS Redshift, Athena, and SQL, enabling efficient regulatory reporting and analysis.
Developed and optimized SQL queries in Athena, facilitating complex data analysis and reporting requirements for regulatory compliance and decision-making.
Managed AWS Redshift data warehouse, optimizing database schemas and query performance to support efficient analysis of depositor data.
Developed a Python-based API to provide access to customer data stored in the Talend-processed database, facilitating seamless integration with other systems and applications.
Containerized the Python API using Docker, leveraging Docker's scalability and high availability features for improved reliability and fault tolerance. Used Kubernetes for orchestration.
Orchestrated job scheduling and monitoring using Autosys, ensuring timely execution of data processing tasks and adherence to project timelines.
Optimized job dependencies and workflows in Autosys, streamlining data pipeline execution and improving overall job efficiency.
ETL Developer
Developed robust data ingestion pipelines using Talend Studio, resulting in a 25% reduction in ingestion time and improved data accuracy.
Enhanced data integration processes, enabling the seamless integration of pharmaceutical data from diverse sources into downstream systems.
Created HQL scripts as per project requirements to insert data into the certified layer, facilitating streamlined data processing and analysis.
Conducted performance tuning activities, resulting in a 40% improvement in data processing throughput and scalability.
Scheduled jobs in Tidal based on job dependencies, ensuring timely and efficient execution of tasks.
Prepared wrapper Talend jobs to orchestrate the execution of child jobs in accordance with dependencies, optimizing job execution and resource utilization.
Provided assistance to the Test team as needed, contributing to testing efforts and ensuring software quality.
Designed and documented Test plans for System Integration Testing (SIT), ensuring comprehensive testing coverage and adherence to project requirements.
Created deployment checklists to ensure smooth and error-free deployment of code to higher environments, minimizing deployment risks.
ETL Developer
Design, develop, and maintain ETL (Extract, Transform, Load) jobs using Talend Open Studio to extract data from various source systems like flat files, excel files.
Implemented data transformations and mappings to ensure compatibility and consistency between source and target data structures.
Monitoring Talend job executions to ensure successful completion and troubleshoot any errors or issues encountered during data processing.
Created deployment checklists to streamline the deployment process, ensuring smooth and error-free transitions between environments.
Addressed and resolved bugs identified during System Integration Testing (SIT) and User Acceptance Testing (UAT), ensuring the quality and reliability of the software.