sindhuja

sindhuja

Data Engineer
United States of America

About Me

Objective:
Seeking a challenging work environment where I can leverage my technical expertise to develop
and implement innovative solutions, driving organizational growth. As a Data Engineer, I aim to
contribute my sk…

Experience

Data Engineer

BPM LINKS
Mar 2024 - Present · 2 years 3 months

• Designed and implemented scalable ETL pipelines using Talend Studio to extract,
transform, and load customer information, loan data, and interest calculations from
heterogeneous data sources, ensuring seamless integration across banking systems.
• Reduced data loading times by 40% by optimizing Talend jobs for parallel execution and
data partitioning, resulting in improved data processing efficiency and faster loan
processing times.
• Implemented data quality checks and validation routines within Talend jobs, ensuring
data accuracy and consistency, leading to a 30% decrease in data discrepancies and
improved decision-making for loan approvals.
• Implemented custom transformation logic using Glue's built-in transformations or
custom Python scripts to standardize data formats and resolve data quality issues.
• Developed and optimized PySpark scripts for processing and analyzing large volumes of
financial data.
• Developed and optimized PySpark scripts for processing and analyzing large volumes of
financial data, enabling timely decision-making and risk assessment.
• Optimized query performance and cost efficiency in AWS Athena by partitioning data
and optimizing query execution plans, resulting in a 40% reduction in query response
times and improved ad-hoc analysis capabilities for financial reporting.
• Achieved seamless data loading and transformation workflows within Redshift by
integrating Talend jobs with Redshift's COPY and UNLOAD commands, ensuring highthroughput data processing and timely insights generation for investment decisions.
• Actively participated in Agile ceremonies and cross-functional collaboration sessions,
contributing to user story refinement and sprint planning to drive continuous
improvement in data management processes

Data Engineer

Data Economy
Jul 2021 - Jul 2022 · 1 year

Designed and implemented scalable ETL pipelines using Talend Studio to extract, transform, and load customer information, loan data, and interest calculations from heterogeneous data sources, ensuring seamless integration across banking systems.
Reduced data loading times by 40% by optimizing Talend jobs for parallel execution and data partitioning, resulting in improved data processing efficiency and faster loan processing times.
Implemented data quality checks and validation routines within Talend jobs, ensuring data accuracy and consistency, leading to a 30% decrease in data discrepancies and improved decision-making for loan approvals.
Implemented custom transformation logic using Glue's built-in transformations or custom Python scripts to standardize data formats and resolve data quality issues.
Developed and optimized PySpark scripts for processing and analyzing large volumes of financial data.
Developed and optimized PySpark scripts for processing and analyzing large volumes of financial data, enabling timely decision-making and risk assessment.
Optimized query performance and cost efficiency in AWS Athena by partitioning data and optimizing query execution plans, resulting in a 40% reduction in query response times and improved ad-hoc analysis capabilities for financial reporting.
Achieved seamless data loading and transformation workflows within Redshift by integrating Talend jobs with Redshift's COPY and UNLOAD commands, ensuring high-throughput data processing and timely insights generation for investment decisions.
Actively participated in Agile ceremonies and cross-functional collaboration sessions, contributing to user story refinement and sprint planning to drive continuous improvement in data management processes.

Data Engineer

Vertiv Software Lab Private Limited
Jun 2020 - Jun 2021 · 1 year

Designed and implemented ETL pipelines using Talend Studio to process depositor data for FDIC-related transactions.
Implemented data validation and quality checks within Talend jobs, ensuring compliance with regulatory requirements and data integrity.
Reduced data processing time by 50% through optimization techniques, enhancing operational efficiency and facilitating timely regulatory reporting.
Leveraged PySpark for data processing tasks, improving scalability and performance in handling Parquet files, and enhancing data processing efficiency.
Processed and analyzed large volumes of depositor data using AWS Redshift, Athena, and SQL, enabling efficient regulatory reporting and analysis.
Developed and optimized SQL queries in Athena, facilitating complex data analysis and reporting requirements for regulatory compliance and decision-making.
Managed AWS Redshift data warehouse, optimizing database schemas and query performance to support efficient analysis of depositor data.
Developed a Python-based API to provide access to customer data stored in the Talend-processed database, facilitating seamless integration with other systems and applications.
Containerized the Python API using Docker, leveraging Docker's scalability and high availability features for improved reliability and fault tolerance. Used Kubernetes for orchestration.
Orchestrated job scheduling and monitoring using Autosys, ensuring timely execution of data processing tasks and adherence to project timelines.
Optimized job dependencies and workflows in Autosys, streamlining data pipeline execution and improving overall job efficiency.

ETL Developer

Capgemini Technology Services India
Oct 2019 - Apr 2020 · 6 months

Developed robust data ingestion pipelines using Talend Studio, resulting in a 25% reduction in ingestion time and improved data accuracy.
Enhanced data integration processes, enabling the seamless integration of pharmaceutical data from diverse sources into downstream systems.
Created HQL scripts as per project requirements to insert data into the certified layer, facilitating streamlined data processing and analysis.
Conducted performance tuning activities, resulting in a 40% improvement in data processing throughput and scalability.
Scheduled jobs in Tidal based on job dependencies, ensuring timely and efficient execution of tasks.
Prepared wrapper Talend jobs to orchestrate the execution of child jobs in accordance with dependencies, optimizing job execution and resource utilization.
Provided assistance to the Test team as needed, contributing to testing efforts and ensuring software quality.
Designed and documented Test plans for System Integration Testing (SIT), ensuring comprehensive testing coverage and adherence to project requirements.
Created deployment checklists to ensure smooth and error-free deployment of code to higher environments, minimizing deployment risks.

ETL Developer

Vincere Semantics Pvt Ltd India
Nov 2014 - Sep 2019 · 4 years 10 months

Design, develop, and maintain ETL (Extract, Transform, Load) jobs using Talend Open Studio to extract data from various source systems like flat files, excel files.
Implemented data transformations and mappings to ensure compatibility and consistency between source and target data structures.
Monitoring Talend job executions to ensure successful completion and troubleshoot any errors or issues encountered during data processing.
Created deployment checklists to streamline the deployment process, ensuring smooth and error-free transitions between environments.
Addressed and resolved bugs identified during System Integration Testing (SIT) and User Acceptance Testing (UAT), ensuring the quality and reliability of the software.

Skills

Big Data Data Repository Hadoop Scala Spark Apache Hive Apache Pig Amazon Web Services (AWS) Kafka NoSQL Redshift S3 Linux Unix Windows Athena AWS EMR Talend Data Integration Talend Data Mapper Hive PySpark Apache Kafka Autosys Crontab Docker Kubernetes Basic Shell Scripting Python SQL Talend Studio AWS Glue AWS Redshift AWS Athena Tidal Oozie SQL Server Postgresql Talend Open Studio MySQL
Report this Profile?