Manjith kumar

Successfully designed, developed, and deployed ETL solutions using Azure Synapse Analytics and Azure Data Factory, resulting in a 30% reduction in data processing time and a 20% improvement in data accuracy.
Ingested data into various Azure Services, including Azure Data Lake, Azure Blob, Azure SQL, and Dedicated SQL Pools in Synapse Analytics, achieving a 25% increase in data availability and accessibility.
Utilized Terraform to version infrastructure on Azure and automated resource management, reducing deployment time by 40% and minimizing errors.
Developed JSON scripts for deploying pipelines in Azure Data Factory, streamlining the data processing workflow and reducing manual intervention by 50%.
Implemented complex business logic through T-SQL stored procedures, functions, and advanced query concepts, resulting in a 15% improvement in data transformation efficiency.
Successfully executed various data activities in Azure Data Factory, including data movement, transformations, control activities, copy data, data flow, metadata retrieval, lookup operations, stored procedures, and pipeline execution, leading to a 25% improvement in data processing speed.
Conducted data transformations and executed actions on the data using the PySpark DataFrame API as part of the ETL process, resulting in a 20% reduction in data processing time and a 15% improvement in data quality.
Created Databricks notebooks using PySpark and SparkSQL for data transformation in Azure Data Lake, leading to a 30% improvement in data transformation efficiency.
Demonstrated expertise in tuning and debugging Spark jobs by analyzing DAG and lineage, resulting in a 20% reduction in job failures and improved job performance.
Worked with data in multiple file formats, including Parquet, Avro, delta, Sequence Files, CSV, and JSON, optimizing data storage and retrieval processes.
Integrated diverse data sources into PowerBI, developing interactive dashboards that reduced reporting time by 20% and enhanced data-driven insights by 30%, providing stakeholders with more actionable insights for decision-making.

• Successfully designed, developed, and deployed ETL solutions using Azure Synapse Analytics and Azure Data Factory, resulting in a 30% reduction in data processing time and a 20% improvement in data accuracy.
• Ingested data into various Azure Services, including Azure Data Lake, Azure Blob, Azure SQL, and Dedicated SQL Pools in Synapse Analytics, achieving a 25% increase in data availability and accessibility.
• Utilized Terraform to version infrastructure on Azure and automated resource management, reducing deployment time by 40% and minimizing errors.
• Developed JSON scripts for deploying pipelines in Azure Data Factory, streamlining the data processing workflow and reducing manual intervention by 50%.
• Implemented complex business logic through T-SQL stored procedures, functions, and advanced query concepts, resulting in a 15% improvement in data transformation efficiency.
• Successfully executed various data activities in Azure Data Factory, including data movement, transformations, control activities, copy data, data flow, metadata retrieval, lookup operations, stored procedures, and pipeline execution, leading to a 25% improvement in data processing speed.
• Conducted data transformations and executed actions on the data using the PySpark DataFrame API as part of the ETL process, resulting in a 20% reduction in data processing time and a 15% improvement in data quality.
• Created Databricks notebooks using PySpark and SparkSQL for data transformation in Azure Data Lake, leading to a 30% improvement in data transformation efficiency.
• Demonstrated expertise in tuning and debugging Spark jobs by analyzing DAG and lineage, resulting in a 20% reduction in job failures and improved job performance.
• Worked with data in multiple file formats, including Parquet, Avro, delta, Sequence Files, CSV, and JSON, optimizing data storage and retrieval processes.
• Integrated diverse data sources into PowerBI, developing interactive dashboards that reduced reporting time by 20% and enhanced data-driven insights by 30%, providing stakeholders with more actionable insights for decision-making.

Worked on creating OLAP Model based on dimensions and facts tables for efficient loads of data based on star schema structure on levels of reports using multi-dimensional models such as Star and Snowflake Schemas in Postgres Database.
Implemented advanced indexing strategies and query optimization techniques to enhance the Postgres database's performance. Achieved a 30% reduction in query response times, ensuring fast and efficient data retrieval for downstream systems.
Devised and executed a data scaling plan to handle rapid platform expansion, ensuring data integrity. Managed data sharding and partitioning, achieving a 40% boost in database scalability and maintaining a 99.9% data availability rate, even during peak usage.
Designed and implemented efficient data models in the Google Firestore database to store unstructured data received from users in the web application.
Boosted the cloud Firestore database from 70% to over 90% capacity by implementing autoscaling solutions to address fluctuating load demands and implemented data backup and recovery strategies to safeguard against data loss, including regular Firestore data exports and backups.
Utilized version control system GitLab to manage database schema changes and collaborated with developers to ensure smooth integration with application code.

Worked on Hortonworks Data Platform Hadoop distribution for data querying using Hive to store and retrieve data.
Implemented and managed Hadoop Distributed File System (HDFS) solutions, optimizing data replication, reliability, and availability, resulting in a 20% reduction in data storage costs.
Developed and executed spark jobs for high-performance data transformations, reducing processing time by 30%.
Involved in data ingestion into HDFS using Sqoop for full load and Kafka for incremental load on a variety of sources like web servers, RDBMS, and Data API’s.
Experience in implementing Spark RDD transformations, actions, and working with accumulators and broadcast variables and involved in writing Queries in Spark SQL using PySpark.
Created Hive external tables and views on the data imported into the HDFS and developed and implemented Hive Scripts for various transformations such as evaluation, filtering, and aggregation.
Improved overall system performance by analyzing ETL job statistics and conducting Hadoop cluster performance assessments using Ambari log data and resource utilization metrics, resulting in a 15% reduction in job failures.
Significantly enhanced data processing, analysis, and reporting capabilities by optimizing Hive queries and Spark jobs within the Hadoop ecosystem, resulting in a 30% improvement in data processing efficiency.
Reduced query runtime by 40% by implementing strategic partitioning and bucketing techniques for Hive tables, while concurrently running scripts in parallel.
Streamlined data management workflows by developing Bash scripts to automate the Extraction, Transformation, and Loading (ETL) processes, reducing manual intervention by 50%.

Used SQL Server Integration Services (SSIS) as a powerful ETL tool, enabling efficient data extraction, transformation, and loading processes.
Developed SQL scripts for ETL processes, reducing data processing errors by 25% and increasing data throughput by 30%. Implemented robust data validation procedures, ensuring 99% data integrity.
Continuously improved the in-house ETL framework by incorporating advanced features like dynamic schema detection and automatic data lineage tracking, enhancing overall data governance.
Conducted workshops for the ETL development team on performance-tuning techniques and best practices, resulting in a 20% increase in team productivity and code efficiency.
Created comprehensive documentation for ETL processes and best practices, facilitating knowledge transfer among team members and reducing onboarding time by 40%.
Designed and implemented custom ETL pipelines in SSIS to process various data sources, ensuring data quality and consistency, resulting in a 25% reduction in data discrepancies.
Developed and executed complex SQL queries, including table creation, updates, joins, aggregate functions, and subqueries, resulting in streamlined data processing and a 20% reduction in query response times.
Managed the development of complex SQL queries to check data accuracy against different reports. This improved data quality and ensured smooth integration with reports, achieving a 99% accuracy rate.
Enhanced existing ETL processes, resulting in a 30% reduction in data loading times by fine-tuning SQL queries.
Collaborated closely with data scientists and analysts to understand their data requirements, leading to the creation of custom ETL pipelines that improved data accessibility and reduced time-to-insight by 30%.