نبذة عني
An astute leader with 12+ years’ experience which embraces 6+ years in building Big Data Pipe line, Data Warehouse & Implementing Data Migration Projects in On-Premises/Cloud platform and 6 years in Application Programme…
An astute leader with 12+ years’ experience which embraces 6+ years in building Big Data Pipe line, Data Warehouse & Implementing Data Migration Projects in On-Premises/Cloud platform and 6 years in Application Programmer using OOPS and SOA concept. Well versed with Big Data Technologies like Hadoop, Apache Spark, Apache Hive, Apache Sqoop, Map Reduce, Kafka using Python, Scala and Java. Working in AWS platform to Build Data Warehouse system using S3, EMR, EC2, Glue, Redshift and Athena. Expertise in End-to-end Architectural Design, Development of Data Intensive, Data Migration Application, Building Data Lake and Data Warehouse System.
الخبرة
Data Engineer
Data Engineer | Samsung, Chennai, India | May 2020 – Present
• Gather business requirements and translate them into technical solutions. Design, develop, test, improve, and maintain new and existing solutions for data integration, data preparation.
• Incorporate deep data management expertise into solutions, including integrity, recovery, security, and monitoring. Validate and qualify solutions. Identify, troubleshoot, and resolve technical issues.
• Work with business stakeholders, IT groups and third-party vendors to understand source systems and enterprise infrastructure offerings. Work with data from multiple data sources to build integrated views that will drive decisions.
• Work with business stakeholders to assist with data-related technical issues and support the data infrastructure needs. Monitor daily execution and fix business critical pipelines to ensure SLAs are met.
• Ensure data reliability, efficiency, quality to enable analytics team to run the sophisticated analytics program. Rewire hive queries and spark application to boost the performance.
Data Engineer
Gather business requirements and translate them into technical solutions.
Design, develop, test, improve, and maintain new and existing solutions for data integration, data preparation.
Incorporate deep data management expertise into solutions, including integrity, recovery, security, and monitoring.
Validate and qualify solutions.
Identify, troubleshoot, and resolve technical issues.
Work with business stakeholders, IT groups and third-party vendors to understand source systems and enterprise infrastructure offerings.
Work with data from multiple data sources to build integrated views that will drive decisions.
Work with business stakeholders to assist with data-related technical issues and support the data infrastructure needs.
Monitor daily execution and fix business critical pipelines to ensure SLAs are met.
Ensure data reliability, efficiency, quality to enable analytics team to run the sophisticated analytics program.
Rewire hive queries and spark application to boost the performance.
Data Engineer
Gather business requirements and translate them into technical solutions
Design, develop, test, improve, and maintain new and existing solutions for data integration, data preparation
Incorporate deep data management expertise into solutions, including integrity, recovery, security, and monitoring
Validate and qualify solutions
Identify, troubleshoot, and resolve technical issues
Work with business stakeholders, IT groups and third-party vendors to understand source systems and enterprise infrastructure offerings
Work with data from multiple data sources to build integrated views that will drive decisions
Work with business stakeholders to assist with data-related technical issues and support the data infrastructure needs
Monitor daily execution and fix business critical pipelines to ensure SLAs are met
Ensure data reliability, efficiency, quality to enable analytics team to run the sophisticated analytics program
Rewire hive queries and spark application to boost the performance
Sr Consultant
Create, maintain and monitor an efficient data pipeline architecture, including working with scheduled jobs, replication, monitoring and partitioning.
Assemble complex data to meet business requirements.
Setup and maintain infrastructure to solve high volume data engineering problems for stream processing and batch processing.
Ensure data integrity and consistency throughout, and enable teams to easily verify this in end analysis.
Follow the proven innovation in the implementation to layout the optimized solutions.
Closely work with client and associate in preparing the sprint plan.
Identify, design, and implement internal process improvements automating processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.
Build the libraries and frameworks that support large, complex data process business rules.
Participate in value stream mapping, backlog grooming, sprint planning, team retros and demos.
Sr. Associate Consultant
Identify, Design and build the proof of concept and prototype data pipeline to ease the business process and optimize performance.
Sr. Engineer
Produce the optimized data pipe line and ensuring data is collected and managed in the central data warehouse system to enable the analytics platform.
Addressing team difficulties to implement the solutions.
Own database and data flow documentation.
Contribute to code, design reviews and suggest improvements.
Follow quality development rules and recommendations.
Suggests and implements improvements and optimisations to development process.
Lead Engineer
Design, develop, test, deploy, maintain, and improve data and new user-facing features.
Write client and server-side code for service-oriented applications, create fast, easy-to-use, high volume production applications, and develop prototypes quickly.
Define the effort estimates, risks and dependencies during or after release planning.
Integrate, build and release software stack; Responsible for delivering component.
Update all the relevant and required technical documents.
Create highly readable & maintainable software.
Provide efficient code.
Sr. Engineer
Design, build and support monitoring and observability tools and policies.
Write code that is tested, readable, and maintainable.
Perform code reviews, evaluate implementations, and provide feedback for improvements.
Engage in technical discussion and decision making to shape the future of making things.
Collaborate with stakeholders to understand requirements, understand use cases and build towards a cohesive technical strategy.
Software Engineer
Design and implement re-usable software components.
Ensure the product is highly configurable and extensible.
Work collaboratively with a core team of architects and developers spread across different locations.
Research and build POCs using available frameworks to ensure feasibility.
Create technical design documents and present the details to a larger audience.
Participate in architecture and design reviews for projects that require complex technical solutions.
Sr Consultant
Create, maintain and monitor an efficient data pipeline architecture, including working with scheduled jobs, replication, monitoring and partitioning
Assemble complex data to meet business requirements
Setup and maintain infrastructure to solve high volume data engineering problems for stream processing and batch processing
Ensure data integrity and consistency throughout, and enable teams to easily verify this in end analysis
Follow the proven innovation in the implementation to layout the optimized solutions
Closely work with client and associate in preparing the sprint plan
Identify, design, and implement internal process improvements automating processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc
Build the libraries and frameworks that support large, complex data process business rules
Participate in value stream mapping, backlog grooming, sprint planning, team retros and demos
Sr. Associate Consultant
Identify, Design and build the proof of concept and prototype data pipeline to ease the business process and optimize performance
Sr. Engineer
Produce the optimized data pipe line and ensuring data is collected and managed in the central data warehouse system to enable the analytics platform
Addressing team difficulties to implement the solutions
Own database and data flow documentation
Contribute to code, design reviews and suggest improvements
Follow quality development rules and recommendations
Suggests and implements improvements and optimisations to development process
Lead Engineer
Design, develop, test, deploy, maintain, and improve data and new user-facing features
Write client and server-side code for service-oriented applications, create fast, easy-to-use, high volume production applications, and develop prototypes quickly
Define the effort estimates, risks and dependencies during or after release planning
Integrate, build and release software stack; Responsible for delivering component
Update all the relevant and required technical documents
Create highly readable & maintainable software
Provide efficient code
Sr. Engineer
Design, build and support monitoring and observability tools and policies
Write code that is tested, readable, and maintainable
Perform code reviews, evaluate implementations, and provide feedback for improvements
Engage in technical discussion and decision making to shape the future of making things
Collaborate with stakeholders to understand requirements, understand use cases and build towards a cohesive technical strategy
Software Engineer
Design and implement re-usable software components
Ensure the product is highly configurable and extensible
Work collaboratively with a core team of architects and developers spread across different locations
Research and build POCs using available frameworks to ensure feasibility
Create technical design documents and present the details to a larger audience
Participate in architecture and design reviews for projects that require complex technical solutions