نبذة عني
Highly skilled Data Engineer with over 1.5 years of experience in designing and implementing scalable
data solutions. Proficient in ETL processes, data warehousing, and real-time analytics, with a strong
background in cl…
Highly skilled Data Engineer with over 1.5 years of experience in designing and implementing scalable
data solutions. Proficient in ETL processes, data warehousing, and real-time analytics, with a strong
background in cloud technologies (AWS and Azure). Demonstrated ability to enhance data quality,
streamline processes, and deliver actionable insights. Adept at leveraging modern data engineering
tools and frameworks to drive efficiency and support business goals. Proven track record in developing
data pipelines, automating workflows, and improving data governance.
الخبرة
Data Engineer
Sure! Here's a concise version under 1000 characters:
Data Engineer at Mitz - Infopark, Kochi (Oct 2022 - Present):
AWS Bedrock PoC: Developed a PoC with AWS Bedrock for automated test case creation using Streamlit and a vector database.
Digital Manufacturing: Created data dictionaries, metadata enrichment, and connectors for MongoDB, enhancing data transparency.
360 Customer View: Maintained a data processing system for a fintech, ensuring data integrity and traceability.
Deduplication API: Built a real-time Dedup API with AWS services, automating deduplication processes.
RFP Platform & Data Warehouse: Developed a platform for data project aggregation and optimized ETL scripts.
AWS Glue Jobs: Created parameterized Glue jobs, improving data processing efficiency.
Master Parts 360: Conducted a PoC for Master Data Management, enhancing data accuracy.
Data Engineer
Created a PoC with AWS Bedrock to generate test cases from user stories via a Streamlit app, automating test case creation.
Developed architecture integrating Streamlit with AWS Bedrock for user interaction and processing.
Implemented embeddings and conducted similarity searches using a vector database.
Collected and analyzed user feedback for continuous improvement.
Developed data dictionaries and enriched metadata using OpenAI and OpenMetadata.
Designed and implemented connectors for data ingestion and schema extraction from MongoDB.
Improved data transparency and management practices.
Enhanced and maintained a configuration-driven data processing system for a comprehensive 360-degree customer view.
Developed new data transformation functions.
Ensured data integrity and traceability.
Built and maintained a Dedup API and Service Lambda for data deduplication.
Implemented real-time data processing with AWS services.
Automated deduplication processes, reducing manual effort.
Developed a platform to aggregate and organize data-related projects and leads.
Designed and built a data warehouse for efficient data storage and analysis.
Optimized ETL scripts for performance and reliability.
Created parameterized Glue jobs for data processing as part of an MVP for a low-code pipeline platform.
Implemented best practices for Glue job development.
Improved data processing efficiency.
Conducted a PoC for Master Data Management by creating SAP data dictionaries and performing data analysis.
Improved data accuracy and streamlined inventory management.