About Me
Around 3+ years of experience as an Azure Data Engineer in designing and building data pipelines, data lakes and data warehouses
Proficient in crafting scalable and efficient data architectures, including data lakes and …
Around 3+ years of experience as an Azure Data Engineer in designing and building data pipelines, data lakes and data warehouses
Proficient in crafting scalable and efficient data architectures, including data lakes and data warehouses, using Azure services like Azure Data Factory, Azure Kafka
Extensive experience in building robust ETL (Extract, Transform, Load) processes, ensuring data quality and integrity, utilizing tools such as, Apache Spark, or custom Python scripts
Adept at implementing monitoring, logging, and performance optimization strategies to maintain high data pipeline reliability and efficiency
Experienced in crafting and enforcing data policies and standards, promoting data consistency, and aligning data practices with industry regulations. Skilled in implementing data governance frameworks to drive organizational compliance
Highly skilled in developing comprehensive data migration strategies, including assessing source data, defining migration goals, and creating detailed plans
Well-versed in designing, implementing, and optimizing NoSQL database solutions, leveraging technologies such as Cosmos DB, to provide high-performance, scalable, and flexible data storage solutions for modern applications
Knowledgeable on PySpark to develop code for various Spark use cases, achieving 30% optimization in Spark cluster performance, and used Python for data analytics on Spark clusters, performing map-side joins on RDD with an efficiency of up to 15%
Skilled in data visualization using Tableau and Power BI to create interactive and insightful dashboards and reports that facilitate data-driven decision-making and provide valuable insights to stakeholders
Effective communicator and team player, able to collaborate with data scientists, analysts, and cross-functional teams to understand and meet data requirements
Proficient in using Atlassian's Jira for agile project management, issue tracking, and workflow customization, and Confluence for collaborative documentation and knowledge sharing, ensuring streamlined project development and effective team communication
Experience
Azure Data Engineer
Developed and maintained data models, including logical and physical models, to support data integration and reporting needs. Ensured data models aligned with industry best practices
Implemented real-time data ingestion solutions using Azure Data Factory Stream Analytics to process and analyze streaming data from IoT devices, applications, and other sources. Leveraged Azure Event Hubs and IoT Hubs for data ingestion, enabling timely insights and decision-making for business stakeholders
Successfully led a complex data migration project, achieving a 50% reduction in downtime, allowing critical systems to remain operational, and realizing a 30% cost savings by optimizing the migration process. This involved transferring 10 terabytes of legacy data to a modern cloud platform with minimal disruption
Designed and implemented scalable data storage solutions using Azure Data Lake and Blob Storage. Utilized features such as Azure Cool and Archive storage tiers to optimize cost while maintaining data availability
Conducted performance monitoring and optimization of Azure Synapse Analytics workloads using Azure Monitor, Azure Data Studio, Azure Kafka and query performance tuning techniques, resulting in improved query efficiency
Utilized Azure HDInsight and Azure Data Lake Analytics for big data processing tasks. Leveraged technologies like Apache Spark and Hive for data analytics and processing at scale
Implemented Azure Purview (formerly Azure Data Catalog) for centralized data cataloging and metadata management, making it easier for stakeholders to discover and understand data assets
Designed data models that align with Azure Cosmos DB's schema-agnostic nature, and implemented performance optimization strategies, including indexing and request unit (RU) provisioning, to ensure efficient and scalable data access
Conducted performance tuning of Azure Stream Analytics jobs, optimizing query performance, and ensuring efficient resource utilization for cost-effective data streaming
Implemented a robust ETL pipeline that efficiently extracted, transformed, and loaded vast amounts of data, resulting in a 50% reduction in data processing time and a 20% improvement in data quality, empowering data-driven decision-making within the organization.
Azure Data Engineer
Developed and maintained data models, including logical and physical models, to support data integration and reporting needs.
Ensured data models aligned with industry best practices.
Implemented real-time data ingestion solutions using Azure Data Factory Stream Analytics to process and analyze streaming data from IoT devices, applications, and other sources.
Leveraged Azure Event Hubs and IoT Hubs for data ingestion, enabling timely insights and decision-making for business stakeholders.
Successfully led a complex data migration project, achieving a 50% reduction in downtime, allowing critical systems to remain operational, and realizing a 30% cost savings by optimizing the migration process.
Transferred 10 terabytes of legacy data to a modern cloud platform with minimal disruption.
Designed and implemented scalable data storage solutions using Azure Data Lake and Blob Storage.
Utilized Azure Cool and Archive storage tiers to optimize cost while maintaining data availability.
Conducted performance monitoring and optimization of Azure Synapse Analytics workloads using Azure Monitor, Azure Data Studio, Azure Kafka and query performance tuning techniques, resulting in improved query efficiency.
Utilized Azure HDInsight and Azure Data Lake Analytics for big data processing tasks.
Leveraged Apache Spark and Hive for data analytics and processing at scale.
Implemented Azure Purview (formerly Azure Data Catalog) for centralized data cataloging and metadata management, making it easier for stakeholders to discover and understand data assets.
Designed data models that align with Azure Cosmos DB's schema-agnostic nature, and implemented performance optimization strategies, including indexing and request unit (RU) provisioning, to ensure efficient and scalable data access.
Conducted performance tuning of Azure Stream Analytics jobs, optimizing query performance, and ensuring efficient resource utilization for cost-effective data streaming.
Implemented a robust ETL pipeline that efficiently extracted, transformed, and loaded vast amounts of data, resulting in a 50% reduction in data processing time and a 20% improvement in data quality, empowering data-driven decision-making within the organization.