About Me
8 years of work experience in to IT in which 4 years in to Azure Data Engineering , in diverse business & technical environments, with demonstrated leadership abilities
Experience
Senior Software Engineer
Data Modeling
Creating robust ETL pipelines in ADF to load data from different sources like sql ,sap , rest API, Salesforce and loading it to data lake
Transforming data as per business needs in data bricks using pyspark
Creating structured data format in synapse also using external tables in Azure synapse
Need to deploy code to UAT and prod by using Azure devops CI/CD
Using HD insights, Kafka and Azure Stream Analytics, logic apps ,function apps also
Azure Data Engineer
Involved in Requirement gathering and identifying the data model requirements and creation of database by designing schema and tables
Using Azure Data Factory for ETL process to migrated the data from on premise to Azure cloud
Transforming data as per business requirement using Python
Created Notebooks in Azure Databricks and running it using Azure Data Factory pipeline
Deployed changes from dev environment to upper environment using Azure DevOps
CI/CD pipelines
By using Visual Studio comparing the databases and keeping in sync across environments
Implemented change data capture for Audit Tables
Created indexes on various tables for better performance and regularly checked databases
Performance recommendations in Azure portal and created indexes
Understand the business model and created data solutions using Azure environment
Coordinated with other teams for testing and deployment practices
Involved in Scrum call, Project planning, review and Retrospective meetings
Updating dashboard to monitor the productivity and burn down chart in Azure Boards
Assure that data is cleansed, mapped, transformed, and otherwise optimized for storage and use according to business and technical requirements
Develop and maintain innovative Azure solutions
Solution design using Microsoft Azure services and other tools
The ability to automate tasks and deploy production standard code (with unit testing, continuous integration, versioning etc.)
Load transformed data into storage and reporting structures in destinations including data warehouse, high speed indexes, real-time reporting systems and analytics applications
Build data pipelines to collectively bring together data
Extracting data, troubleshooting and maintaining the data warehouse