About Me
6+ years of Full Software Development Life Cycle (SDLC) experience in Software, System Analysis, Design, Development, Testing, Deployment, Maintenance, Enhancements, Re-Engineering, Migration, Troubleshooting, and Suppor…
6+ years of Full Software Development Life Cycle (SDLC) experience in Software, System Analysis, Design, Development, Testing, Deployment, Maintenance, Enhancements, Re-Engineering, Migration, Troubleshooting, and Support of multi-tiered web applications in high-performing environments. Analysis, Design, Development, Documentation, Deployment and Integration of Cloud, Data technologies and Java/J2EE technologies with AWS, AZURE, specialized in Data Ecosystem including Data Aggregation, Querying, Storage, Analysis and Developing data models. Experience in Data Warehousing, ETL, Reporting, Visualization and Database-Driven Solutions with expertise in Development, Testing, Debugging, Documentation, and Production Support.
Experience
Data Engineer
Selected and processed information into CSV records.
Stored records in AWS S3 by utilizing AWS EC2.
Organized and stored data in AWS Redshift.
Developed Glue Jobs to read data in CSV format in the raw layer and write data to parquet format in the publish layer.
Scheduled Glue jobs along with AWS Lambda functions using step functions.
Developed a Glue job that deletes, updates and incrementally loads from source to target.
Implemented Jenkins and built pipelines to drive all microservices builds out to Docker registry and deploying to Kubernetes.
Worked on creating data models for Cassandra from the existing Oracle data model.
Responsible for transforming and loading large sets of structured, semi-structured and unstructured data.
Implemented robust network security measures and optimized networking configurations to ensure the integrity and confidentiality of data transmissions within the organization's infrastructure.
Data Engineer
The Project implementation uses additional productivity and data ops tooling for running jobs, implementing business workflows, and authoring. AWS Glue allows users to connect to diverse data sources and manage data in a centralized data catalog. Used AWS Glue, a serverless cloud data integration tool that facilitates discovery, preparation, movement and data integration from multiple sources for machine learning (ML) analytics and application development.
Data Analyst
Created pipelines in ADF using Linked Services/Datasets/Pipeline to extract, transform, and load data from different sources like Azure SQL, Blob storage, SQL Data warehouse and write-back tool backward.
Developed JSON scripts for deploying the pipeline in Azure Data Factory (ADF) that processes the data using the Cosmos Activity.
Worked with Data Governance and Data Quality to design various models and processes.
Involved in all the steps and scope of the project reference data approach to MDM.
Created a Data Dictionary and Mapping from Sources to the Target in the MDM Data Model.
Created pipelines in ADF using Linked Services/Datasets/Pipeline to Extract, Transform and Load data from different sources like Azure SQL, Blob storage, Azure SQL Data warehouse, write-back tool and backward.
Extracted, Transformed and Loaded data from source systems to Azure Data Storage services using a combination of Azure Data Factory, T-SQL, Spark SQL and U-SQL Azure Data Lake Analytics.
Ingested data to one or more Azure Services - (Azure Data Lake, Azure Storage, Azure SQL, Azure DW) and processed the data in Azure Databricks.
Worked with various formats of files like delimited text files, click stream log files, Apache log files, Avro files, JSON files and XML files.
Mastered in using different columnar file formats like RC, ORC and Parquet formats.
Data Modeler
Created several types of data visualizations using Python and Tableau.
Extracted Mega Data from AWS using SQL Queries to create reports.
Performed reverse engineering using Erwin to redefine entities, attributes and relationships in an existing database.
Analyzed functional and non-functional business requirements, translated them into technical data requirements, created and updated existing logical and physical data models.
Used SQL Server Integration Services (SSIS) to extract, transform, and load data from multiple sources into the target system.
Developed complex reports like Drill Down, Drill through, Cross tab reports and Tableau scorecards.
Created quick filters, table calculations, calculated fields and performed conditional sorting and filtering as per the requirements.
Designed dashboard templates as per the requirements and dashboard content, including complex tabular reports and utilized Tableau capabilities such as Data extraction, Data blending, Forecasting, Dashboard actions and Table calculations.
Stored Procedures, Functions, Database Triggers, and Packages using PL/SQL.
Excellent skills in Business Modeling and Use Case Development using UML methodology.
Experience in Agile PLM integration with other Enterprise/legacy applications is desirable.