Abhilash

Design and develop Security Framework to provide fine grained access to objects in AWS S3 using AWS Lambda, DynamoDB.
Setup and work on Kerberos authentication principals to establish secure network communication on cluster and testing of HDFS, Hive, Pig and MapReduce to access cluster for new users.
Perform end- to-end Architecture & implementation assessment of various AWS services like Amazon EMR, Redshift, S3.
Performed logical and physical data structure designs and DDL generation to facilitate the implementation of database tables and columns out to the DB2, SQL Server, AWS Cloud (Snowflake) and Oracle DB schema environment using Erwin Data Modeler Model Mart Repository version.
Implement the machine learning algorithms using Python to predict the quantity a user might want to order for a specific item so we can automatically suggest using kinesis firehose and S3 data lake.
Use AWS EMR to transform and move large amounts of data into and out of other AWS data stores and databases, such as Amazon Simple Storage Service (Amazon S3) and Amazon DynamoDB.
Conduct Data blending, Data preparation using Alteryx and SQL for Tableau consumption and publishing data sources to Tableau server.
Developed web-based applications using Python, DJANGO, QT, C++, XML, CSS3, HTML5, DHTML, JavaScript and jQuery.

• Design and develop Security Framework to provide fine grained access to objects in AWS S3 using AWS Lambda, DynamoDB.
• Setup and work on Kerberos authentication principals to establish secure network communication on cluster and testing of HDFS, Hive, Pig and MapReduce to access cluster for new users.
• Perform end- to-end Architecture & implementation assessment of various AWS services like Amazon EMR, Redshift, S3.
• Performed logical and physical data structure designs and DDL generation to facilitate the implementation of database tables and columns out to the DB2, SQL Server, AWS Cloud (Snowflake) and Oracle DB schema environment using Erwin Data Modeler Model Mart Repository version.
• Implement the machine learning algorithms using Python to predict the quantity a user might want to order for a specific item so we can automatically suggest using kinesis firehose and S3 data lake.
• Use AWS EMR to transform and move large amounts of data into and out of other AWS data stores and databases, such as Amazon Simple Storage Service (Amazon S3) and Amazon DynamoDB.
• Conduct Data blending, Data preparation using Alteryx and SQL for Tableau consumption and publishing data sources to Tableau server.

Developed dashboards and visualizations to help business users analyze data as well as providing data insight to upper management with a focus on Microsoft products like SQL Server Reporting Services (SSRS) and Power BI.
Developed the features, scenarios, step definitions for BDD (Behavior Driven Development) and TDD (Test Driven Development) using Cucumber, Gherkin and Ruby.
Worked on Azure Data Factory to integrate data of both on-prem (MY SQL, Cassandra) and cloud (Blob Storage, Azure SQL DB) and applied transformations to load back to Azure Synapse.
Managed, configured and scheduled resources across the cluster using Azure Kubernetes Service.
Monitored Spark cluster using Log Analytics and Ambari Web UI. Transitioned log storage from Cassandra to Azure SQL Datawarehouse and improved the query performance.
Used Jenkins pipelines to drive all micro-services builds out to the Docker registry and then deployed to Kubernetes, Created Pods and managed using Kubernetes.
Developing ETL pipelines in and out of data warehouse using combination of Python and Snowflakes SnowSQL Writing SQL queries against Snowflake.
Created various pipelines to load the data from Azure data lake into Staging SQLDB and followed by to Azure SQL DB.
Data Ingestion to one or more Azure Services - (Azure Data Lake, Azure Storage, Azure SQL, Azure DW) and processing the data in in Azure Databricks.
Utilized Azure Logic Apps to build workflows to schedule and automate batch jobs by integrating apps, ADF pipelines, and other services like HTTP requests, email triggers etc.
Worked extensively on Azure data factory including data transformations, Integration Runtimes, Azure Key Vaults, Triggers and migrating data factory pipelines to higher environments using ARM Templates.
Ingested data in mini-batches and performs RDD transformations on those mini-batches of data by using Spark Streaming to perform streaming analytics in Data bricks.

Imported required modules such as Keras and NumPy on Spark Session, also created directories for data and output.
Read train and test data into the data directory as well as into Spark variables for easy access and proceeded to train the data based on a sample submission.
The images upon being displayed are represented as NumPy arrays, for easier data manipulation all the images are stored as NumPy arrays.
Created a validation set using Keras2DML in order to test whether the trained model was working as intended or not.
Defined multiple helper functions that are used while running the neural network in session.
Also defined placeholders and number of neurons in each layer.
Created neural networks computational graph after defining weights and biases.
Created a TensorFlow session which is used to run the neural network as well as validate the accuracy of the model on the validation set.
After executing the program and achieving acceptable validation accuracy a submission was created that is stored in the submission directory.
Executed multiple SparkSQL queries after forming the Database to gather specific data corresponding to an image.