Suresh

 Collected the log data from Web Servers and integrated into HDFS using Flume
 Supporting Continuous storage in AWS using Elastic Block Storage, S3, and Glacier. Created Volumes and configured Snapshots for EC2 instances.
 Involved in importing data from MS SQL Server, MySQL, and Teradata into HDFS using Sqoop
 Played a key role in dynamic partitioning and Bucketing of the data stored in Hive Metadata
 Created data pipelines for different events to load the data from DynamoDB to the AWS S3 bucket and then into the HDFS location Developed ETL pipelines in and out of data warehouse using combination of Python and Snowflakes SnowSQL Writing SQL queries against Snowflake.
 Analyzing Large volumes of structured data using SparkSQL
 Migrated HiveQL queries into SparkSQL to improve performance
 Involved in loading data into the HBase NoSQL database
 Worked on extending Hive and Pig core functionality by writing custom UDFs using Java
 Worked on ETL Migration services by developing and deploying AWS Lambda functions for generating a serverless data pipeline which can be written to Glue Catalog and can be queried from Athena.
 Used Power BI, and Power Pivot to develop data analysis prototypes, and used Power View and Power Map to visualize reports.

Creating an Application Interface Document for the downstream to create a new interface to transfer and receive the files through Azure Data Share.
Designed and developed user-defined functions, stored procedures, and triggers for Cosmos DB
Analyzing the data flow from different sources to target to provide the corresponding design Architecture in the Azure environment.
Creating DA specs and MappingData flow and providing the details to the developer along with HLDs.
Creating Build definition and Release definition for Continuous Integration and Continuous Deployment.
Creating pipelines, data flows, and complex data transformations and manipulations using ADF and PySpark with Databricks.
Hands-on experience in Designing UI Web Applications using Front End Technologies like HTML, CSS, JavaScript, jQuery, Angular JS, and Bootstrap.
Published Power BI Reports in the required originations and Made Power BI Dashboards available to Web clients and mobile apps
Created, and provisioned different Databricks clusters needed for batch and continuous streaming data processing and installed the required libraries for the clusters.
Implemented the machine learning algorithms using python to predict the quantity a user might want to order for a specific item so we can automatically suggest using kinesis firehose and S3 data lake.
Integrated Azure Active Directory authentication to every Cosmos DB request sent and demoed feature to Stakeholders
Perform ongoing monitoring, automation, and refinement of data engineering solutions prepare complex SQL views, stored procs in AzureSQL DW and Hyperscale
Created a Linked service to land the data from the SFTP location to Azure Data Lake.
Extensively used SQL Server Import and Export Data tool.
Working with complex SQL, Stored Procedures, Triggers, and packages in large databases from various servers.
Used Spark SQL for Scala & amp, Python interface that automatically converts RDD case classes to schema RDD
Involved in designing and deploying multi-tier applications using all the AWS services like (EC2, Route53, S3, RDS, Dynamo DB, SNS, SQS, and IAM) focusing on high-availability, fault tolerance, and auto-scaling in AWS Cloud Formation
Implementations of generalized solution model using AWSSageMaker.

Collected the log data from Web Servers and integrated into HDFS using Flume
Supporting Continuous storage in AWS using Elastic Block Storage, S3, and Glacier.
Created Volumes and configured Snapshots for EC2 instances.
Involved in importing data from MS SQL Server, MySQL, and Teradata into HDFS using Sqoop
Played a key role in dynamic partitioning and Bucketing of the data stored in Hive Metadata
Created data pipelines for different events to load the data from DynamoDB to the AWS S3 bucket and then into the HDFS location
Developed ETL pipelines in and out of data warehouse using combination of Python and Snowflakes SnowSQL
Writing SQL queries against Snowflake.
Analyzing Large volumes of structured data using SparkSQL
Migrated HiveQL queries into SparkSQL to improve performance
Involved in loading data into the HBase NoSQL database
Worked on extending Hive and Pig core functionality by writing custom UDFs using Java
Worked on ETL Migration services by developing and deploying AWS Lambda functions for generating a serverless data pipeline which can be written to Glue Catalog and can be queried from Athena.
Used Power BI, and Power Pivot to develop data analysis prototypes, and used Power View and Power Map to visualize reports.

Worked on migration project which included migrating WebMethods code to Informatica cloud.
Implemented Proof of concepts for SOAP&RESTAPIs.
Built web services mappings and expose them as SOAP wsdl.
Worked with Reporting developers to oversee the implementation of reports/dashboard designs in Tableau.
Assisted users in creating/modifying worksheets and data visualization dashboards in Tableau.
Tuned and performed optimization techniques for improving report/dashboard performance.
Assisted report developers with writing required logic and achieve desired goals.
Met End Users for gathering and analyzing the requirements.
Worked with Business users to identify root causes for any data gaps and developing corrective actions accordingly.
Created Ad hoc Oracle data reports for presenting and discussing the data issues with Business.
Performed Gap Analysisafter reviewing requirements.
Identified data issues within DWH dimension and fact tables like missing keys, joins, etc.
Wrote SQL queries to identify and validate data inconsistencies in data warehouse against source system.