Pranay Gulipilli

Participate in design, implementation, and support of a data warehouse and analytics platform utilizing Azure cloud technology.
Design and implement data load processes from disparate data sources into Azure Data Lake and subsequent Azure SQL & SQL Data Warehouse.
Migrate existing processes and data from on premises SQL Server and other environments to Azure Data Lake.
Design, develop, and support Azure SQL Database Data Marts for functional area data consumers.
Design and build modern data pipelines and data streams.
Analyze current business practices, processes and procedures and identify future opportunities for leveraging Microsoft Azure data & analytics PaaS services.
Collaborate with DW developers for products that require reporting data and ensure that datasets are in place and are used consistently internally/externally.
Collaborate with data governance to ensure all existing data is created in the right way and that new data is created according to appropriate standards and with proper documentation.
Work closely with the Lead Data Engineer and other Data Engineers to develop and document solutions for providing data to the enterprise.
Created data sharing between two snowflake accounts.
Redesigned the Views in snowflake to increase the performance.
Build the logical and physical data model for snowflake as per the changes required.
Generate and schedule ETL tasks using SSIS packages to load the data warehouse.
Helped synchronize Salesforce with SQL Server using SSIS and create and scheduled SSIS jobs through SQL Server agent.
Perform data cleaning, features scaling, and features engineering using pandas and NumPy packages in python.
Create several types of data visualizations using Python and Tableau.
Developed rest API's using python with flask framework and done the integration of various data sources including Java, JDBC, RDBMS, Shell Scripting, Spreadsheets, and Text files.
Designed the ETL processes using Informatica tool to load data from Oracle and flat files into the target Oracle Database.
Created detailed ETL specification for design.
End-to-End Integration testing of ETL-process from source-system to target data mart.
Use of Docker and Kubernetes to manage micro services for development of continuous integration and continuous delivery.
Developed CI/CD system with Jenkins on Google's Kubernetes container.
Scheduling Batch jobs through Amazon Batch performing Data processing jobs by leveraging Apache Spark APIs through Scala.
Expertise on Testing REST API using Robot Framework and SOAP UI.
Strong knowledge in web services testing via Tool SOAP UI/ Restful web services.
Great familiarity with PySpark RDDs, Spark Actions & Transformations, PySpark Data Frames, PySpark-SQL Spark.
Developed Web services component using XML, Restful and SOAP with DOM parser to transfer and transform data between applications.
Developed Service layer using Core JAVA, spring boot, spring, RESTful web-services.

•
Participate in design, implementation, and support of a data warehouse and analytics platform utilizing Azure cloud technology.
•
Design and implement data load processes from disparate data sources into Azure Data Lake and subsequent Azure SQL & SQL Data Warehouse.
•
Migrate existing processes and data from our On Premises SQL Server and other environments to Azure Data Lake.
•
Design, develop, and support Azure SQL Database Data Marts for functional area data consumers.
•
Design and build modern data pipelines and data streams.
•
Analyze current business practices, processes and procedures and identify future opportunities for leveraging Microsoft Azure data & analytics PaaS services.
•
Collaborate with DW developers for products that require reporting data and ensure that datasets are in place and are used consistently internally/externally.
•
Collaborate with data governance; ensure all existing data is created in the right way, and that new data is created according to appropriate standards and with proper documentation.
•
Work closely with the Lead Data Engineer and other Data Engineers to develop and document solutions for providing data to the enterprise.
•
Created data sharing between two snowflake accounts.
•
Redesigned the Views in snowflake to increase the performance.
•
Build the logical and physical data model for snowflake as per the changes required.
•
Generate and schedule ETL tasks using SSIS packages to load the data warehouse.
•
Helped Synchronize Salesforce with SQL Server using SSIS, Create and scheduled SSIS jobs through SQL Server agent.
•
Perform Data Cleaning, features scaling, features engineering using pandas and NumPy packages in python.
•
Create several types of data visualizations using Python and Tableau.
•
Developed rest API's using python with flask framework and done the integration of various data sources including Java, JDBC, RDBMS, Shell Scripting, Spreadsheets, and Text files.
•
Designed the ETL processes using Informatica tool to load data from Oracle, flat files into the target Oracle Database.
•
Created detailed ETL specification for design.
•
End-to-End Integration testing of ETL-process from source-system to target data mart.
•
Use of Docker and Kubernetes to manage micro services for development of continuous integration and continuous delivery.
•
Developed CI/CD system with Jenkins on Google's Kubernetes container.
•
Scheduling Batch jobs through Amazon Batch performing Data processing jobs by leveraging Apac

Develop New Spark-SQL ETL logics in Big Data for the migration and availability of the Facts and Dimensions used for the Analytics.
Develop of PySpark SQL application, Big Data Migration from Teradata to Hadoop and reduce Memory utilization in Teradata analytics.
Requirement Gathering and Leading Team for the development of the Big Data environment and Spark ETL logics migrations.
Responsible for end-to-end design on PySpark-Sql and development to meet the requirements.
Advise the business on best practices in the PySpark-Sql while making sure the solution meets the business needs.
Involved in preparation, distribution, and collaboration of client specific quality documentation on developments for Big Data and Spark along with regular monitoring on reflecting the modifications or enhancements done in Confidential Schedulers.
Migrate the Data from Teradata to Hadoop and data preparation using HIVE Tables.
Analyzed large amounts of data sets to determine optimal way to aggregate and report on it.
Accessing the hive tables using Spark Hive context (Spark sql) and used Scala for interactive operations.
Develop the Spark Sql logics which mimics the Teradata ETL logics and point the output Delta back to Newly Created Hive Tables and as well the existing TERADATA Dimensions, Facts, and Aggregated Tables.
Make sure Data is matched with Teradata and Pyspark-Sql logics.
Creating Views on Top of the Hive tables and giving it to customers for the analytics.
Analyzing Hadoop cluster and different big data analytic tools including Pig, HBase and Sqoop.
Worked with Linux systems and RDBMS database on a regular basis in order to ingest data using Sqoop.
Collected and aggregated large amounts of web log data from different sources such as web servers, mobile and network devices using Apache and stored the data into HDFS for analysis.
Strong knowledge of creating and monitoring cluster on Hortonworks Data platform.
Developed Unix shell scripts to load large number of files into HDFS from Linux File System.
Developed Custom Input Formats in MapReduce jobs to handle custom file formats and to convert them into key-value pairs.
Involved in making Hive tables, stacking information, composing hive inquiries, producing segments and basins for enhancement.
Designed and implemented HIVE queries and functions for evaluation, filtering, loading, and storing of data.
Well versed with snowflake features like clustering, time travel, cloning, logical data warehouse, caching etc.
Extensive experience in writing UNIX shell scripts and automation of the ETL processes using UNIX shell scripting.
Design, develop, test, implement and support Data Warehousing ETL and Hadoop Technologies.
Involved in performance tuning of the ETL process by addressing various performance issues at the extraction and transformation stages.
Worked on the Ad hoc queries, Indexing, Replication, Load balancing, Aggregation in MongoDB.
Experience in managing MongoDB environment from availability, performance, and scalability perspectives.
Developed Spark code using Scala and Spark-SQL for faster testing and data processing.

Developed tools using Python, Shell scripting, XML to automate some of the menial tasks.
Interfacing with supervisors, artists, systems administrators, and production to ensure production deadlines are met.
Developed frontend and backend modules using Python on Django including Tasty Pie Web Framework using Git.
Supported various client projects and internal efforts involving AWS Engineering, managing the Innovation Center specifically the Big Data Platform & Analytics tools-platforms available on Hortonworks.
Administered and monitored multi-Data center Cassandra cluster based on the understanding of the Cassandra Architecture.
Created automated archive process to remove unused tables to ensure optimal database speed.
Implemented 3rd party data transformation process using Redshift, Lambda S3, Kinesis & EDI Exchange software reducing integration time by a factor of 10.
Involved in general application development using Python, JavaScript, HTML/CSS, and AngularJS, with strong integration with Cloud Technologies.
Involved in the migration from Sqlite3 to Apache Cassandra database and Cassandra data model designing, implementation, maintaining and monitoring.
Configured various big data workflows to run on top of Hadoop and these workflows comprise of heterogeneous jobs like MapReduce.
Involved in evaluating existing server and virtualization environments for needed and useful upgrade opportunities.
Designed Spark based real-time data ingestion and real-time analytics, wrote Kafka producer to synthesize alarms using Scala and used Spark-SQL to load JSON data and create SchemaRDD and loaded it into Hive Tables and handled structured data using Spark SQL.
Built Single Page Applications, Responsive Web Design UI, Rich Restful Service Applications, and HTML Wireframes using HTML5 Grid Structures/Layouts, CSS3 Media Queries, Ajax, AngularJS, and Bootstrap.
Built a new CI pipeline with testing and deployment automation using Docker, Jenkins and Puppet.
Utilized continuous integration and automated deployments with Jenkins and Docker.
Involved in development of Python APIs to dump the array structures in the Processor at the failure point for debugging and used Django APIs for database access.
Used ETL warehouse IBM DataStage for filtering and visualizing the raw data.
Created Server instances on AWS and installed Swagger for deploying Microservices.
Used Amazon Web Services for improved efficiency of storage and fast access and working on development and testing of many features for dashboard using Python, Java, Bootstrap, CSS3, and JavaScript.
Analyze data and identify leading SaaS, PaaS or IaaS solutions for clients.
Involved in front end and utilized Bootstrap and Angular.js for page design and using the advanced python packages like NumPy and SciPy for various sophisticated numerical and scientific calculations.
Developed ETL procedures and Data Conversion Scripts using pre-Stage, Stage, Pre-Target, and Target tables.
Developed web applications implementing MVT/MVC architecture using Django, Flask, Webapp2 and spring web application frameworks.
Dynamic implementation of SQL server work on website using SQL developer tool and experience with continuous integration and automation using Jenkins and implemented Service Oriented Architecture using JMS for sending and receiving messages while creating web services.