Data Engineering Expert with Masters in MIS from the University of Missouri-Saint Louis, where I honed my analytical and problem-solving skills for complex data challenges. I am passionate about learning new technologies…
Data Engineering Expert with Masters in MIS from the University of Missouri-Saint Louis, where I honed my analytical and problem-solving skills for complex data challenges. I am passionate about learning new technologies and sharing tips and tricks about data engineering, as evidenced by my Databricks, Microsoft, and IBM certifications. Currently, I work as a Sr Data Engineer at Common Spirit Health, where I manage data from multiple sources and build automation frameworks for data validation and testing. I leverage my expertise in Azure, AWS, and Hadoop technologies to deliver scalable and reliable data solutions that support the health care industry. Previously, I worked as a Data Engineer at Credit Suisse, BNSF Railway, T-Mobile, and Raymond James Ltd., where I gained experience in designing and implementing data warehouses, writing Map Reduce and Spark programs, and configuring and administering Hadoop clusters. I am a motivated team player with excellent communication and interpersonal skills, and I value collaboration, innovation, and quality in my work.
*** Profile - Looking for a new Sr Data engineer opportunity! *** (AZURE/AWS/Databricks certified)
Hi,
I am Venkata Sai Akhil having 10 years of hands-on experience in data engineering and showcasing my expertise in leveraging GCP, Big Data Engineering, Snowflake, SQL, Hadoop, and Hive. SAS and SQL for data processing and analysis. I am eager to be part of a dynamic team and play a key role in driving innovation.
Looking forward to the opportunity to contribute my skills and expertise as a Senior Data Engineer.
I am available for any further discussions or interviews at your convenience. Please feel free to reach me at 470-228-0809 or [email protected]
Best regards,
Venkata
470-228-0809
https://www.linkedin.com/in/venkata-balijepalli-5186aa207/
Thanks & Regards!!
Architected and implemented ETL solutions leveraging Snowflake Data Stage in conjunction with Apache Spark on AWS cloud infrastructure, enabling efficient data processing and analytics capabilities.
Designed and optimized Spark-based ETL pipelines to extract, transform, and load large volumes of data into Snowflake data warehouse, ensuring scalability and performance in AWS environments.
Led cross-functional teams in requirements analysis, solution design, and implementation phases.
Collaborating closely with stakeholders to deliver ETL solutions aligned with business objectives and technical requirements.
Leveraged Sparks distributed computing capabilities to parallelize data transformations and accelerate processing speeds, enhancing the efficiency and throughput of ETL workflows in AWS ecosystem.
Implemented robust error handling mechanisms and data quality checks within Spark-based ETL processes.
Built Automation Framework for Hive tables to validate Source (text files, fixed width, XML, Json, Orc-format files) to Target (Hive tables, Phoenix, Json and XML files) Testing and generated reports.
Published the report in the project Dashboard.
Evaluating technology stack for building Analytics solutions on cloud by doing research and finding right strategies, tools for building end to end analytics solutions and help designing technology road map for Data Ingestion, Data lakes, Data processing and Visualization.
Construct data transformation by writing PySpark in Databricks to rename, drop, clean, validate and reformat into parquet files and load them into Azure Blob storage container.
Designed and developed Security Framework to provide fine grained access to objects in AWS S3 using AWS Lambda, DynamoDB.
Having extensive knowledge on Hadoop technology experience in Storage, writing Queries, processing and analysis of data.
Develop Azure linked services to construct connections with on-premises Oracle Database, SQLServer, Apache Hive with Azure datasets in the cloud.
Worked on Teradata, EDW and Hadoop technologies (Teradata, DB2, Hive, Sqoop, Phoenix, HBase).
Used tools like Data Meer to validate HBase tables.
Developed custom ETL solutions, batch processing and real-time data ingestion pipeline to move data in and out of Hadoop using PySpark and shell scripting.
Used Spark and Spark-SQL to read the ORC data and create the tables in hive using the Scala API.
Responsible to manage data coming from different sources and involved in HDFS maintenance and loading of structured data.
Written extensive Spark/Scala programming using Data Frames, Data Sets & RDD's for transforming transnational database data and load it into hive/HBase tables.
Involved in helping the UNIX and Splunk administrators to deploy Splunk across the UNIX and windows environment.
Implemented the machine learning algorithms using python to predict the quantity a user might want to order for a specific item so we can automatically suggest using kinesis firehose and S3 data lake.
Implemented various frameworks like Data Quality Analysis, Data Governance, Data Trending, Data Validation and Data Profiling with the help of technologies like Bigadata, Data Stage, Spark, Python, Mainframe with databases like Netezza and DB2 & Hive.
Worked on integrating Apache Kafka with Spark Streaming process to consume data from external Rest APIs and run custom functions.
Exploring with Spark for improving the performance and optimization of the existing algorithms.
Involved in writing queries in SparkSQL using Scala.
Worked with Splunk to analyze and visualize data.
Building Reusable Data ingestion and Data transformation frameworks using Python.
Optimize the Pyspark jobs to run on Kubernetes Cluster for faster data processing.
Involved in coding & unit testing the new codes.
Prepare Test Plan & Test data.
Deployed web modules in Comcat web server.
Implemented Migration using ngUpgrade Module which allows AngularJS and Angular 2.0 application toco-exist and upgrade application part by part working harmoniously.
The AI-first Recruitment Operating System that unifies and automates the entire hiring lifecycle for modern enterprises worldwide.
Dr Job FZ LLC. 2026 © All Rights Reserved
