Data Engineer

Job Description

1. 4-6 years of relevant experience
2. Skills – Python, SQL (writing complex queries), ETL, Pipelines, Apache Airflow, Azure, AWS
3. Databases – Vertica, PostgreSQL, Netezza
4. Documentation of pipelines
5. Reengineering and Migration of pipelines
6. Onsite/Offshore/Near shore
7. SAP Data Services is added advantage

Scope:

1. Detailed documentation of existing ETLs and complete workflow with all dependent systems.
2. Detailed analysis and strategy for Migration
3. Identifying and agreeing on changes required in system w.r.t. technology, tools, design, development etc.
4. Migration of ETLs for Database changes
5. Migration of ETLs from Airflow to SAP DS
6. Finetuning of ETLs wherever required
7. Migration of supporting code like Functions, Stored procedures, Views etc from Vertica to Netezza
8. Changes / finetuning of Python code
9. Changes / finetuning of SQLs

Requirements

Skillset:

• Senior Data Engineer with Data Pipeline, Python, SQL & R experience.
• Experienced in SAP Data Services
• Experienced in ETL migrations.
• Experience in applying best practices, tuning / optimizing the pipelines and its execution.
• Exposure to technologies and databases – Airflow, Hassura, Vertica, Netezza, Postgres, Python, SQL

Skillset: Senior Data Engineer with Data Pipeline, Python, SQL & R experience. Experienced in SAP Data Services Experienced in ETL migrations. Experience in applying best practices, tuning / optimizing the pipelines and its execution. Exposure to technologies and databases – Airflow, Hassura, Vertica, Netezza, Postgres, Python, SQL