Staff Engineer (Platform Engineer)
Job Location:
Monthly Salary:
Posted:
Application Deadline:
Vacancies:
Department:
Job Summary
Must have Skills : AWS Glue Sodacore Lakehouse architecture CDC Medallion design principles
Key Responsibilities
Data Ingestion Development
o Build and implement AWS Glue jobs for Bronze layer ingestion using defined standards and templates.
o Implement correct loading methods based on source requirements (CDC full load delta snapshot).
o Design and execute historical loading mechanisms to bring legacy data into the Lakehouse.
Performance Optimization
o Optimize Glue job performance (DPU allocation parallelization partitioning) according to best practices.
o Collaborate with platform teams to ensure tooling and optimization alignment.
Migration & Automation
o Aggressively migrate source tables to Bronze layer initially using manual approaches with standards/templates later leveraging AI-enabled acceleration.
o Ensure jobs are version-controlled and production deployment is automated via Git and Terraform.
Governance & Monitoring
o Implement source system connectivity into CDP in collaboration with source system owners.
o Ensure jobs comply with data contracts and are properly monitored.
o Prepare documentation and handover to operational support teams.
Collaboration
o Work closely with Data Architect for ingestion patterns and standards.
o Coordinate with Data Assurance Lead to apply quality checks across all jobs.
o Partner with platform engineers for tooling and optimization.
Required Skills & Experience
Technical Expertise
o Strong experience in AWS Glue PySpark and ETL pipeline development.
o Knowledge of Lakehouse architecture and Medallion design principles. - imp
o Familiarity with CDC delta loads and historical data ingestion strategies.
Tools & Technologies
o AWS services: Glue S3 Athena Lambda. - Imp
o Git Terraform for CI/CD automation.
o Understanding of data quality frameworks (e.g. Soda Core) is a plus. - imp
o Able to identify ways to automate their work / repetitive tasks.
Soft Skills
o Ability to work in a fast-paced environment and deliver aggressive migration targets.
o Strong collaboration and communication skills.
o Work with Jira and agile way of working.
Remote Work :
No
Employment Type :
Full-time
Key Skills
- Computer Science
- Docker
- Kubernetes
- Python
- VMware
- C/C++
- Go
- System Architecture
- gRPC
- OS Kernels
- Perl
- Distributed Systems
About Company
Nagarro helps future-proof your business through a forward-thinking, fluidic, and CARING mindset. We excel at digital engineering and help our clients become human-centric, digital-first organizations, augmenting their ability to be responsive, efficient, intimate, creative, and susta ... View more
