Senior Data Warehouse Engineer

Responsibilities:

• As a Data Warehouse Engineer at Careem, you'll be part of a team that builds and supports solutions to organize, process and visualize large amounts of data
• You will be working with technologies such as Hive, Spark, Spark streaming, Kafka, Python, Redash, Presto, Tableau and many others to help Careem become a data-informed company
• Some of the problems the team is working on are: Customer 360, ELT engineering, reporting infrastructure, data reliability, data discovery and access management
• Build and manage ETL workflows to integrate data from many sources into a single source
• Design a framework to ensure data is accessible, up-to-date, secure, consistent and complete
• Work across teams to define organizational needs and ensure business requirements are met You have:
• 6+ years of experience with designing, building and maintaining scalable ETL pipelines
• good understanding of data warehousing concepts and modeling techniques
• Hands-on experience working with real time data processing using Kafka, Spark Streaming or similar technology
• An understanding of Spark core/internals to read and understand spark catalyst plan and perform optimizations
• Understanding of spark concepts like RDD, spark data frame, spark APIs
• Knowledge of data modeling and schema design especially for distributed, column-oriented databases
• Hands-on experience experience with workflow processing engines like Airflow, Luigi
• Ability to dig deeper into the issues of the ETL pipelines, understand the business logic and provide permanent fixes Good to have:
• Experience with CICD using Jenkins, Terraform or other related technologies
• Familiarity with Docker and Kubernetes