Senior Data Engineer - The Emirates Group

Job Purpose: At Emirates we believe in connecting the world to and through our global hub in Dubai; and in constantly innovating to ensure our customers Fly Better. Join the Emirates Group IT Team in Dubai. Emirates Group IT thrives on the dynamic nature of technology. Being pioneers in aviation innovation were always at the forefront pushing boundaries. Were on the lookout for exceptional IT professionals to fortify our position as leaders in the industry. Embark on a journey with the worlds largest international airline and become a vital part of our cuttingedge information and technology team as Senior Data Engineer Quality & Automation .

The Senior Data Quality & Automation Engineer is a fully participating member of a crossfunctional team working autonomously on technology development and problem resolution in the Enterprise Data & Analytics space. The role involves handson contributing to quality practices along with designing and implementing data quality and automation platforms. They will also provide support to technical analytics solutions and products that support Emirates Airlines and the Emirates Group businesses.

Job Outline:

• Work closely with product owners analysts software engineers and architects to understand the technical landscape and context of deliveries to refine complex functional and nonfunctional requirements and translate it into fitforpurpose acceptance tests.
• Work on problems of diverse scope where analysis of data requires evaluation of identifiable factors and selection methods to validate and automate the solution.
• Build test strategies and test plans validating the core business problem and translating them into tests around code functionality data quality performance and security.
• Build and support generating or mocking of test data for exploratory analysis and running tests.
• Design and build automated tests/jobs of moderate to high scope and complexity across all stages of the data pipeline while demonstrating good coding & design practices and adhering to published coding standards and guidelines.
• Build and enhance data quality rules and platforms for observability across the data pipeline
• Debug complex issues resolve blockers and follow design documents with minimal or no supervision.
• Conduct data analysis activities such as source system analysis data modeling data dictionary collection data profiling and sourcetotarget mapping ensuring delivery of business needs.
• Support in updating data inventories and registries as required to keep metadata and data lineage uptodate following agreed Data Governance standards guidelines and principles.

Qualifications & Experience: Qualification:

• Information Technology & Other Degree or Honours (123 or equivalent).
• Degree in a relevant field such as Computer Science Computational Mathematics Computer Engineering or Software Engineering.

Experience:

• Specialization or electives in a Data & Analytics field (e.g. Data Warehousing Data Science Business Intelligence) a nicetohave Experience 2 years of Data Engineering experience with a focus on quality & automation
• Minimum 2 years of testing automation and support experience in analytics applications such as Data Lake and Data Warehouse (preferably using the Big Data stack and Microsoft Azure cloud infrastructure)
• Seasoned in identifying quality issues across complex data pipelines running on big data technologies and defining rules for validating the health of the data
• Experienced in a wide variety of testing methods and tools covering functional performance and security tests across individual jobs pipelines and endtoend across enterprises.
• Wellversed in building automated checks running in a CI pipeline to validate that the ETL / ELT jobs are performing as expected
• Experience with batch and realtime data ingestion/integration tools and technologies handling massive quantities of data (structured and unstructured)
• Exposed to data architecture concepts such as data modeling Big Data storage and dimensional modeling
• Exposed to working with jobs in data pipelines and defining metrics/measures to ensure the correctness of the data
• Programming (Python or Scala) and SQL querying skills are required
• Exposure to Spark & airline industry experience is nice to have

Knowledge/Skills:

• Ability to drive the quality of data assets independently
• Able to deliver solutions (and associated value) interactively
• Strong ability to conduct data analysis (e.g. source system identification data dictionary/metadata collection data profiling sourcetotarget mapping) is preferred
• Operates with a You Code It You Own It mindset (i.e. supports the products they build)
• Team player able to collaborate with others to remove blockers solve complex design problems and debug/resolve issues
• Is accountable and displays a positive attitude Selfstarter and has a passion for exploring and learning new technologies especially those in the Enterprise Data & Analytics space Technology Domain

Key Technologies/Tools Big Data & distributed processing: Spark Hadoop (HDFS Hive HBase Oozie) Airflow Apache Nifi Azure (ADLS DataBricks Azure Data Factory) Elasticsearch AVRO / PARQUET file formats Data Analysis Modelling and Reporting: Snowflake SQL Data Vault 2.0 MicroStrategy Power BI Cloud Technologies: Microsoft Azure and Cloudera technology stacks Integration and Messaging: Streaming (e.g. Spark Streaming) SnapLogic TIBCO Kafka CI/CD: GIT Bitbucket Jenkins Azure DevOps Kubernetes docker SonarQube Gatling Languages: Scala Python

Leadership Role: NO