Data Engineer

• Data Engineers are tasked with transforming data into a format that can be easily analyzed and utilized.

• The Data Engineer works closely other stakeholders to ensure the quality of data and the systems implemented is upheld to the highest standards possible

• Support in the development of use cases and build the data models and data flows required to support analytical model development

• Build and monitor the integrations of the company including ETL / ELT

• Build and deploy inference pipelines to productionize the models built and refactor code for new future algorithms as a reusable library

• Set up of CI/CD (continuous integration and continuous deployment) pipelines required for the delivery of the company s data services (analytics, data mart, etc.) on their own data platform

• Set up model monitoring capabilities and model retraining workflow to ensure continuous improvement of data models

• Work with the data architect and external vendors in the implementation of any technology infrastructure or tools

• Work closely with Data Architect to produce conceptual and logical data architecture design across the platform

• Build, test and maintain data connectors and data pipelines required for data acquisition in the platform

• Set up data management tools to ensure data is handled appropriately on the platform

• Perform the relevant tests required to ensure data quality and perform holistic analysis of the datasets defined for the company

• Maintain the data quality of dashboards and reports consistently with the relevant internal and external stakeholders (e.g., Data Director, etc.)

• Work with data owners, stewards and data governance and compliance team to understand the business rules and to deploy them into the ETL / ELTs as well as integrate them with the existing dashboards

• Test the outcome of ETL integration, and initiate reconciliation between ETL outputs and ingestions inputs

• Build the test automation rules relating to data ingestion, if applicable

• Provide advice on data topics to ecosystem stakeholders whenever required

Requirements

• Minimum of 5 years of experience in a Data Engineer role

• Preferred degree in Computer Science, Statistics, Informatics, Information Systems or another quantitative field

• Experience in Big Data platforms such as AWS, Azure, Spark, CouchDB, Hive, Pig, etc.

• Experience in integrating data into analytical platforms, ingestion technologies, data profiling, source-target mappings, ETL development, SQL optimization, testing and implementation

• Experience with building processes supporting data transformation, data structures, metadata, dependency, and workload management

• Experience supporting and working with cross-functional teams in a dynamic environment

• Strong analytic skills related to working with unstructured datasets

Minimum of 5 years of experience in a Data Engineer role Preferred degree in Computer Science, Statistics, Informatics, Information Systems or another quantitative field Experience in Big Data platforms such as AWS, Azure, Spark, CouchDB, Hive, Pig, etc. Experience in integrating data into analytical platforms, ingestion technologies, data profiling, source-target mappings, ETL development, SQL optimization, testing and implementation Experience with building processes supporting data transformation, data structures, metadata, dependency, and workload management Experience supporting and working with cross-functional teams in a dynamic environment Strong analytic skills related to working with unstructured datasets