Senior Data Engineer

Dubai - UAE

Monthly Salary: Not Disclosed

Posted on: 30+ days ago

Vacancies: 1 Vacancy

Department:

Data Engineering

Job Summary

Key Responsibilities

Ingestion&Pipelines: Architect batchstream pipelines (Airflow Kafka dbt) for diverse structured and unstructured marked data. Provide reusable SDKs in Python and Go for internal data producers.
Storage&Modeling: Implement and tune S3 columnoriented and timeseries data storage for petabytescale analytics; own partitioning compression TTL versioning and cost optimisation.
Tooling & Libraries: Develop internal libraries for schema management data contracts validation and lineage; contribute to shared libraries and services for internal data consumers for research backtesting and real-time trading purposes.
Reliability & Observability: Embed monitoring alerting SLAs SLOs and CI/CD; champion automated testing data quality dashboards and incident runbooks.
Collaboration: Partner with Data Science QuantResearch Backend and DevOps to translate requirements into platform capabilities and evangelise best practices.

Qualifications :

Required Skills & Experience

7years building productiongrade data systems.
Familiarity with market data formats (e.g. MDP ITCH FIX proprietary exchange APIs) and market data providers.
Expertlevel Python (Go and C nice to have).
Handson with modern orchestration (Airflow) and event streams (Kafka).
Strong SQL proficiency: aggregations joins subqueries window functions (first last candle histogram) indexes query planning and optimization.
Designing highthroughput APIs (REST/gRPC) and data access libraries.
Strong Linux fundamentals containers (Docker) and cloud object storage (AWSS3 / GCS).
Proven track record of mentoring code reviews and driving engineering excellence.

Additional Information :

What we offer:

Working in a modern international technology company without bureaucracy legacy systems or technical debt.
Excellent opportunities for professional growth and self-realization.
We work remotely from anywhere in the world with a flexible schedule.
We offer compensation for health insurance sports activities and professional training.

Remote Work :

Yes

Employment Type :

Full-time

Key ResponsibilitiesIngestion&Pipelines: Architect batchstream pipelines (Airflow Kafka dbt) for diverse structured and unstructured marked data. Provide reusable SDKs in Python and Go for internal data producers.Storage&Modeling: Implement and tune S3 columnoriented and timeseries data storage for ...

Key Responsibilities

Ingestion&Pipelines: Architect batchstream pipelines (Airflow Kafka dbt) for diverse structured and unstructured marked data. Provide reusable SDKs in Python and Go for internal data producers.
Storage&Modeling: Implement and tune S3 columnoriented and timeseries data storage for petabytescale analytics; own partitioning compression TTL versioning and cost optimisation.
Tooling & Libraries: Develop internal libraries for schema management data contracts validation and lineage; contribute to shared libraries and services for internal data consumers for research backtesting and real-time trading purposes.
Reliability & Observability: Embed monitoring alerting SLAs SLOs and CI/CD; champion automated testing data quality dashboards and incident runbooks.
Collaboration: Partner with Data Science QuantResearch Backend and DevOps to translate requirements into platform capabilities and evangelise best practices.

Qualifications :

Required Skills & Experience

7years building productiongrade data systems.
Familiarity with market data formats (e.g. MDP ITCH FIX proprietary exchange APIs) and market data providers.
Expertlevel Python (Go and C nice to have).
Handson with modern orchestration (Airflow) and event streams (Kafka).
Strong SQL proficiency: aggregations joins subqueries window functions (first last candle histogram) indexes query planning and optimization.
Designing highthroughput APIs (REST/gRPC) and data access libraries.
Strong Linux fundamentals containers (Docker) and cloud object storage (AWSS3 / GCS).
Proven track record of mentoring code reviews and driving engineering excellence.

Additional Information :

What we offer:

Working in a modern international technology company without bureaucracy legacy systems or technical debt.
Excellent opportunities for professional growth and self-realization.
We work remotely from anywhere in the world with a flexible schedule.
We offer compensation for health insurance sports activities and professional training.

Remote Work :

Yes

Employment Type :

Full-time

Key Skills

Apache Hive
S3
Hadoop
Redshift
Spark
AWS
Apache Pig
NoSQL
Big Data
Data Warehouse
Kafka
Scala

Apply Now

About Company

BHFT

BHFT is a proprietary algorithmic trading firm. Our team manages the full trading cycle, from software development to creating and coding strategies and algorithms.Our trading operations cover key exchanges. The firm trades across a broad range of asset classes, including equities, eq ... View more

View Profile View Profile

AI AutoApply

Apply to 100+ jobs with one click