Data Engineer with Microsoft Power Platform experience - Riyadh - Saudi Arabia - Scarlatt

Data Engineer with Microsoft Power Platform Experience

Summary

The main objective of this position is to create construct test and uphold data pipelines facilitating the movement of data from source systems to data lakes data warehouses and other storage and processing systems. Additionally the role involves the development and upkeep of data APIs ETL processes and systems for data integration.

Key Responsibilities

• Design create and manage scalable data pipelines for the ingestion processing and analysis of extensive datasets.
• Implement and refine ETL processes to guarantee the effective extraction transformation and loading of data from diverse sources into our data lake/data warehouse.
• Collaborate with crossfunctional teams including data scientists analysts and the BI team to comprehend their data needs and provide technical solutions.
• Ensure the quality integrity and security of data by implementing best practices and industrystandard techniques.
• Monitor and troubleshoot data pipelines resolving issues related to data ingestion processing or storage.
• Assess and integrate new technologies tools and frameworks to enhance the performance and capabilities of our data infrastructure.

Key Skills

• 2 years of handson experience with Databricks.
• Familiarity with cloud platforms (preferably AWS but also Azure and GCP) and their associated data services (such as S3 Redshift and BigQuery).
• Handson experience with stream processing technologies including Kafka Spark Streaming and Kinesis.
• Expertise in the design and optimization of ETL processes.
• Significant experience with various data storage technologies including SQL databases NoSQL databases and data warehousing solutions.
• Understanding of workflow management platforms for data pipelines such as Apache Airflow.
• Proficiency in programming languages particularly Python or Scala.
• Familiarity with containerization and orchestration tools such as Docker and Kubernetes will be advantageous.
• Knowledge of code versioning tools like Git is considered a plus.

What We Offer

Capability to dissect intricate problems analyze data and formulate logical solutions.

Readiness and aptitude to adapt to evolving technologies tools and project requirements.

Aptitude to work efficiently within a team communicate concisely and contribute to collective objectives.

Ability to collaborate and coordinate with internal analysts BI teams and external vendors.

Adherence to ethical standards and upholding professional integrity while managing sensitive data.