Data Engineer

Job Title: Data Engineer

Overview:

We are looking for a skilled and experienced Data Engineer to join our team. The Data Engineer will be responsible for designing building and maintaining scalable data pipelines to support our data analytics and machine learning initiatives. This role requires expertise in data engineering technologies strong programming skills and a deep understanding of data architecture principles.

Responsibilities:

Design develop and maintain robust and scalable data pipelines for ingesting processing and transforming large volumes of data from various sources.

Implement data integration solutions to extract load and transform data from disparate systems and databases.

Develop and optimize ETL processes to cleanse transform and enrich raw data for analysis and reporting purposes.

Ensure data quality and integrity throughout the ETL lifecycle through validation and errorhandling mechanisms.

Document data engineering processes data flows and system configurations for reference and knowledge sharing.

Requirements

Qualifications:

Bachelors degree in Computer Science Engineering or related field; advanced degree preferred.

5 years of experience in data engineering or related roles with a focus on designing and building data pipelines and infrastructure.

Proficiency in programming languages such as Python Java Scala etc.

Experience with distributed computing frameworks such as Apache Hadoop Spark or Flink.

Knowledge of cloud platforms and services such as AWS Google Cloud Platform or Microsoft Azure.

Familiarity with containerization technologies such as Docker and orchestration tools like Kubernetes.

Excellent problemsolving and analytical skills.

Strong communication and collaboration skills.

Ability to work effectively in a fastpaced dynamic environment.

Preferred Qualifications:

Experience with stream processing frameworks such as Apache Kafka or Apache Beam.

Knowledge of NoSQL databases such as MongoDB Cassandra or Redis.

Experience with data visualization tools such as Tableau Power BI or Looker.

Certification in cloud platforms or data engineering technologies.

Experience working in an Agile or DevOps environment.