Data Engineer

Experience of working with Real Time/Streaming data
• Ability to Execute BEAM pipelines on Apache FLINK Runner
• Experience of working on Beam SDK
• Knowledge of Microservices using JAVA
• Experience of working on Github
• Experience of using Spark Streaming
• Experience of working with SQL and NOSQL Databases
• Experience of executing or working with Real Time Data pipelines
• Experience of topic setting using Kafka(Confluent).
• Knowledge of real time data pipelines by developing Kafka producers and streaming applications for consuming
• Experience in Kubernetes
• Well versed with CI/CD principles (GitHub, Jenkins etc.), and actively involved in solving troubleshooting issues in distributed services ecosystem
• High level knowledge of compliance and regulatory requirements of data including but not limited to encryption, anonymization, data integrity, policy control features in large scale infrastructures