استخدام Senior Data Engineer
شرح موقعیت شغلی
Responsibilities:
- Setup large-scale and high-performance data pipelines (batch and stream) handling billions of records using Apache Kafka and Apache Flink.
- Maintain, monitor, and debug data services, big data platforms, and real-time analytics stack in order to make sure of their availability and reliability.
- Collaborating with data scientists, software engineers, DevOps engineers, BI team, etc in order to build a fit data platform to achieve the business goals.
- Propose new data architectures for new requirements and fine-tune the existing ones.
Requirements:
- Fluent in Java and Python (Familiarity with Scala is a bonus)
- Experienced with SQL and NoSQL databases such as MySQL, PostgreSQL, Elasticsearch, MongoDB, Redis, etc.
- Ability to design data pipelines and ETL solutions.
- Familiar with data warehouse technologies.
- Experienced with Big Data tools like Apache Hadoop, Apache Spark, etc.
- Experienced with stream processing technologies such as Apache Kafka, Apache Flink, Apache Beam, etc.
- Experienced with Linux machines and Docker (Familiarity with Kubernetes and KubeFlow is a bonus)
- Familiar with ELK stack.
- Familiar with monitoring systems such as Grafana and Prometheus.
مهارتهای مورد نیاز
- Data engineer
- Big Data
- Python
- NoSQL
- ETL
حداقل سابقه کار
- سه تا شش سال
جنسیت
- مهم نیست
وضعیت نظام وظیفه
- مهم نیست