استخدام Senior Data Engineer

اسنپ تریپ | Snapp Trip
تهران، تهران

Roles and Responsibilities:

Developing and deploying applications in Scala, Spark(Scala or Python api) and Python to ingest, consume, analyze data in batch or streaming manner.
Proposing solutions for some problematic issues and maintaining the whole Data stack includes PostgreSQL, HDFS, Kafka, Airflow, Metabase, Spark, Kubernetes and PrestoSQL (You will be central for the whole Data stack).
You will support other roles to work easily with open source tools and resolve their performance issues in SQL and Spark

Qualifications:

Bachelor's or a Master’s degree in Computer engineering (Software | AI)
At least three years of practical experience in programming Scala, Java, or Python (preferably a JVM based language)
Having good knowledge of Linux
At least two years of practical experience of developing Spark applications
Having experience of working with relational databases (preferably PostgreSQL)
Having good knowledge of HDFS, SQL and Git
Having experience of working with CI tools (CI/CD or Jenkins)
Familiar with Prometheus/Grafana
Having good understanding of parallel and distributed computing (We run Spark applications deployed on Kubernetes cluster and process data on HDFS
Being proactive
Having effective communication

** Good to have knowledge of Kubernetes and PrestoSQL (Trino)

تمام وقت

۱۴۰۱/۰۴/۱۳ (منقضی‌شده)

مشاغل مشابه