استخدام Data Engineer
شرح موقعیت شغلی
- Design and develop data pipeline architecture and monitor them
- Contribute to the definition of data architecture standards and establish best practices for data management
- Collaborate with Data Scientists to implement advanced analytics algorithms that exploit our rich data sets for statistical analysis, prediction, clustering and machine learning
- Contribute to the definition of data architecture standards and establish best practices for data management
- Collaborate with Data Scientists to implement advanced analytics algorithms that exploit our rich data sets for statistical analysis, prediction, clustering and machine learning
Requirements:
- Familiar with Big data tools: Hadoop (YARN, HDFS), Spark, Kafka, etc.
- Familiar with Data pipeline and workflow management tools: Airflow, Luigi, etc.
- Familiar with Stream-processing systems: Storm, Spark-Streaming, etc
- Skilled in working Relational SQL and NoSQL database
- Background in at least one programming languages (e.g., R, Python, Java, Scala)
- Experience with data modelling, ETL (Extraction, Transformation & Load) concepts
- Experience with Linux command-line and Linux servers
- Knowledge of version control practices using an existing source control tool
- Familiar with Data pipeline and workflow management tools: Airflow, Luigi, etc.
- Familiar with Stream-processing systems: Storm, Spark-Streaming, etc
- Skilled in working Relational SQL and NoSQL database
- Background in at least one programming languages (e.g., R, Python, Java, Scala)
- Experience with data modelling, ETL (Extraction, Transformation & Load) concepts
- Experience with Linux command-line and Linux servers
- Knowledge of version control practices using an existing source control tool
مهارتهای مورد نیاز
- yarn
- NoSQL
- HDFS
- Hadoop
- SQL
حداقل سابقه کار
- کمتر از سه سال
جنسیت
- مهم نیست
وضعیت نظام وظیفه
- مهم نیست