استخدام Data Engineer
شرح موقعیت شغلی
Responsibilities:
As a data engineer you need to design and build systems for collecting, storing, and analyzing data at
scale. Here, we are using modern and lesser known tools, software and libraries, so you need to be
open to learning and flexible enough to catch up.
Responsibilities:
Designing and building systems for collecting, storing, and analyzing data at scale
Acquire datasets that align with business needs
Collecting data from different data sources, including XML files, JSON files, REST
APIs, NBI, etc.
Develop algorithms to transform data into useful, actionable information
Build, test, and maintain database pipeline architectures
Collaborate with management to understand company objectives
Create new data validation methods and data analysis tools
Ensure compliance with data governance and security policies
General Skills:
Debian based Linux distros
Git
Docker
Docker-Compose
Python programming language
Discipled coding practices
Experienced with code reviews and pull request.
English language proficient
Special Skills:
Python data analysis libraries (pandas, numpy, etc.)
Python SQLAlchemy
PostgreSQL
Additional Skills:
Apache Hadoop framework
Apache Spark
As a data engineer you need to design and build systems for collecting, storing, and analyzing data at
scale. Here, we are using modern and lesser known tools, software and libraries, so you need to be
open to learning and flexible enough to catch up.
Responsibilities:
Designing and building systems for collecting, storing, and analyzing data at scale
Acquire datasets that align with business needs
Collecting data from different data sources, including XML files, JSON files, REST
APIs, NBI, etc.
Develop algorithms to transform data into useful, actionable information
Build, test, and maintain database pipeline architectures
Collaborate with management to understand company objectives
Create new data validation methods and data analysis tools
Ensure compliance with data governance and security policies
General Skills:
Debian based Linux distros
Git
Docker
Docker-Compose
Python programming language
Discipled coding practices
Experienced with code reviews and pull request.
English language proficient
Special Skills:
Python data analysis libraries (pandas, numpy, etc.)
Python SQLAlchemy
PostgreSQL
Additional Skills:
Apache Hadoop framework
Apache Spark
مهارتهای مورد نیاز
- Data engineer
- Python
- SQL
- Git
زبانهای مورد نیاز
- انگلیسی
جنسیت
- مهم نیست
وضعیت نظام وظیفه
- مهم نیست