استخدام Data Engineer
شرح موقعیت شغلی
Our Data team at Haf-hashtad is looking for a member as "Data Engineer" to implement, test, verify and keep
infrastructural additives that record data flow and design. Realistically, the position of a data engineer may be
blended into a single person with a set of capabilities that could be very close.
Data Engineering often includes data design, build, and installation of the data systems as part of their
responsibilities. Data Engineers fuel machine learning and AI analytics in collaboration with Data scientists in the
data teams including data acquisition, data transformation, data modeling, and more around data processing.
Role and Responsibilities
Develop, construct, test, verify and maintain data pipelines.
Develop and maintain data APIs.
Design, install and maintain data modeling and ETL tools.
Design, implement and maintain data warehouses.
Collect, store, archive, and aggregate raw and processed data from various sources.
Build data infrastructure and flows.
Communicate with DevOps team for building data infrastructure.
Communicate with Data scientists to provide required data.
Communicate with other data team members to further improve the system.
Required Skill
Bachelor’s degree in computer science, IT, or relevant fields or equivalent work experience.
Strong understanding of standard SQL (preferably PL/SQL, or PostgreSQL)
Strong understanding of indexing and partitioning concepts
Good knowledge of query building and tuning techniques like sub-querying, CTEs, windowing, pivoting, etc.
Good knowledge of common data types and structures
Experience with Oracle Database, Oracle SQL Developer and Oracle Data Modeler
Good knowledge of Data Quality, Operations, Governance and OLTP concepts
Experience with relational and NoSQL databases like Oracle Database, PostgreSQL, Redis, and MongoDB
Good knowledge of a programming language like Python, Java
Experience with containerization and orchestration
Understanding of message brokers or stream services like Kafka, RabbitMQ, NATS
Work proficiency in Linux and Bash, and Windows and PowerShell
Experience with version controlling, especially git and GitLab (GitLab Runners and CI)
Familiarity with Big Data and its toolsets like Hadoop, Hive, Spark, and Pig
At least 3 years of experience in relevant fields
Proficiency in English
Communication skills and teamwork mindset
Preferred Skills
Experience with Kubernetes and designing Helm charts.
Experience with Infrastructure as Code, e.g., Ansible
Experience with CI/CD, e.g., GitLab Runners and CI
Experience with building optimized Docker images
Good understanding of networking principles
Familiar with Oracle Data Integrator, Airflow, Oracle Analytics, SSIS, and Power BI
Familiar with Big Data and its toolsets like Hadoop, Hive, Spark, and Pig
Experience in DataOps
Good understanding of Data Driven design patterns and organizations
Familiar with Data Lake concepts
Familiar with journaling, redoes, and flashbacks
infrastructural additives that record data flow and design. Realistically, the position of a data engineer may be
blended into a single person with a set of capabilities that could be very close.
Data Engineering often includes data design, build, and installation of the data systems as part of their
responsibilities. Data Engineers fuel machine learning and AI analytics in collaboration with Data scientists in the
data teams including data acquisition, data transformation, data modeling, and more around data processing.
Role and Responsibilities
Develop, construct, test, verify and maintain data pipelines.
Develop and maintain data APIs.
Design, install and maintain data modeling and ETL tools.
Design, implement and maintain data warehouses.
Collect, store, archive, and aggregate raw and processed data from various sources.
Build data infrastructure and flows.
Communicate with DevOps team for building data infrastructure.
Communicate with Data scientists to provide required data.
Communicate with other data team members to further improve the system.
Required Skill
Bachelor’s degree in computer science, IT, or relevant fields or equivalent work experience.
Strong understanding of standard SQL (preferably PL/SQL, or PostgreSQL)
Strong understanding of indexing and partitioning concepts
Good knowledge of query building and tuning techniques like sub-querying, CTEs, windowing, pivoting, etc.
Good knowledge of common data types and structures
Experience with Oracle Database, Oracle SQL Developer and Oracle Data Modeler
Good knowledge of Data Quality, Operations, Governance and OLTP concepts
Experience with relational and NoSQL databases like Oracle Database, PostgreSQL, Redis, and MongoDB
Good knowledge of a programming language like Python, Java
Experience with containerization and orchestration
Understanding of message brokers or stream services like Kafka, RabbitMQ, NATS
Work proficiency in Linux and Bash, and Windows and PowerShell
Experience with version controlling, especially git and GitLab (GitLab Runners and CI)
Familiarity with Big Data and its toolsets like Hadoop, Hive, Spark, and Pig
At least 3 years of experience in relevant fields
Proficiency in English
Communication skills and teamwork mindset
Preferred Skills
Experience with Kubernetes and designing Helm charts.
Experience with Infrastructure as Code, e.g., Ansible
Experience with CI/CD, e.g., GitLab Runners and CI
Experience with building optimized Docker images
Good understanding of networking principles
Familiar with Oracle Data Integrator, Airflow, Oracle Analytics, SSIS, and Power BI
Familiar with Big Data and its toolsets like Hadoop, Hive, Spark, and Pig
Experience in DataOps
Good understanding of Data Driven design patterns and organizations
Familiar with Data Lake concepts
Familiar with journaling, redoes, and flashbacks
مهارتهای مورد نیاز
- Data engineer
- spark
- Hadoop
زبانهای مورد نیاز
- انگلیسی
حداقل سابقه کار
- سه تا شش سال
جنسیت
- مهم نیست
وضعیت نظام وظیفه
- مهم نیست