استخدام Data Engineer
شرح موقعیت شغلی
We are seeking an experienced Data Engineer to join our team in Intellare and play a critical role in designing, building, and maintaining our data pipelines and infrastructure. This position requires a strong background in both data engineering and software development, coupled with excellent communication skills for seamless client interaction.
Responsibilities:
Responsibilities:
- Design, develop, and maintain scalable and fault-tolerant data pipelines using Apache Kafka, Confluent, and other streaming technologies
- Collaborate with data scientists, analysts, and engineers to understand business requirements and translate them into technical solutions
- Implement data ingestion, transformation, and processing workflows using various tools and technologies
- Ensure data quality, integrity, and consistency across multiple data sources
- Optimize data pipelines for performance, reliability, and efficiency
- Develop and maintain data engineering solutions using programming languages such as Python, Scala, or Java
- Work closely with cross-functional teams to understand and address data-related challenges
- Document data pipelines, processes, and architectures for ease of maintenance and knowledge-sharing
Requirements:
- Strong background in data engineering and software development
- Expert knowledge of Apache Kafka and Confluent ecosystem
- Proficiency in programming languages such as Python, Scala, or Java
- Experience with big data technologies (e.g., Hadoop, Spark, Hive)
- Familiarity with cloud platforms (e.g., AWS, GCP, Azure) and containerization technologies (e.g., Docker, Kubernetes)
- Strong understanding of data modeling, ETL/ELT processes, and data warehousing concepts
- Excellent English communication skills, both written and verbal, for effective client interaction
- Ability to work collaboratively in a team environment and mentor junior engineers
- Bachelor's degree in Computer Science, Engineering, or a related field (or equivalent experience)
Preferred Qualifications:
- Experience with streaming data architectures and real-time data processing
- Knowledge of data governance, data lineage, and data security best practices
- Familiarity with machine learning and deep learning frameworks
- Experience with agile software development methodologies
- Certifications in relevant technologies (e.g., Apache Kafka, Confluent, AWS, GCP, Azure)
مهارتهای مورد نیاز
- Python
- SQL
- Scala
- Data engineer
- Java
- spark
حداقل سابقه کار
- مهم نیست
جنسیت
- مهم نیست
وضعیت نظام وظیفه
- مهم نیست