As a Data Engineer you will develop and maintain our Data platform and improve existing architecture for incoming projects. Your passion is to work with the latest and greatest technologies that make working with large amounts of data easy. You’re pro-active in keeping yourself up to date and are always searching for new ways to discover new technologies. You also enjoy laying the architectural foundations for the things you’re working on. You combine both thinking of the future and a hands on, right now attitude. You will work in a team with highly skilled people and enjoy a creative atmosphere where trying things out is encouraged.
Responsibilities
Be a core part of our analytics team and take ownership of managing and extending our data pipelines
Design, implement, and maintain effective and consistent ETL processes that scale using tools such as Apache Kafka, Apache Nifi, LogStash, Flume, etc.
Design and implement data model in RDBMS-style and NoSQL databases.
Design and implement batch jobs using MapReduce and stream processing using Apache Kafka and Apache Spark
Address performance problems in databases through query optimization, indexing, sharding, etc.
Design, develop and maintain software that analyzes large volume of data employing big data tools
Work closely with our analytics, data science, and product teams to ensure their data storage/processing needs are met
Constantly explore and evaluate the implementation of new technologies to address infrastructure scalability and performance challenges
Enable integration and deployment tools and methods for managing reliable development life cycle routines
Prepare documentation conveying design and support information
Qualifications:
BS or MS in Computer Science/Engineering or equivalent relevant experience
Good knowledge on big data concepts such as well-known architectures, processing models, indexing and its types, etc.
Good knowledge and experience of Python or Scala software development
Familiarity with Java Core programming and OOD principles.
Experience with ETL facilitating tools such as Apache related tools, i.e. Flume and Nifi and also other tools like Fluent’d, LogStash
Experience in setting up both SQL as well as NoSQL databases such as Redis, HBase, Impala, etc.
Practical experience in using Big Data technologies such as Hadoop, Kafka and Spark
Deployment and provisioning automation tools e.g. Docker, kubernetes, CI/CD, Ansible.
Experience with version control (Git, TFS, JIRA, etc.) and test-driven development
Familiar to bash scripting and Linux systems administration
معرفی شرکت
اسنپ محصولی ایرانی است که از تیمی خلاق، جوان و تحصیلکرده قدرت میگیرد و در تلاش است تا صنعت فناوری اطلاعات و ارتباطات را به زندگی روزمره جامعه پیوند بزند.
اهداف بزرگی در سر داریم و بلند پروازیم. قصد داریم اسنپ را به بهترین راهکار برای سفرهای درونشهری ایران تبدیل کنیم و در این راه به کمک افراد خلاق، سختکوش و بلندپرواز احتیاج داریم. اگر چنین خصوصیاتی دارید خوشحال میشویم که رزومه خود را برایمان ارسال کنید.