استخدام Data Engineer
شرح موقعیت شغلی
As a Data Engineer, your goal is to provide an efficient, stable platform that reliably and predictably delivers accurate, high-quality data to other teams within the company including Data Wrangling, Data Science, and Business. You will collaborate with the Data Science team to build and extend procedures, tools, pipelines, and systems necessary for productionizing insights and analyses.
Responsibilities:
Responsibilities:
- Build the infrastructure required for optimal Extraction, Transformation, and Loading of (non)structured data from a wide variety of data sources
- Data Acquisition and Extraction from various sources using provided API or other solutions like web scrapping
- Building data pipeline systems in event-driven or streaming architectures using systems such as Kafka or other equivalents
- Building AI Model pipelines
- Deployment and management of systems used for data processing and analytics
- Building analytics tools that utilize the data pipelines and model artifacts to provide actionable insights
Requirements :
- At least Bachelor in computer science, physics, statistics, and other data related majors
- 2+ years experience of working as a data engineer or other data related positions
- Familiarity with source and data version control
- 2+ years experience in Python or Golang
- Experience with relational SQL and NoSQL databases, including Postgres, Elasticsearch, and MongoDB.
- Experience with data processing systems such as Apache Spark and Dask
- Experience with stream-processing systems such as Spark-Streaming
- Experience with AI Model Lifecycle Management paradigms and systems such as MLflow
- Experience with visualization techniques and frameworks such as Dash and Bokeh
مهارتهای مورد نیاز
- Data engineer
- SQL
- NoSQL
- spark
حداقل سابقه کار
- کمتر از سه سال
جنسیت
- مهم نیست
وضعیت نظام وظیفه
- مهم نیست