We are building a modern, on-premise data platform from the ground up to power analytics, risk management, compliance, and AI-driven insights across our brokerage operations. This is a rare opportunity to architect a greenfield lakehouse platform end-to-end — from ingestion through governance to an AI-powered analytics agent — in a regulated financial environment with real budget and headcount to build it right. We're looking for a hands-on Senior Data Platform Engineer who has done this before: someone who can make sound architectural trade-offs, mentor a growing team, and isn't afraid to operate complex distributed systems in production.
What You'll Work On
Design and operate a Lakehouse architecture (Medallion: Bronze/Silver/Gold) on Apache Iceberg with Nessie as a versioned catalog, backed by on-premise object storage (MinIO)
Build and maintain CDC pipelines from multiple OLTP databases using Debezium and Kafka Connect
Own the Kafka-based streaming backbone serving both batch ingestion and real-time (sub-second) use cases such as market data and trading event processing
Develop stateful stream processing with Apache Flink for real-time risk exposure calculations, anomaly detection, and alerting
Build and maintain transformation pipelines with dbt, orchestrated via Airflow
Support analytical query workloads on Trino and BI delivery through Superset
Design and enforce data governance: cataloging and lineage (OpenMetadata/DataHub), fine-grained access control (Apache Ranger), and audit logging suitable for a regulated environment
Deploy and operate platform components across a hybrid Kubernetes/VM footprint (stateless services on K8s, stateful services such as Kafka and MinIO on VMs)
Contribute to capacity planning, on-call practices, and platform reliability as the system scales toward production-critical status
Must-Have Qualifications
5+ years of experience building and operating production data infrastructure
Strong hands-on experience with Apache Kafka (including Kafka Connect and CDC patterns, ideally Debezium)
Practical experience with a modern table format (Apache Iceberg or Delta Lake) and lakehouse patterns
Proficiency with a distributed SQL query engine (Trino, Presto, or similar)
Experience with dbt for data transformation and modeling
Experience with workflow orchestration tools (Airflow or equivalent)
Solid understanding of Kubernetes for deploying and operating stateful and stateless workloads
Strong SQL and at least one of Python/Scala/Java for pipeline development
Demonstrated experience designing data architectures with data governance, security, and auditability as first-class concerns
Strongly Preferred
Experience with Apache Flink or another stream processing engine for stateful, low-latency workloads
Experience with data catalog/lineage tools (OpenMetadata, DataHub, or Amundsen)
Experience with fine-grained access control systems (Apache Ranger or equivalent)
Prior experience in financial services, trading, or another regulated industry with strict data residency requirements
Experience operating fully on-premise (non-cloud) infrastructure at scale
Experience with Nessie or git-like data versioning for lakehouse catalogs
What Makes This Role Different
Greenfield build: you'll shape the architecture, not inherit legacy debt
Strict on-premise requirement means solving real distributed-systems problems without leaning on managed cloud services
Direct exposure to both the batch/analytical side and the real-time/streaming side of the platform
توسعه فناوری گروه فارابی از سال ۱۳۹۲ با یک هدف روشن پا به عرصه گذاشت: ساختن بستری که بتواند مسیرهای مالی و سرمایهگذاری را برای همه آسانتر کند.
در این سالها، همیشه به دنبال نوآوری و ارائه راهکارهایی بودیم که کاربران را یک قدم به آیندهای مطمئنتر نزدیک کند. از پلتفرم فارابیکسو که معاملات بورس را روانتر میکند، تا فارابیزون که سرمایهگذاری را سادهتر و کارآمدتر میسازد. جیبیمو هم راهی سریع و ایمن برای پرداختهای الکترونیکی فراهم کرده، و پینوست به عنوان یک دستیار مالی هوشمند، کاربران را در تصمیمات مالی همراهی میکند.
ما در توسعه فناوری گروه فارابی باور داریم که فناوری میتواند تحولی بزرگ در دنیای مالی ایجاد کند، و هر روز تلاش میکنیم تا ابزارهایی بسازیم که زندگی مالی شما را راحتتر و هوشمندتر کند.