senior data engineer

Develop, monitor, and operate the most used and most critical curated data pipeline - Sales Order Data (incl. Post-order information, e.g. shipment, return, payment). This pipeline is processing hundreds of millions of records to provide high-quality datasets for analytical and machine learning use-cases

Consulting with analysts, data scientists, and product managers to build and continuously improve "Single Source of Truth" KPI for business steering such as the central Profit Contribution measurement (PC II)

Leverage and improve a cloud-based tech stack that includes AWS, Databricks, Kubernetes, Spark, Airflow, Python, and Scala

3+ years of relevant experience

Expertise in Apache Spark along with Spark streaming & Spark SQL

Good hands on experience with Databricks and delta-lake

Fluency in Scala programming language

Good understanding & hands-on experience with CI/CD

Rich working experience with Github

Fluency working with AWS landscape

Ability to build Apache Airflow pipelines

Upper-intermediate or higher English level, both spoken and written (B2+)

Presto

Superset

Starburst

Exasol