senior data engineer (databricks and aws)

Data Software Engineering, Amazon Web Services, Databricks, Python, Apache Airflow, Apache Spark, Scala

Poland

Sorry the job is no longer available.

Senior Data Engineer (Databricks and AWS)

We are looking for a remote Senior Data Engineer to join our project. The candidate should have expert-level skills in AWS and Databricks, as well as extensive knowledge of Big Data technologies such as Spark, Python, Airflow, and Scala/Java. The candidate should have excellent communication skills and be able to work independently as well as in a team of professionals.

responsibilities

Work closely with Business Analysts and data analysts to understand requirements
Automate data ingestion processes and pre-processing
Enhance data sets by creating automated match rules
Develop ETL transformations for data from various data sources
Report activities through agreed project tools and timely report about risks/issues/dependencies

requirements

Minimum of 3 years of experience in Data Software Engineering
Extensive knowledge of AWS and Databricks
Expert-level skills in Python, Airflow and Spark
Experience working with unstructured files and creating automated match rules
Strong problem-solving skills and the ability to work independently or in a team
Good written and verbal communication skills in English (B2+)

nice to have