senior big data engineer with databricks

Data Software Engineering, Amazon Web Services, Apache Airflow, Apache Spark, Databricks, Python, Scala

Poland

Facebook LinkedIn Twitter Send via email

Senior Big Data Engineer with Databricks

Facebook LinkedIn Twitter Send via email

We are seeking a remote Senior Big Data Engineer with expertise in Databricks to join our innovative team.

The ideal candidate should have extensive experience in Spark/Scala and AWS, as well as a solid foundation in Python.

In this position, you will be responsible for designing, developing, and deploying large-scale data processing systems with Databricks. You will also be responsible for optimizing data pipelines, monitoring performance, and resolving any related issues. The successful candidate should be able to collaborate closely with other teams, evaluate technical trade-offs, and ensure high-quality solutions are delivered.

responsibilities

Design, develop and maintain high-performance data processing systems using Databricks
Optimize data pipelines and monitor their performance regularly
Identify and troubleshoot issues that occur during data processing
Collaborate with other teams to integrate data solutions into existing platforms
Evaluate technical trade-offs and provide recommendations to improve the platform's performance and scalability

requirements

Minimum 3 years of relevant experience in Data Software engineering
Expertise in DSE Python, AWS, and Databricks
Solid foundation in Spark/Scala and Elastic
Excellent communication skills and the ability to collaborate with cross-functional teams
Fluent in English with an Upper-Intermediate level (B2+)

nice to have