senior data engineer

Data Software Engineering, Amazon Web Services, Apache Spark, Python, SQL, Amazon Athena, Amazon Redshift

Poland

Sorry the job is no longer available.

Senior Data Engineer

We are seeking a talented Senior Data Software Engineer to join our remote team, working with a leading agriculture company that helps to improve global food security by enabling millions of farmers to make better use of available resources. As a Senior Data Software Engineer, you will be responsible for designing and implementing data processing pipelines, ensuring data quality, and optimizing data delivery. You will work with an AWS platform for running data pipelines, which is a serverless AWS-based data lake that loads data from numerous DBs. If you are passionate about data engineering and have a strong background in AWS, Apache Spark, and Python, we invite you to join our team.

responsibilities

Design and implement data processing pipelines, ensuring data quality and optimizing data delivery
Develop and maintain data processing infrastructure using AWS services
Optimize data processing pipelines for performance and scalability
Troubleshoot and resolve issues with data processing pipelines
Ensure data security and compliance with data protection regulations
Document data processing pipelines and related processes
Collaborate with cross-functional teams to understand business requirements and develop data solutions that meet those requirements

requirements

A minimum of 3 years of experience in Data Software Engineering, demonstrating your expertise in designing and implementing data processing pipelines
Strong experience with Amazon Web Services, including AWS Lambda, S3, EC2, Athena and Redshift
Proficiency in Apache Spark, Python, and SQL, enabling you to develop robust and scalable data processing solutions
Excellent communication skills and strong critical thinking capabilities, enabling you to collaborate effectively with cross-functional teams
Ability to work independently and manage stress effectively, maintaining a high level of performance even under pressure
Fluent spoken and written English at an Upper-Intermediate level or higher, enabling effective communication

nice to have