go to searchsenior big data engineer with python
Senior Big Data Engineer with Python
We are looking for an experienced remote Senior Big Data Engineer with a passion for data to develop, monitor, and operate the curated data pipeline for our project.
The ideal candidate has a strong background in Data Software Engineering and extensive experience with DSE Python AWS Databricks and SQL.
In this position, you will be responsible for redeveloping legacy pipelines into new, advanced, and scalable versions, consulting with data scientists and product managers, building and improving KPIs, and maintaining our cloud-based tech stack.
responsibilities
- Develop, monitor, and operate the most critical curated data pipeline in the project
- Consult with data scientists and product managers to improve KPIs for business steering
- Redevelop legacy pipelines into advanced, scalable versions that are easy to maintain
- Leverage and improve our cloud-based tech stack that includes AWS, Databricks, Kubernetes, Spark, Airflow, Python, and Scala
- Build, monitor, and maintain Apache Airflow pipelines
requirements
- Minimum of 3 years of experience in Data Software Engineering
- Expertise in Apache Spark, Spark streaming, and Databricks
- Fluency in Scala programming language and SQL
- Experienced working with AWS landscape and Github
- Ability to build Apache Airflow pipelines
- B2+ English level
nice to have
- Knowledge of Presto, Superset, and Starburst
- Experience with Oracle & Exasol
benefits for locations
Poland
For you
- Discounts on health insurance, sport clubs, shopping centers, cinema tickets, etc.
- Stable income
- Flexible roles
For your comfortable work
- 100% remote work forever
- EPAM hardware
- EPAM software licenses
- Access to offices and co-workings
- Stable workload
- Relocation opportunities
- Flexible engagement models
For your growth
- Free trainings for technical and soft skills
- Free access to LinkedIn Learning platform
- Language courses
- Free access to internal and external e-Libraries
- Certification opportunities
- Skill advisory service
don't have time? Apply later!We send you a link to the job in your e-mail
get job alerts in your inboxHundreds of open jobs for Software Engineers, QA, DevOps, Business Analysts and other tech professionals
