Skip To Main Content
backgo to search

senior big data engineer with python

bullets
Data Software Engineering, Databricks, Amazon Web Services, Python, SQL
warning.png
Sorry the job is no longer available.

We are looking for an experienced remote Senior Big Data Engineer with a passion for data to develop, monitor, and operate the curated data pipeline for our project.

The ideal candidate has a strong background in Data Software Engineering and extensive experience with DSE Python AWS Databricks and SQL.

In this position, you will be responsible for redeveloping legacy pipelines into new, advanced, and scalable versions, consulting with data scientists and product managers, building and improving KPIs, and maintaining our cloud-based tech stack.

responsibilities
  • Develop, monitor, and operate the most critical curated data pipeline in the project
    • Consult with data scientists and product managers to improve KPIs for business steering
      • Redevelop legacy pipelines into advanced, scalable versions that are easy to maintain
        • Leverage and improve our cloud-based tech stack that includes AWS, Databricks, Kubernetes, Spark, Airflow, Python, and Scala
          • Build, monitor, and maintain Apache Airflow pipelines
            requirements
            • Minimum of 3 years of experience in Data Software Engineering
              • Expertise in Apache Spark, Spark streaming, and Databricks
                • Fluency in Scala programming language and SQL
                  • Experienced working with AWS landscape and Github
                    • Ability to build Apache Airflow pipelines
                      • B2+ English level
                        nice to have
                        • Knowledge of Presto, Superset, and Starburst
                          • Experience with Oracle & Exasol

                            These jobs are for you

                            benefits for locations

                            poland.svg
                            ImageImage
                            For you
                            • Discounts on health insurance, sport clubs, shopping centers, cinema tickets, etc.
                            • Stable income
                            • Flexible roles
                            ImageImage
                            For your comfortable work
                            • 100% remote work forever
                            • EPAM hardware
                            • EPAM software licenses
                            • Access to offices and co-workings
                            • Stable workload
                            • Relocation opportunities
                            • Flexible engagement models
                            ImageImage
                            For your growth
                            • Free trainings for technical and soft skills
                            • Free access to LinkedIn Learning platform
                            • Language courses
                            • Free access to internal and external e-Libraries
                            • Certification opportunities
                            • Skill advisory service
                            get job alerts in your inboxHundreds of open jobs for Software Engineers, QA, DevOps, Business Analysts and other tech professionals
                            a smiling man wearing sunglasses