backgo to search

lead big data engineer with scala expertise

We are looking for a remote Lead Big Data with 5+ years of experience as a Java Developer, proficiency in data engineering, expertise in Big Data - Hadoop, Spark, and Kafka, and with strong knowledge of Scala to join our team remotely.


Our customer exposes an API for other businesses to be able to access risks associated with a person, e.g. understand their credit score. Behind the scene there is a sophisticated decisioning system and large data volumes. Currently, back-end of this API has number of legacy versions, serving hundreds of clients, with individual installations for most clients.

We need to create a data lake for one of the biggest data analytics company working with personal information both domestically and internationally. In a nutshell, this includes replatforming of on-premise Enterprise Data Hub from Hadoop cluster into GCP. Day to day tasks include but not limited to creating spark application that manipulates with data from different sources including Oracle, Google Cloud Storage, BigQuery; creating pipelines via GCP Dataflow; working with Jenkins and AirFlow.

requirements
  • 5+ years of experience as a Java Developer
    • Proficiency in data engineering
      • Expertise in Big Data: Hadoop, Spark, Kafka
        • Strong knowledge of Scala
          • Expertise in Microservices Architecture
            • Ability to work with high volumes of data
              • Experience in working with AWS
                • Experience in working with GCP: Dataproc (Apache Spark), Dataflow (Apache Beam), BigQuery, Cloud Storage
                  • Good understanding of Design Patterns, Clean Code, Unit testing
                    • Experience working in Agile environment
                      • Data modelling skills would be a plus
                        • Experience in Jenkins, AirFlow with Groovy and Python
                          • Excellent written and verbal communication skills
                            • Intermediate or higher English level, both verbal and written (B1+)

                              benefits for locations

                              location.svg

                              For you

                              • Paid time off
                              • Paid sick leave days
                              • Stable income
                              • Meal and home office compensation

                              For your comfortable work

                              • 100% remote work 
                              • Hybrid work opportunities
                              • Free licensed software
                              • Possibility to work on your own device (BYOD)
                              • Stable workload
                              • Relocation opportunities
                              • Flexible engagement models

                              For your growth

                              • Free trainings for technical and soft skills
                              • Free access to LinkedIn Learning platform
                              • Free access to internal and external e-Libraries
                              • Certification opportunities
                              • Skill advisory service
                              • Language courses
                              subscribe to EPAM Anywhere vacancies!Hundreds of open jobs for Software Engineers, QA, DevOps, Business Analysts and other tech professionals
                              Girl in front of laptop

                              looking for something else?

                              Find a vacancy that works for you. Send us your CV to receive a personalized offer.