Skip To Main Content
backgo to search

lead big data engineer with databricks

bullets
Data Software Engineering, Amazon Web Services, Apache Airflow, Apache Spark, Databricks, Python, Scala
warning.png
Sorry the job is no longer available.

We are looking for a skilled remote Lead Big Data Engineer with Databricks experience to join our team.

Your primary responsibility will be to lead the development of our ETL pipelines on Databricks using Spark/Scala, Databricks, and AWS. You must have extremely strong experience with Databricks, AWS, Python, and Elastic.

responsibilities
  • Lead the development of ETL pipelines on Databricks using Spark/Scala, Databricks, and AWS
  • Work closely with cross-functional teams to define data requirements
  • Develop and maintain scalable data infrastructure
  • Ensure the reliability, efficiency, and scalability of data pipelines
  • Coach and mentor other members of the team
requirements
  • At least 5 years of experience in Big Data Engineering
  • 1+ years of relevant leadership experience
  • Expertise in Databricks, AWS, Python, and Elastic
  • Deep understanding of Spark/Scala and data architecture principles
  • Knowledge of data models, ETL design, implementation, and maintenance
  • Strong communication skills and ability to collaborate with cross-functional teams
  • B2+ English level
nice to have
  • Experience with Apache Airflow
  • Familiarity with Agile methodologies and SDLC practices

These jobs are for you

benefits for locations

india.svg
For you
  • Insurance Coverage 
  • Paid Leaves – including maternity, bereavement, paternity, and special COVID-19 leaves. 
  • Financial assistance for medical crisis 
  • Retiral Benefits – VPF and NPS 
  • Customized Mindfulness and Wellness programs 
  • EPAM Hobby Clubs
For your comfortable work
  • Hybrid Work Model 
  • Soft loans to set up workspace at home 
  • Stable workload 
  • Relocation opportunities with ‘EPAM without Borders’ program

For your growth
  • Certification trainings for technical and soft skills 
  • Access to unlimited LinkedIn Learning platform 
  • Access to internal learning programs set up by world class trainers 
  • Community networking and idea creation platforms 
  • Mentorship programs 
  • Self-driven career progression tool

get job alerts in your inboxHundreds of open jobs for Software Engineers, QA, DevOps, Business Analysts and other tech professionals
a smiling man wearing sunglasses