Skip To Main Content
backgo to search

lead data engineer pyspark and python

bullets
Data Software Engineering, Python, Apache Spark, Python.Core, Agile, Databricks, PySpark, SQL, Cloud Pipeline, Gradle, DevOps
warning.png
Sorry the job is no longer available.

We are seeking a remote Lead Data Engineer with expertise in PySpark and Python to join our team. The successful candidate will be working primarily with big data engineering. The main project focuses on pipeline design and development in the FS-Insurance sector.

responsibilities
  • Design and develop data pipelines
  • Collaborate with cross-functional teams to develop and implement data solutions
  • Ensure data quality and reliability
  • Lead and mentor junior data engineers
  • Utilize Agile/Scrum software development methodologies
requirements
  • At least 5 years of experience in Big Data engineering
  • Expertise in DSE Python, AWS and Databricks
  • Strong proficiency in PySpark, Spark and Python programming
  • Good communication skills
  • Proficiency in SQL (Spark SQL preferred)
  • Experience with distributed computing on enterprise data platforms
  • Upper-Intermediate English level (B2+)
nice to have
  • Knowledge of Java and Gradle
  • Familiarity with JavaScript/HTML/CSS

These jobs are for you

benefits for locations

india.svg
For you
  • Insurance Coverage 
  • Paid Leaves – including maternity, bereavement, paternity, and special COVID-19 leaves. 
  • Financial assistance for medical crisis 
  • Retiral Benefits – VPF and NPS 
  • Customized Mindfulness and Wellness programs 
  • EPAM Hobby Clubs
For your comfortable work
  • Hybrid Work Model 
  • Soft loans to set up workspace at home 
  • Stable workload 
  • Relocation opportunities with ‘EPAM without Borders’ program

For your growth
  • Certification trainings for technical and soft skills 
  • Access to unlimited LinkedIn Learning platform 
  • Access to internal learning programs set up by world class trainers 
  • Community networking and idea creation platforms 
  • Mentorship programs 
  • Self-driven career progression tool

get job alerts in your inboxHundreds of open jobs for Software Engineers, QA, DevOps, Business Analysts and other tech professionals
a smiling man wearing sunglasses