backgo to search

lead data software engineer

bullets
Data Software Engineering, Python, Amazon Web Services, Apache Spark, Databricks, Apache Airflow
warning.png
Sorry the job is no longer available.

We are looking for a highly skilled Lead Data Software Engineer to join our remote team, working with a multinational e-commerce company specializing in shoes and fashion. As a Lead Data Software Engineer, you will be responsible for designing, developing, and maintaining data pipelines and data infrastructure using Apache Spark, Databricks, and other data engineering tools. You will work closely with the Data Science team to provide high-quality data solutions that support business decisions and drive growth.

responsibilities
  • Design, develop, and maintain data pipelines and data infrastructure using Apache Spark, Databricks, and other data engineering tools
  • Collaborate with the Data Science team to provide high-quality data solutions that support business decisions and drive growth
  • Optimize and tune data pipelines for performance, scalability, and reliability
  • Ensure data quality and integrity throughout the data pipeline, from data ingestion to data consumption
  • Implement and maintain CI/CD pipelines for data engineering workflows
  • Develop and maintain documentation for data engineering processes and data infrastructure
  • Provide technical leadership and mentorship to junior data engineers
requirements
  • 5+ years of experience in Data Software Engineering, showcasing your expertise in designing and developing complex data solutions
  • 1+ years of experience in a leadership role, demonstrating your ability to lead a team of engineers
  • Expertise in Apache Spark along with Spark streaming & Spark SQL, showcasing your ability to design and develop complex data pipelines
  • Fluency working with AWS landscape, including EC2, S3, and EMR, among others
  • Good hands-on experience with Databricks and delta-lake, enabling you to manage large-scale data infrastructures
  • Good understanding & hands-on experience with CI/CD, enabling you to automate software development processes
  • Rich working experience with Github, demonstrating your proficiency in version control and collaboration tools
nice to have
  • Familiarity with Presto, Superset, Starburst, or Exasol, demonstrating your broader knowledge of data engineering tools and technologies

These jobs are for you

benefits for locations

india.svg
For you
  • Insurance Coverage 
  • Paid Leaves – including maternity, bereavement, paternity, and special COVID-19 leaves. 
  • Financial assistance for medical crisis 
  • Retiral Benefits – VPF and NPS 
  • Customized Mindfulness and Wellness programs 
  • EPAM Hobby Clubs
For your comfortable work
  • Hybrid Work Model 
  • Soft loans to set up workspace at home 
  • Stable workload 
  • Relocation opportunities with ‘EPAM without Borders’ program

For your growth
  • Certification trainings for technical and soft skills 
  • Access to unlimited LinkedIn Learning platform 
  • Access to internal learning programs set up by world class trainers 
  • Community networking and idea creation platforms 
  • Mentorship programs 
  • Self-driven career progression tool

get job alerts in your inboxHundreds of open jobs for Software Engineers, QA, DevOps, Business Analysts and other tech professionals
a smiling man wearing sunglasses