backgo to search

senior data engineer

Data Software Engineering, Python, Amazon Web Services, AWS Glue, Amazon EMR, PySpark, Databricks
Sorry the job is no longer available.

We are seeking a highly skilled Senior Data Software Engineer to join our remote team, working with a British-American publicly traded analytics company that operates a collection of subscription-based services. As a Senior Data Software Engineer, you will be responsible for developing and deploying production-grade ETL pipelines using Apache Spark. If you are passionate about data software engineering and have a keen eye for detail, we invite you to be part of our team.

  • Develop and deploy production-grade ETL pipelines using Apache Spark (PySpark) to handle large data volumes
  • Contribute to the design, development, and deployment of scalable and reliable data pipelines
  • Perform data analysis and data quality checks to ensure data accuracy and integrity
  • Create and maintain technical documentation for all ETL processes and data pipelines
  • Collaborate with cross-functional teams to review software requirements and ensure seamless integration with other systems
  • Provide technical guidance and mentorship to junior team members
  • Minimum of 3 years of experience in Data Software Engineering, showcasing your expertise in Python scripting for data handling (usage of pandas, numpy)
  • Strong expertise of Apache Spark (PySpark) for handle large data volumes (few hundred GBs)
  • Experience in writing UDFs in Spark
  • Experience with Relational DB, PostgreSQL/Snowflake preferred
  • Basic knowledge of Big data technologies like HBase and Hive
  • Hands-on experience with AWS Cloud, AWS Glue, Amazon EMR, and Databricks
  • Fluent spoken and written English at an Upper-Intermediate level or higher
nice to have
  • Prior experience on Apache Airflow is desirable
  • Experience in AWS data services like Glue & EMR

These jobs are for you

benefits for locations

For you
  • Discounts on health insurance, sport clubs, shopping centers, cinema tickets, etc.
  • Stable income
  • Flexible roles
For your comfortable work
  • 100% remote work forever
  • EPAM hardware
  • EPAM software licenses
  • Access to offices and co-workings
  • Stable workload
  • Relocation opportunities
  • Flexible engagement models
For your growth
  • Free trainings for technical and soft skills
  • Free access to LinkedIn Learning platform
  • Language courses
  • Free access to internal and external e-Libraries
  • Certification opportunities
  • Skill advisory service
get job alerts in your inboxHundreds of open jobs for Software Engineers, QA, DevOps, Business Analysts and other tech professionals
a smiling man wearing sunglasses