backgo to search

senior data engineer

bullets
Data Software Engineering, Python, Amazon Web Services, AWS Glue, Amazon EMR, PySpark, Databricks
warning.png
Sorry the job is no longer available.

We are seeking a highly skilled Senior Data Software Engineer to join our remote team, working with a British-American publicly traded analytics company that operates a collection of subscription-based services. As a Senior Data Software Engineer, you will be responsible for developing and deploying production-grade ETL pipelines using Apache Spark. If you are passionate about data software engineering and have a keen eye for detail, we invite you to be part of our team.

responsibilities
  • Develop and deploy production-grade ETL pipelines using Apache Spark (PySpark) to handle large data volumes
  • Contribute to the design, development, and deployment of scalable and reliable data pipelines
  • Perform data analysis and data quality checks to ensure data accuracy and integrity
  • Create and maintain technical documentation for all ETL processes and data pipelines
  • Collaborate with cross-functional teams to review software requirements and ensure seamless integration with other systems
  • Provide technical guidance and mentorship to junior team members
requirements
  • Minimum of 3 years of experience in Data Software Engineering, showcasing your expertise in Python scripting for data handling (usage of pandas, numpy)
  • Strong expertise of Apache Spark (PySpark) for handle large data volumes (few hundred GBs)
  • Experience in writing UDFs in Spark
  • Experience with Relational DB, PostgreSQL/Snowflake preferred
  • Basic knowledge of Big data technologies like HBase and Hive
  • Hands-on experience with AWS Cloud, AWS Glue, Amazon EMR, and Databricks
  • Fluent spoken and written English at an Upper-Intermediate level or higher
nice to have
  • Prior experience on Apache Airflow is desirable
  • Experience in AWS data services like Glue & EMR

These jobs are for you

benefits for locations

poland.svg
ImageImage
For you
  • Discounts on health insurance, sport clubs, shopping centers, cinema tickets, etc.
  • Stable income
  • Flexible roles
ImageImage
For your comfortable work
  • 100% remote work forever
  • EPAM hardware
  • EPAM software licenses
  • Access to offices and co-workings
  • Stable workload
  • Relocation opportunities
  • Flexible engagement models
ImageImage
For your growth
  • Free trainings for technical and soft skills
  • Free access to LinkedIn Learning platform
  • Language courses
  • Free access to internal and external e-Libraries
  • Certification opportunities
  • Skill advisory service
get job alerts in your inboxHundreds of open jobs for Software Engineers, QA, DevOps, Business Analysts and other tech professionals
a smiling man wearing sunglasses