ir a búsquedasenior data engineer
We are seeking a highly skilled Senior Data Software Engineer to join our remote team, working with a British-American publicly traded analytics company that operates a collection of subscription-based services. As a Senior Data Software Engineer, you will be responsible for developing and deploying production-grade ETL pipelines using Apache Spark. If you are passionate about data software engineering and have a keen eye for detail, we invite you to be part of our team.
responsibilities
- Develop and deploy production-grade ETL pipelines using Apache Spark (PySpark) to handle large data volumes
- Contribute to the design, development, and deployment of scalable and reliable data pipelines
- Perform data analysis and data quality checks to ensure data accuracy and integrity
- Create and maintain technical documentation for all ETL processes and data pipelines
- Collaborate with cross-functional teams to review software requirements and ensure seamless integration with other systems
- Provide technical guidance and mentorship to junior team members
requirements
- Minimum of 3 years of experience in Data Software Engineering, showcasing your expertise in Python scripting for data handling (usage of pandas, numpy)
- Strong expertise of Apache Spark (PySpark) for handle large data volumes (few hundred GBs)
- Experience in writing UDFs in Spark
- Experience with Relational DB, PostgreSQL/Snowflake preferred
- Basic knowledge of Big data technologies like HBase and Hive
- Hands-on experience with AWS Cloud, AWS Glue, Amazon EMR, and Databricks
- Fluent spoken and written English at an Upper-Intermediate level or higher
nice to have
- Prior experience on Apache Airflow is desirable
- Experience in AWS data services like Glue & EMR
These jobs are for you
beneficios por ubicaciones
Descuentos en seguros de salud, clubes deportivos, centros comerciales, entradas de cine, etc.
Ingreso estable
Roles flexibles
Trabajo remoto al 100% para siempre
Hardware de EPAM
Licencias de software de EPAM
Acceso a oficinas y espacios de coworking
Carga de trabajo estable
Oportunidades de reubicación
Modelos de participación flexibles
Formaciones gratuitas en habilidades técnicas y blandas
Acceso gratuito a la plataforma de aprendizaje de LinkedIn
Cursos de idiomas
Acceso gratuito a bibliotecas internas y externas
Oportunidades de certificación
Servicio de asesoramiento de habilidades
Envíanos tu CV para recibir una oferta personalizada