ir a búsqueda
We are actively looking for a Lead Java Data Pipeline Engineer to join our remote team. The primary focus of this exciting project involves developing and implementing robust data pipelines using Java and Apache Beam in the Google DataFlow environment, to process extensive volumes of data effectively.In this role, your responsibilities include designing and optimizing data pipelines for performance and scalability. You will work collaboratively with data scientists and analysts to comprehend data requirements and implement solutions that aid data modeling, mining, and extraction processes. Additionally, your role entails working with the Spring Framework supporting our pipelines and using Google Cloud Platform services to deploy and manage data pipelines, ensuring their high availability and reliability.
responsibilities
- Design and implement robust data pipelines using Java and Apache Beam in the Google DataFlow environment
- Co-operate with data scientists and analysts to understand data requirements and implement solutions that assist data modeling, mining, and extraction processes
- Optimize existing data pipelines for performance and scalability, identify bottlenecks and implement improvements
- Develop and maintain documentation regarding data pipeline architectures, design decisions, and operational procedures
- Ensure data integrity and compliance with data governance and security policies throughout the data processing lifecycle
- Monitor pipeline performance and implement logging and alerting mechanisms to preemptively address issues
- Stay abreast of the latest technologies and framework advancements in data processing, exploring new tools and practices to enhance pipeline efficiency and functionality
- Guide junior engineers on best practices in data pipeline development and cloud-based data processing
requirements
- Bachelor's or Master's degree in Computer Science, Engineering, Information Technology, or a related field
- Minimum 5 years of experience in data pipeline development, a robust background in Java and cloud-based data processing technologies is essential
- At least 1 year of relevant leadership experience
- Strong proficiency in Java programming with experience developing large-scale data processing solutions
- Solid understanding of the Spring Framework, including Spring Boot, for building high-performance applications
- Hands-on experience with Google DataFlow and Apache Beam for building and managing data pipelines
- Experience with GCP services, particularly those related to data storage, processing, and analytics
- Proficiency in using version control systems, such as Git, for code management and collaboration
- Fluency in verbal and written English at a B2+ level
These jobs are for you
beneficios por ubicaciones
Medicina prepagada con Colsanitas para ti y tus dependientes legales
Seguro de vida MetLife para ti
Miles de proyectos para marcas de renombre
Ingreso estable
Trabajo remoto 100% permanente
Software con licencia gratuita
Posibilidad de trabajar en tu propio dispositivo (BYOD)
Carga de trabajo estable
Oportunidades de reubicación
Modelos de participación flexibles
Capacitación gratuita en habilidades técnicas y blandas
Acceso gratuito a la plataforma de aprendizaje de LinkedIn
Apoyo de un Asesor de Habilidades personal
Cursos de idiomas
Acceso gratuito a bibliotecas electrónicas internas y externas
Acceso a comunidades internas y centros de competencia
Oportunidades de certificación
Envíanos tu CV para recibir una oferta personalizada