go to search
Lead Big Data Engineer
Expired job
We are currently looking for a remote Lead Big Data Engineer with 5+ years of experience in software development with Big Data technologies (administration, configuration management, monitoring, debugging, and performance tuning) to join our team.
responsibilities
- Lead, design and implement innovative analytical solution using Hadoop, NoSQL and other Big Data related technologies, evaluating new features and architecture in Cloud/on premise/Hybrid solutions
- Work with product and engineering teams to understand requirements, evaluate new features and architecture to help drive decisions
- Build collaborative partnerships with architects and key individuals within other functional groups.
- Perform detailed analysis of business problems and technical environments and use this in designing quality technical solution
- Actively participate in code review and test solutions to ensure it meets best practice specifications
- Build and foster a high performance engineering culture, mentor team members and provide team with the tools and motivation
- Write project documentation
requirements
- More than 5 years of experience in software development with Big Data technologies (e.g. administration, configuration management, monitoring, debugging and performance tuning)
- Engineering experience and practice in Data Management, Data Storage, Data Visualization, Disaster Recovery, Integration, Operation, Security
- Strong experience building data ingestion pipelines (simulating Extract, Transform, Load workload), Data Warehouse or Database architecture
- Strong experience with data modeling; hands-on development experience with modern Big Data components
- Cloud: experience in designing, automation, provisioning, deploying and administering scalable, available and fault tolerant systems
- Good understanding of CI/CD principles and best practices
- Analytical approach to problem-solving with an ability to work at an abstract level and gain consensus; excellent interpersonal, leadership and communication skills
- Data-oriented personality and possessing compliance awareness, such as PI, GDPR, HIPAA
- Motivated, independent, efficient and able to handle several projects; work under pressure with a solid sense for setting priorities
- Ability to work in a fast-paced (startup like) agile development environment
- Strong experience in high load and IoT Data Platform architectures and infrastructures
- Vast experience with Containers and Resource Management systems: Docker, Kubernetes, Yarn
- Experience in direct customer communications
- Experience in technology/team leading of data oriented projects
- Solid skills in infrastructure troubleshooting, support and practical experience in performance tuning and optimization, bottleneck problem analysis
- Experienced in different business domains
- English proficiency – B2 and higher
- Advanced understanding of distributed computing principles
- 1+ year of relevant leadership experience
technologies
- Programming Languages: Java/Scala/Python/SQL/Bash
- Big Data stack: Hadoop, Yarn, HDFS, MapReduce, Hive, Spark, Kafka, Flume, Sqoop, ZooKeeper
- NoSQL: Cassandra/Hbase/MongoDB
- Queues and Stream processing: Kafka Streams, Flink, Spark Streaming, Storm, Event Hub, IOT Hub MQTT, Storage Queues, Service Bus, Stream Analytics
- Data Visualization: Tableau/QlikView
- ETL & Streaming Pipelines: Pentaho, Talend, Apache Oozie, Airflow, NiFi, Streamsets
- Operation: Cluster operation, Cluster planning
- Search: Solr, Elasticsearch/ELK
- InMemory: Ignite, Redis
- Cloud (AWS/Azure/GCP): Storage, Compute, Networking, Identity and Security, NoSQL, RDBMS and Cubes, Big Data Processing, Queues and Stream Processing, Serverless, Data Analysis and Visualization, ML as a service (SageMaker, Tensorflow)
- Enterprise Design Patterns (ORM, Inversion of Control etc.)
- Development Methods (TDD, BDD, DDD)
- Version Control Systems (Git, SVN)
- Testing: Component/Integration Testing, Unit testing (JUnit)
- Deep understanding of SQL queries, joins, stored procedures, relational schemas; SQL optimization
- Messaging systems, such as Kafka, ZeroMQ/RabbitMQ
- Rest, Thrift, GRPC, SOAP
- Build Systems: Maven, SBT, Ant, Gradle
- Docker, Kubernetes, Yarn, Mesos
benefits for locations
Greece
For you
- Paid time off
- Paid sick leave days
- Stable income
- Meal and home office compensation
For your comfortable work
- 100% remote work
- Hybrid work opportunities
- Free licensed software
- Possibility to work on your own device (BYOD)
- Stable workload
- Relocation opportunities
- Flexible engagement models
For your growth
- Free trainings for technical and soft skills
- Free access to LinkedIn Learning platform
- Free access to internal and external e-Libraries
- Certification opportunities
- Skill advisory service
- Language courses
get job alerts in your inboxHundreds of open jobs for Software Engineers, QA, DevOps, Business Analysts and other tech professionals
looking for something else?
Find a vacancy that works for you. Send us your CV to receive a personalized offer.