Remote Lead Big Data Engineer | EPAM Anywhere
Back icon

Lead Big Data Engineer

Lead Big Data Engineer 40 hrs/week, 12+ months

We are currently looking for a remote Lead Big Data Engineer with 5+ years of experience in software development with Big Data technologies (administration, configuration management, monitoring, debugging, and performance tuning) to join our team.

Responsibilities

  • Lead, design and implement innovative analytical solution using Hadoop, NoSQL and other Big Data related technologies, evaluating new features and architecture in Cloud/on premise/Hybrid solutions
    • Work with product and engineering teams to understand requirements, evaluate new features and architecture to help drive decisions
      • Build collaborative partnerships with architects and key individuals within other functional groups.
        • Perform detailed analysis of business problems and technical environments and use this in designing quality technical solution
          • Actively participate in code review and test solutions to ensure it meets best practice specifications
            • Build and foster a high performance engineering culture, mentor team members and provide team with the tools and motivation
              • Write project documentation

                Requirements

                • More than 5 years of experience in software development with Big Data technologies (e.g. administration, configuration management, monitoring, debugging and performance tuning)
                  • Engineering experience and practice in Data Management, Data Storage, Data Visualization, Disaster Recovery, Integration, Operation, Security
                    • Strong experience building data ingestion pipelines (simulating Extract, Transform, Load workload), Data Warehouse or Database architecture
                      • Strong experience with data modeling; hands-on development experience with modern Big Data components
                        • Cloud: experience in designing, automation, provisioning, deploying and administering scalable, available and fault tolerant systems
                          • Good understanding of CI/CD principles and best practices
                            • Analytical approach to problem-solving with an ability to work at an abstract level and gain consensus; excellent interpersonal, leadership and communication skills
                              • Data-oriented personality and possessing compliance awareness, such as PI, GDPR, HIPAA
                                • Motivated, independent, efficient and able to handle several projects; work under pressure with a solid sense for setting priorities
                                  • Ability to work in a fast-paced (startup like) agile development environment
                                    • Strong experience in high load and IoT Data Platform architectures and infrastructures
                                      • Vast experience with Containers and Resource Management systems: Docker, Kubernetes, Yarn
                                        • Experience in direct customer communications
                                          • Experience in technology/team leading of data oriented projects
                                            • Solid skills in infrastructure troubleshooting, support and practical experience in performance tuning and optimization, bottleneck problem analysis
                                              • Experienced in different business domains
                                                • English proficiency – B2 and higher
                                                  • Advanced understanding of distributed computing principles
                                                    • 1+ year of relevant leadership experience

                                                      Technologies

                                                      • Programming Languages: Java/Scala/Python/SQL/Bash
                                                        • Big Data stack: Hadoop, Yarn, HDFS, MapReduce, Hive, Spark, Kafka, Flume, Sqoop, ZooKeeper
                                                          • NoSQL: Cassandra/Hbase/MongoDB
                                                            • Queues and Stream processing: Kafka Streams, Flink, Spark Streaming, Storm, Event Hub, IOT Hub MQTT, Storage Queues, Service Bus, Stream Analytics
                                                              • Data Visualization: Tableau/QlikView
                                                                • ETL & Streaming Pipelines: Pentaho, Talend, Apache Oozie, Airflow, NiFi, Streamsets
                                                                  • Operation: Cluster operation, Cluster planning
                                                                    • Search: Solr, Elasticsearch/ELK
                                                                      • InMemory: Ignite, Redis
                                                                        • Cloud (AWS/Azure/GCP): Storage, Compute, Networking, Identity and Security, NoSQL, RDBMS and Cubes, Big Data Processing, Queues and Stream Processing, Serverless, Data Analysis and Visualization, ML as a service (SageMaker, Tensorflow)
                                                                          • Enterprise Design Patterns (ORM, Inversion of Control etc.)
                                                                            • Development Methods (TDD, BDD, DDD)
                                                                              • Version Control Systems (Git, SVN)
                                                                                • Testing: Component/Integration Testing, Unit testing (JUnit)
                                                                                  • Deep understanding of SQL queries, joins, stored procedures, relational schemas; SQL optimization
                                                                                    • Messaging systems, such as Kafka, ZeroMQ/RabbitMQ
                                                                                      • Rest, Thrift, GRPC, SOAP
                                                                                        • Build Systems: Maven, SBT, Ant, Gradle
                                                                                          • Docker, Kubernetes, Yarn, Mesos

                                                                                            Benefits in eligible locations

                                                                                            location.svg
                                                                                            ImageImage

                                                                                            For you

                                                                                            • Voluntary health insurance for you (100% covered) and your family (with corporate discount)
                                                                                            • Individual sessions with professional psychologists and coaches
                                                                                            • Free corporate sports and wellness programs
                                                                                            • Stable income
                                                                                            • Paid time off
                                                                                            ImageImage

                                                                                            For your comfortable work

                                                                                            • 100% remote work forever
                                                                                            • Free licensed software
                                                                                            • Possibility to work on your own device (BYOD)
                                                                                            • Stable workload
                                                                                            • Relocation opportunities
                                                                                            • Flexible engagement models
                                                                                              ImageImage

                                                                                              For your growth

                                                                                              • Free trainings for technical and soft skills
                                                                                              • Access to LinkedIn Learning platform
                                                                                              • Language courses
                                                                                              • Access to internal and external e-Libraries
                                                                                              • Certification opportunities
                                                                                              • Skill advisory service

                                                                                                40 hrs/week

                                                                                                Hours per week

                                                                                                12+ months

                                                                                                Project length

                                                                                                Belarus, Russia, Ukraine

                                                                                                Locations eligible for the position