Senior Big Data Engineer Jobs

EPAM Anywhere is looking for remote Senior Big Data Engineers.

Competitive compensation

We back all payments and operate in compliance with country-specific labor regulations. Your annual paycheck can range from $15k to $200k.

Numerous benefits

Make use of 1,500+ online and location-specific benefits across 30+ countries that also include healthcare and sports programs for you and your family.

Limitless career growth opportunities

Get access to a large-scale ecosystem that consists of educational and career development services created for your growth. A personal skill advisor will support you on each step of this journey.

senior big data engineer

Data Software Engineering

Data Analytics and Visualization, Databricks, AWS Glue

40 hrs/week

12+ months

Greece, Poland,

and others

Greece, Italy, Poland, Portugal

Learn more

We are currently hiring for the position of Senior Big Data Engineer for a remote work setup.

The ideal candidate will work closely with technical teams and business stakeholders in developing, designing, and delivering high-quality data products that meet business requirements. The role is responsible for end-to-end data integration, modeling, and reporting on MicroStrategy and Databricks platforms.

responsibilities

Translate business requirements into technical solutions
Design, develop, monitor, and visualize Customer Data Access Overhaul data product
Integrate high-quality datasets for analytical use-cases
Design incident management process
Help other data teams follow the customer standards
Conduct testing (preparation and execution)
Operate data pipelines and address alerts, bugs, etc.

requirements

Bachelor's or Master's degree in Computer Science, Engineering, or related field
Minimum of 3 years of experience in Databricks or similar
Minimum of 3 years of experience in MicroStrategy
Strong skills in data visualization and data modeling
Good understanding of ETL concepts
Proficient in Apache Spark, PySpark, Python, and SQL
Familiarity with AWS Glue, Amazon S3, GitHub, and CI/CD
Ability to work independently and as a team player
Strong communication and organizational skills
Strong problem-solving skills and attention to detail
Ability to multitask in a fast-paced environment
Willingness to learn and keep updated with emerging technologies

nice to have

Advanced Databricks skills like delta, cluster configuration, or Databricks APIs
Experience in new technologies like BigQuery, Airflow, Redshift, or Apache NiFi

senior big data engineer

Data Software Engineering

Data Analytics and Visualization, Databricks, AWS Glue

40 hrs/week

12+ months

Greece, Poland,

and others

Greece, Italy, Poland, Portugal

Learn more

We are seeking a remote Senior Big Data Engineer to join our team.

The candidate should be able to work closely with business stakeholders and technical teams to design, develop and deliver high-quality data products that meet the business requirements. The role involves end-to-end responsibility for data integration, data modeling, and reporting on MicroStrategy, with the option to work with Databricks as well.

responsibilities

Gather business requirements and translate them into technical solutions
Design, develop, monitor, and visualize Customer Data Access Overhaul data product
Integrate high-quality datasets for analytical use-cases
Support in designing incident management process
Enable other data teams to follow the customer standards
Testing (preparation and execution)
Operations of data pipelines including alerting, bug fixing, etc.

requirements

3+ years of hands-on experience in Databricks or similar
3+ years of hands-on experience in MicroStrategy
Bachelor's or Master's degree in Computer Science, Engineering or a related field
Strong skills in data visualization and data modeling
Good understanding of ETL concepts
Apache Spark, PySpark and Python, SQL
AWS Glue + Amazon S3
GitHub + CI/CD
Team player (easy & respectful communications, shares responsibilities for the team's overall success)
Ability to work independently
Communication skills
Organizational skills
Strong problem-solving skills and attention to detail
Ability to handle multiple tasks in a fast-paced environment
Willingness to learn and stay up-to-date with emerging technologies

nice to have

Advanced Databricks skills like delta, cluster configuration, or Databricks APIs
Experience in new technologies like BigQuery, Airflow, Redshift, or Apache NiFi

senior big data engineer

Big Data

Management, Docker, Kubernetes

40 hrs/week

12+ months

Armenia, Kazakhstan,

and others

Armenia, Georgia, Kazakhstan, Poland, Uzbekistan

Learn more

We are inviting a remote Senior Big Data Engineer with 3+ years of experience in enterprise software development, to join our team remotely.

We are currently looking for a remote Senior Big Data Engineer with 3+ years of experience in enterprise software development to join our team.

responsibilities

Design and implement innovative analytical solution using Hadoop, NoSQL and other Big Data related technologies
Work with product and engineering teams to understand requirements, evaluate new features and architecture to help drive decisions
Perform detailed analysis of business problems and technical environments
Participate in code review and test solutions to ensure it meets best practice specifications
Build and foster a high-performance engineering culture, mentor team members and provide team with the tools and motivation
Write project documentation

requirements

Over 3 years of experience in enterprise software development
Solid background in BigData and distributed computing for 3+ years
Experienced and highly self-motivated professional with outstanding analytical and problem solving skills
Able to work closely with customers and other stakeholders
Good experience in developing BigData, highly available, largely scalable applications and systems
Able to play mentoring role on a project and ensure that solutions meet business requirements and expectations
Experienced in working with modern Agile developing methodologies and tools
Advanced experience in software development with Big Data technologies (e.g. administration, configuration management, monitoring, debugging and performance tuning)
Engineering experience and practice in Data Management, Data Storage, Data Visualization, Disaster Recovery, Integration, Operation, Security
Experience building data ingestion pipelines, Data Warehouse or Database architecture.
Experience with data modeling; hands-on development experience with modern Big Data components
Cloud: experience in designing, deploying and administering scalable, available and fault tolerant systems
Good understanding of CI/CD principles and best practices
Analytical approach to problem; excellent interpersonal, mentoring and communication skills
Data-oriented personality and possessing compliance awareness, such as PI, GDPR, HIPAA
Motivated, independent, efficient and able work under pressure with a solid sense for setting priorities
Ability to work in a fast-paced (startup like) agile development environment
Experience in high load and IoT Data Platform architectures and infrastructures
Experience with Containers and Resource Management systems: Docker, Kubernetes, Yarn
Experience in direct customer communications
Solid skills in infrastructure troubleshooting, support and practical experience in performance tuning and optimization, bottleneck problem analysis
Experienced in different business domains
English proficiency
Advanced understanding of distributed computing principles

technologies

Programming Languages: Java, Scala, Python, SQL, Bash
Big Data stack: Hadoop, Yarn, HDFS, MapReduce, Hive, Spark, Kafka, Flume, Sqoop, ZooKeeper
NoSQL: Cassandra/Hbase, MongoDB
Queues and Stream processing: Kafka Streams, Spark Streaming, Event Hub, IoT Hub, Storage Queues, Service Bus, Stream Analytics
Data Visualization: Tableau, QlikView
ETL & Streaming Pipelines: Pentaho, Talend, Apache Oozie, Airflow, NiFi, Streamsets
Operation: Cluster operation, Cluster planning
Search: Solr, Elasticsearch/ELK
InMemory: Ignite, Redis
Cloud (AWS/Azure/GCP): Storage, Compute, Networking, Identity and Security, NoSQL, RDBMS and Cubes, Big Data Processing, Queues and Stream Processing, Serverless. Data Analysis and Visualization, ML as a service (SageMaker, Tensorflow)
Enterprise Design Patterns (ORM, Inversion of Control etc.)
Development Methods (TDD, BDD, DDD)
Version Control Systems (Git, SVN)
Testing: Component/ Integration Testing, Unit testing (JUnit)
Deep understanding of SQL queries, joins, stored procedures, relational schemas, SQL optimization
Experience in various messaging systems, such as Kafka, RabbitMQ
Rest, Thrift, GRPC, SOAP
Build Systems: Maven, SBT, Ant, Gradle
Docker, Kubernetes, Yarn

senior big data engineer for data analytics company

Big Data

Java, Scala, Jenkins

40 hrs/week

12+ months

Armenia, Poland

Learn more

We are looking for a Senior Big Data Engineer with 5+ years of experience as a Java Developer, proficiency in data engineering, expertise in Big Data: Hadoop, Spark, Kafka, strong knowledge of Scala, to join our team remotely.

Our customer exposes an API for other businesses to be able to access risks associated with a person, e.g. understand their credit score. Behind the scene there is sophisticated decisioning system and large data volumes. Currently back-end of this API has number of legacy versions, serving hundreds of clients, with individual installations for most clients. We need to create a data lake for one of the biggest data analytics company working with personal information both domestically and internationally. In a nutshell, this includes replatforming of on-premises Enterprise Data Hub from Hadoop cluster into GCP. Day to day tasks include but not limited to creating spark application which manipulate with data from different sources including Oracle, Google Cloud Storage, BigQuery; creating pipelines via GCP Dataflow; working with Jenkins and AirFlow.

requirements

5+ years of experience as a Java Developer
Proficiency in data engineering
Expertise in Big Data: Hadoop, Spark, Kafka
Strong knowledge of Scala
Expertise in Microservices Architecture
Ability to work with high volumes of data
Experience in working with AWS
Experience in working with GCP: Dataproc (Apache Spark), Dataflow (Apache Beam), BigQuery, Cloud Storage
Good understanding of Design Patterns, Clean Code, Unit testing
Experience working in Agile environment
Data modelling skills would be a plus
Experience in Jenkins, AirFlow with Groovy and Python
Excellent written and verbal communication skills
Intermediate or higher English level, both verbal and written (B1+)

senior big data engineer for semiconductor manufacturer

Big Data

Java, Scala, Python.Core

40 hrs/week

12+ months

Armenia, Kazakhstan,

and others

Armenia, Georgia, Kazakhstan, Poland, Uzbekistan

Learn more

We are looking for a Senior Big Data Engineer with experience in Java, Scala, Python, Apache Spark, Apache Parquet, Apache EMR, Jupyter, NoSQL, AWS, to join our team remotely.

Currently we are looking for remote Senior Big Data Engineer to make the team even stronger. Our client is one of biggest semiconductor manufacturers. The main idea is to implement a ‘smart manufacturing’ approach and this project is one of the first steps in this direction. As part of the solution, we should apply complex processing using cutting-edge Big Data technologies to process the original data to produce comprehensive block information.

responsibilities

Increase production line throughput
Improve manufacturing equipment utilization
Relate the events during the processing to the resulting product quality
Identify the reasons for delays in the processing pipeline and quantifying their impact
Manufacture control systems capture process data in various formats including relational storage, files, and system snapshots
Build a technology platform to bring all the data together in one place, process it, and map to a unified data model, handling the proprietary unstructured data formats
Analyze the data to get the big picture on manufacturing delays across the fab to potentially optimize the manufacturing process and increase manufacturing capacity and cost effectiveness

requirements

Java
Scala
Python
Apache Spark
Apache Parquet
Apache EMR
Jupyter
NoSQL
AWS
Intermediate or higher English level, both verbal and written (B1+)

senior big data engineer for beauty retailer

Big Data

Java, Scala, Python.Core

40 hrs/week

12+ months

Armenia, Kazakhstan,

and others

Armenia, Georgia, Kazakhstan, Poland, Uzbekistan

Learn more

We are looking for a Senior Big Data Enginee with 3+ years of experience in Java/Scala or Python, experience with HIVE, Spark will be a plus, Apache Hadoop will be a plus, SQL, Hadoop, Kafka, Spark, to join our team remotely.

Today we are a global team of technologists and thinkers, who help transform the world with the power of software, enabling our customers to be competitive and disruptive in the marketplace through innovative technology solutions. As a remote Senior Big Data Engineer , you will live on the front lines of the products we create, and build features used by millions of people every day. Our customer is a leader in global prestige retail, teaching and inspiring clients to play in a world of beauty. The company has earned its reputation all over the world as a beauty trailblazer with its expertise, innovation, and entrepreneurial spirit. Our project is about developing a full-stack e-Commerce application for the world's biggest cosmetics retailer chain of stores.

requirements

3+ years of experience in Java/Scala or Python
Experience with HIVE, Spark will be a plus
Experience with Apache Hadoop will be a plus
English level: Intermediate+ (B1+)

senior big data engineer for healthcare company

Big Data

Docker, Kubernetes, Cloud.AWS

40 hrs/week

12+ months

Armenia, Georgia, Poland

Learn more

We are looking for a remote Senior Big Data Engineer with 4+ years of experience building large-scale software systems, required skills: Scala, Spark, Spark Unit Testing, Data Wrangling, experience in AWS cloud technologies and stack, to join our team. Our client is s a global healthcare company that provides solutions to address the needs of patients worldwide. It is a strategic initiative a targeting developing of the next-gen platform for data analytics.

responsibilities

Work closely with internal and external stakeholders
Develop the next-gen platform for data analytics

requirements

4+ years of experience building large scale software systems
Required skills: Scala, Spark, Spark Unit Testing, Data Wrangling
Experience in AWS cloud technologies and stack
Experience with Docker and Kubernetes
Desired skills: Event / Stream experience (Kafka or similar), Elasticsearch, Databricks
Familiarity with agile process
Strong communication and interpersonal skills
English level: Intermediate+ (B1+)

senior big data engineer for an insurance company

Big Data

Data Integration, SQL, Scala

40 hrs/week

12+ months

Georgia, Poland

Learn more

Currently, we are looking for a remote Senior Big Data Engineer with 3+ years of Big Data engineering experience, and Hands-on experience with Cloudera Hadoop stack to join our team.

Our customer offers automobile, property, liability, agriculture, and surety insurance.

responsibilities

Contribute to designing, developing, and monitoring systems and solutions for collecting, storing, processing, and analyzing large data sets
Work in a team made up of other Data Engineers
Understand and implement certain platform capabilities
Understand coding standards and implement them within code being developed

requirements

3+ years of Big Data engineering experience
Hands-on experience with Cloudera Hadoop stack including Hive and Impala performance optimization
Data Quality management knowledge is highly desirable
Experience with real-time data processing technologies: Flume, Message Queues (preferably IBM WebSphere MQ)
2+ years' experience with Hadoop stack: HDFS, Hive, Impala
2+ years' experience with Spark Scala
1+ year of ETL tools experience, preferably Pentaho
Experience with Cloudera 6+
Knowledge of SQL/HQL
Experience with scripting (bash preferably or Perl)
Knowledge of RDBMS technologies (Oracle preferably and experience with SQL Server, SSRS - reporting, SSIS - ETL)
Experience with AWS or Cloud technologies
Experience in messaging tech – MQ or Kafka or both and general cloud awareness would be extremely helpful
Knowledge of Insurance field and Guidewire system
Data Modelling CA Erwin/Sparx
Experience in OOZIE workflow
Proficient English (written and spoken) B2

senior big data engineer for a software company

Big Data

Java, SQL

40 hrs/week

12+ months

Armenia, Kazakhstan,

and others

Armenia, Georgia, Kazakhstan, Uzbekistan

Learn more

Currently, we are looking for a Remote Senior Big Data Engineer with 3+ years of software engineering experience in Core Java, knowledge and experience with large-scale data tools and techniques to join our team.

Our customer is a software company. Its mission is to help organizations better manage their software supply chain, enabling customers to build applications faster, with better security, and with confidence. The company is a remote and talented product development group that works in small autonomous teams to create high-quality products. Thousands of organizations and millions of developers use their software. The customer is expanding the Data team, responsible for unlocking insight from vast amounts of software component data, powering a suite of products enabling their customers in making informed and automated decisions to manage their software supply chain. We are looking for smart people, who get things done in a team environment. We have a deeply strong team technically, though many places probably claim this. If you love working on and solving some of the most difficult software engineering challenges, you will love working here. The ideal candidate has the internal drive and initiative to make the product vision a reality. Flow should be the predominant state of mind.

responsibilities

Contribute to designing, developing and monitoring systems and solutions for collecting, storing, processing, and analyzing large data sets
Work in a team made up of other Data Engineers

requirements

3+ years of software engineering experience in Core Java
Database and data manipulation skills working with relational or non-relational models
Deep curiosity for how things work and desire to make them better
Proficient English (written and spoken) B2

nice to have

Degree in Computer Science, Engineering, or another quantitative field
Knowledge and experience with non-relational databases (i.e., HBase, MongoDB, Cassandra)
Knowledge and experience with large scale data tools and techniques (i.e., MapReduce, Hadoop, Hive, Spark)
Knowledge and experience with AWS Big Data services (i.e., EMR, Elasticsearch)
Experience in Data Management (Big Data)
Experience in NoSQL DB Development, NoSQL in Big Data, SQL
Experience working in a highly distributed environment, using modern collaboration tools to facilitate team communication
Experience in an Agile development environment
Experience working in a remote environment

senior big data engineer for a design company

Big Data

Python.Core

40 hrs/week

12+ months

Armenia, Poland, Uzbekistan

Learn more

Currently, we are looking for a remote Senior Big Data Engineer with 3+ years of Big Data engineering experience, and excessive knowledge of Airflow, Spark (pySpark), and Hive to join our team.

The customer is an American multinational corporation that is engaged in the design, development, manufacturing, and worldwide marketing and sales of footwear, apparel, equipment, accessories, and services. The team is focused on implementation of ELT processes, preparing Data Marts, optimized for BI/Reporting.

responsibilities

Collaborate with Client's SMEs and help to reverse-engineer existing data flow (Teradata, Informatica) to discover and investigate Data Sources, Data Flow related requirements (NFR, Validation, Ingestion, Consumers etc.)
Participate building new Data Platform Foundation as per provided Architecture, Policies and Engineering Standards
Understand and implement certain platform capabilities
Understand coding standards and implement them within code being developed
Design & Implement ELT processes (pySpark, Airflow, Hive).
Proactively adopt EngX practices, e.g., Automated Testing, CI/CD etc.
Review and Supervise L1/L2 Developers

requirements

3+ years of experience in Big Data Engineering
Experience (or excessive knowledge) and descent understanding of Architecture, Usage Patterns, Tuning & Troubleshooting capabilities of Apache Airflow, Apache Spark (pySpark), Apache Hive
Knowledge of Python (or ability to learn it quickly)
Strong Troubleshooting skills; problem-solving mindset
Experience working with CI/CD; understanding and following main principles
Eager to learn; collaboration focused
Strong Team Player; able to prioritize work
Good command of English (B1+)

nice to have

Cloud Experience: AWS
DevOps experience

Senior Big Data Engineer jobs at EPAM Anywhere

A senior big data engineer is a specialist with vast job experience who simultaneously performs technical and managerial tasks. They organize technological and innovative approaches to the design of big data platforms and manage a team of developers and engineers. The output, under the clear guidance of a professional, should be a complex computing environment that processes hundreds of terabytes of information from various sources.

What a remote senior big data engineer does

The specialist takes part in software development and is responsible for the quality of the work performed. Their goal is to create reliable and scalable technologies for working with big data according to customer requirements. At the same time, a remote senior big data engineer is engaged in the selection of job tools, the distribution of tasks between employees, the management of junior colleagues, and quality control.

Main job responsibilities

In addition to participating in the development, testing, quality assessment, and support of big data systems according to the business goals, the job duties of a senior big data engineer include:

Search for the most suitable technologies for integration with the system
Training of junior employees
Continuous improvement of the product
Search for errors and their correction
Managing employees
Proper distribution of tasks between team members
Building working relationships with business analysts to improve the quality of the project
Ensuring system security
Team productivity improvement

Specialists should also be able to not only create new large-scale platforms but also modify existing ones.

Primary job requirements

The main requirements for getting a job as a senior big data engineer include:

Job experience as a big data platform developer for more than four years
A Bachelor's or Master's degree in computer science
Ability to load a large amount of data from databases such as MPP, SMP, etc.
Ability to debug work processes
Deep knowledge of programming languages such as Python and Java
Great experience with large processing systems
Knowledge of the most effective technologies in the world of database development
Ability to write documentation and present projects
Test writing and testing skills
Ability to correctly compose technical tasks for all team members

The candidate for the position must be purposeful, have leadership qualities, and be able to set up the job of the entire team. A senior big data engineer must communicate confidently not only with the subordinates but also with colleagues from other departments, company management, and customers. It is on the basis of proper communication that productive cooperation will be built.

Benefits of choosing senior big data engineer jobs at EPAM Anywhere

On the EPAM Anywhere platform, there is a constant stream of remote senior big data engineer jobs. We are looking for the best managers for junior and intermediate specialists who are constantly moving forward and looking for new sources of knowledge. Our huge database of literature, lectures, and video materials will be open for you so you can work in your favorite role and constantly improve. Apply right now!

Senior Big Data Engineer Jobs

Senior Big Data Engineer jobs at EPAM Anywhere

What a remote senior big data engineer does

Main job responsibilities

Primary job requirements

Benefits of choosing senior big data engineer jobs at EPAM Anywhere

Related skills