Big Data Engineer Jobs

EPAM Anywhere is looking for Big Data Engineers.

Competitive compensation

We back all payments and operate in compliance with country-specific labor regulations. Your annual paycheck can range from $15k to $200k.

Numerous benefits

Make use of 1,500+ online and location-specific benefits across 30+ countries that also include healthcare and sports programs for you and your family.

Limitless career growth opportunities

Get access to a large-scale ecosystem that consists of educational and career development services created for your growth. A personal skill advisor will support you on each step of this journey.

middle big data engineer

Data Software Engineering

Data Analytics and Visualization, Databricks, AWS Glue

40 hrs/week

12+ months

Greece, Poland,

and others

Greece, Italy, Poland, Portugal

Learn more

We are seeking a remote Middle Big Data Engineer to join our team.

The candidate should be able to work closely with business stakeholders and technical teams to design, develop and deliver high-quality data products that meet the business requirements. The role involves end-to-end responsibility for data integration, data modeling, and reporting on MicroStrategy, with the option to work with Databricks as well.

responsibilities

Gather business requirements and translate them into technical solutions
Design, develop, monitor, and visualize Customer Data Access Overhaul data product
Integrate high-quality datasets for analytical use-cases
Support in designing incident management process
Enable other data teams to follow the customer standards
Testing (preparation and execution)
Operations of data pipelines including alerting, bug fixing, etc.

requirements

2+ years of hands-on experience in Databricks or similar
2+ years of hands-on experience in MicroStrategy
Bachelor's or Master's degree in Computer Science, Engineering or a related field
Strong skills in data visualization and data modeling
Good understanding of ETL concepts
Apache Spark, PySpark and Python, SQL
AWS Glue + Amazon S3
GitHub + CI/CD
Team player (easy & respectful communications, shares responsibilities for the team's overall success)
Ability to work independently
Communication skills
Organizational skills
Strong problem-solving skills and attention to detail
Ability to handle multiple tasks in a fast-paced environment
Willingness to learn and stay up-to-date with emerging technologies

nice to have

Advanced Databricks skills like delta, cluster configuration, or Databricks APIs
Experience in new technologies like BigQuery, Airflow, Redshift, or Apache NiFi

senior big data engineer

Data Software Engineering

Data Analytics and Visualization, Databricks, AWS Glue

40 hrs/week

12+ months

Greece, Poland,

and others

Greece, Italy, Poland, Portugal

Learn more

We are seeking a remote Senior Big Data Engineer to join our team.

responsibilities

Gather business requirements and translate them into technical solutions
Design, develop, monitor, and visualize Customer Data Access Overhaul data product
Integrate high-quality datasets for analytical use-cases
Support in designing incident management process
Enable other data teams to follow the customer standards
Testing (preparation and execution)
Operations of data pipelines including alerting, bug fixing, etc.

requirements

3+ years of hands-on experience in Databricks or similar
3+ years of hands-on experience in MicroStrategy
Bachelor's or Master's degree in Computer Science, Engineering or a related field
Strong skills in data visualization and data modeling
Good understanding of ETL concepts
Apache Spark, PySpark and Python, SQL
AWS Glue + Amazon S3
GitHub + CI/CD
Team player (easy & respectful communications, shares responsibilities for the team's overall success)
Ability to work independently
Communication skills
Organizational skills
Strong problem-solving skills and attention to detail
Ability to handle multiple tasks in a fast-paced environment
Willingness to learn and stay up-to-date with emerging technologies

nice to have

Advanced Databricks skills like delta, cluster configuration, or Databricks APIs
Experience in new technologies like BigQuery, Airflow, Redshift, or Apache NiFi

senior big data engineer

Big Data

Management, Docker, Kubernetes

40 hrs/week

12+ months

Armenia, Kazakhstan,

and others

Armenia, Georgia, Kazakhstan, Poland, Uzbekistan

Learn more

We are inviting a remote Senior Big Data Engineer with 3+ years of experience in enterprise software development, to join our team remotely.

We are currently looking for a remote Senior Big Data Engineer with 3+ years of experience in enterprise software development to join our team.

responsibilities

Design and implement innovative analytical solution using Hadoop, NoSQL and other Big Data related technologies
Work with product and engineering teams to understand requirements, evaluate new features and architecture to help drive decisions
Perform detailed analysis of business problems and technical environments
Participate in code review and test solutions to ensure it meets best practice specifications
Build and foster a high-performance engineering culture, mentor team members and provide team with the tools and motivation
Write project documentation

requirements

Over 3 years of experience in enterprise software development
Solid background in BigData and distributed computing for 3+ years
Experienced and highly self-motivated professional with outstanding analytical and problem solving skills
Able to work closely with customers and other stakeholders
Good experience in developing BigData, highly available, largely scalable applications and systems
Able to play mentoring role on a project and ensure that solutions meet business requirements and expectations
Experienced in working with modern Agile developing methodologies and tools
Advanced experience in software development with Big Data technologies (e.g. administration, configuration management, monitoring, debugging and performance tuning)
Engineering experience and practice in Data Management, Data Storage, Data Visualization, Disaster Recovery, Integration, Operation, Security
Experience building data ingestion pipelines, Data Warehouse or Database architecture.
Experience with data modeling; hands-on development experience with modern Big Data components
Cloud: experience in designing, deploying and administering scalable, available and fault tolerant systems
Good understanding of CI/CD principles and best practices
Analytical approach to problem; excellent interpersonal, mentoring and communication skills
Data-oriented personality and possessing compliance awareness, such as PI, GDPR, HIPAA
Motivated, independent, efficient and able work under pressure with a solid sense for setting priorities
Ability to work in a fast-paced (startup like) agile development environment
Experience in high load and IoT Data Platform architectures and infrastructures
Experience with Containers and Resource Management systems: Docker, Kubernetes, Yarn
Experience in direct customer communications
Solid skills in infrastructure troubleshooting, support and practical experience in performance tuning and optimization, bottleneck problem analysis
Experienced in different business domains
English proficiency
Advanced understanding of distributed computing principles

technologies

Programming Languages: Java, Scala, Python, SQL, Bash
Big Data stack: Hadoop, Yarn, HDFS, MapReduce, Hive, Spark, Kafka, Flume, Sqoop, ZooKeeper
NoSQL: Cassandra/Hbase, MongoDB
Queues and Stream processing: Kafka Streams, Spark Streaming, Event Hub, IoT Hub, Storage Queues, Service Bus, Stream Analytics
Data Visualization: Tableau, QlikView
ETL & Streaming Pipelines: Pentaho, Talend, Apache Oozie, Airflow, NiFi, Streamsets
Operation: Cluster operation, Cluster planning
Search: Solr, Elasticsearch/ELK
InMemory: Ignite, Redis
Cloud (AWS/Azure/GCP): Storage, Compute, Networking, Identity and Security, NoSQL, RDBMS and Cubes, Big Data Processing, Queues and Stream Processing, Serverless. Data Analysis and Visualization, ML as a service (SageMaker, Tensorflow)
Enterprise Design Patterns (ORM, Inversion of Control etc.)
Development Methods (TDD, BDD, DDD)
Version Control Systems (Git, SVN)
Testing: Component/ Integration Testing, Unit testing (JUnit)
Deep understanding of SQL queries, joins, stored procedures, relational schemas, SQL optimization
Experience in various messaging systems, such as Kafka, RabbitMQ
Rest, Thrift, GRPC, SOAP
Build Systems: Maven, SBT, Ant, Gradle
Docker, Kubernetes, Yarn

lead big data engineer

Data Software Engineering

Data Analytics and Visualization, Databricks, AWS Glue

40 hrs/week

12+ months

Greece, Poland,

and others

Greece, Italy, Poland, Portugal

Learn more

We are seeking a remote Lead Big Data Engineer to join our team.

responsibilities

Gather business requirements and translate them into technical solutions
Design, develop, monitor, and visualize Customer Data Access Overhaul data product
Integrate high-quality datasets for analytical use-cases
Support in designing incident management process
Enable other data teams to follow the customer standards
Testing (preparation and execution)
Operations of data pipelines including alerting, bug fixing, etc.

requirements

5+ years of hands-on experience in Databricks or similar
5+ years of hands-on experience in MicroStrategy
Bachelor's or Master's degree in Computer Science, Engineering or a related field
Strong skills in data visualization and data modeling
Good understanding of ETL concepts
Apache Spark, PySpark and Python, SQL
AWS Glue + Amazon S3
GitHub + CI/CD
Team player (easy & respectful communications, shares responsibilities for the team's overall success)
Ability to work independently
Communication skills
Organizational skills
Strong problem-solving skills and attention to detail
Ability to handle multiple tasks in a fast-paced environment
Willingness to learn and stay up-to-date with emerging technologies

nice to have

Advanced Databricks skills like delta, cluster configuration, or Databricks APIs
Experience in new technologies like BigQuery, Airflow, Redshift, or Apache NiFi

senior big data engineer

Data Software Engineering

Data Analytics and Visualization, Databricks, AWS Glue

40 hrs/week

12+ months

Greece, Poland,

and others

Greece, Italy, Poland, Portugal

Learn more

We are currently hiring for the position of Senior Big Data Engineer for a remote work setup.

The ideal candidate will work closely with technical teams and business stakeholders in developing, designing, and delivering high-quality data products that meet business requirements. The role is responsible for end-to-end data integration, modeling, and reporting on MicroStrategy and Databricks platforms.

responsibilities

Translate business requirements into technical solutions
Design, develop, monitor, and visualize Customer Data Access Overhaul data product
Integrate high-quality datasets for analytical use-cases
Design incident management process
Help other data teams follow the customer standards
Conduct testing (preparation and execution)
Operate data pipelines and address alerts, bugs, etc.

requirements

Bachelor's or Master's degree in Computer Science, Engineering, or related field
Minimum of 3 years of experience in Databricks or similar
Minimum of 3 years of experience in MicroStrategy
Strong skills in data visualization and data modeling
Good understanding of ETL concepts
Proficient in Apache Spark, PySpark, Python, and SQL
Familiarity with AWS Glue, Amazon S3, GitHub, and CI/CD
Ability to work independently and as a team player
Strong communication and organizational skills
Strong problem-solving skills and attention to detail
Ability to multitask in a fast-paced environment
Willingness to learn and keep updated with emerging technologies

nice to have

Advanced Databricks skills like delta, cluster configuration, or Databricks APIs
Experience in new technologies like BigQuery, Airflow, Redshift, or Apache NiFi

lead big data engineer

Data Software Engineering

Data Analytics and Visualization, Databricks, AWS Glue

40 hrs/week

12+ months

Greece, Poland,

and others

Greece, Italy, Poland, Portugal

Learn more

We are seeking a remoteWe are currently hiring for the position of Lead Big Data Engineer for a remote work setup.

We are currently hiring for the position of Lead Big Data Engineer for a remote work setup. The ideal candidate will work closely with technical teams and business stakeholders in developing, designing, and delivering high-quality data products that meet business requirements. The role is responsible for end-to-end data integration, modeling, and reporting on MicroStrategy and Databricks platforms.

responsibilities

Translate business requirements into technical solutions
Design, develop, monitor, and visualize Customer Data Access Overhaul data product
Integrate high-quality datasets for analytical use-cases
Design incident management process
Help other data teams follow the customer standards
Conduct testing (preparation and execution)
Operate data pipelines and address alerts, bugs, etc.

requirements

Bachelor's or Master's degree in Computer Science, Engineering, or related field
Minimum of 5 years of experience in Databricks or similar
Minimum of 5 years of experience in MicroStrategy
1+ years of relevant leadership experience
Strong skills in data visualization and data modeling
Good understanding of ETL concepts
Proficient in Apache Spark, PySpark, Python, and SQL
Familiarity with AWS Glue, Amazon S3, GitHub, and CI/CD
Ability to work independently and as a team player
Strong communication and organizational skills
Strong problem-solving skills and attention to detail
Ability to multitask in a fast-paced environment
Willingness to learn and keep updated with emerging technologies

nice to have

Advanced Databricks skills like delta, cluster configuration, or Databricks APIs
Experience in new technologies like BigQuery, Airflow, Redshift, or Apache NiFi

senior big data software engineer

Big Data

40 hrs/week

12+ months

Armenia, Kazakhstan,

and others

Armenia, Georgia, Kazakhstan, Poland, Uzbekistan

Learn more

We are inviting a Senior Big Data Software Engineer with 5+ years of experience with Java, Scala or Python, Data engineering skills (data ingestion, storage and processing) in batch and streaming solutions using Kafka and Spark, to join our team remotely.

We are currently looking for a remote Senior Big Data Software Engineer with 5+ years of experience with Java, Scala or Python to join our team. The customer provides Direct-to-Consumer services encompassing global entertainment and news TV properties, TV stations group, and radio businesses.

responsibilities

Ensure high quality performance by implementing and refining robust data processing using Java, Scala or Python
Build scalable analytics solution, including data processing, storage and serving large-scale data through batch and stream
Help operationalize machine learning models and build apps
Contribute to making our data platform more scalable, resilient and reliable
Participate in code review sessions

requirements

5+ years of experience with Java, Scala or Python
Data engineering skills (data ingestion, storage and processing) in batch and streaming solutions using Kafka and Spark
Understanding and practical experience with AWS
Skills with big data framework such as Hadoop & Apache Spark, NoSQL systems such as Cassandra or DynamoDB, streaming technologies such as Apache Kafka
Understand reactive programming and dependency injection such as Spring to develop REST services
Experience working with data scientists to operationalize machine learning models and build apps to make use of power of machine learning
Experience with newer technologies relevant to the data space such as Spark, Kafka, Apache Druid (or any other OLAP databases)
Good problem-solving skills
Spoken and written English level - B2 and higher

senior big data engineer for data analytics company

Big Data

Java, Scala, Jenkins

40 hrs/week

12+ months

Armenia, Poland

Learn more

We are looking for a Senior Big Data Engineer with 5+ years of experience as a Java Developer, proficiency in data engineering, expertise in Big Data: Hadoop, Spark, Kafka, strong knowledge of Scala, to join our team remotely.

Our customer exposes an API for other businesses to be able to access risks associated with a person, e.g. understand their credit score. Behind the scene there is sophisticated decisioning system and large data volumes. Currently back-end of this API has number of legacy versions, serving hundreds of clients, with individual installations for most clients. We need to create a data lake for one of the biggest data analytics company working with personal information both domestically and internationally. In a nutshell, this includes replatforming of on-premises Enterprise Data Hub from Hadoop cluster into GCP. Day to day tasks include but not limited to creating spark application which manipulate with data from different sources including Oracle, Google Cloud Storage, BigQuery; creating pipelines via GCP Dataflow; working with Jenkins and AirFlow.

requirements

5+ years of experience as a Java Developer
Proficiency in data engineering
Expertise in Big Data: Hadoop, Spark, Kafka
Strong knowledge of Scala
Expertise in Microservices Architecture
Ability to work with high volumes of data
Experience in working with AWS
Experience in working with GCP: Dataproc (Apache Spark), Dataflow (Apache Beam), BigQuery, Cloud Storage
Good understanding of Design Patterns, Clean Code, Unit testing
Experience working in Agile environment
Data modelling skills would be a plus
Experience in Jenkins, AirFlow with Groovy and Python
Excellent written and verbal communication skills
Intermediate or higher English level, both verbal and written (B1+)

senior big data engineer for semiconductor manufacturer

Big Data

Java, Scala, Python.Core

40 hrs/week

12+ months

Armenia, Kazakhstan,

and others

Armenia, Georgia, Kazakhstan, Poland, Uzbekistan

Learn more

We are looking for a Senior Big Data Engineer with experience in Java, Scala, Python, Apache Spark, Apache Parquet, Apache EMR, Jupyter, NoSQL, AWS, to join our team remotely.

Currently we are looking for remote Senior Big Data Engineer to make the team even stronger. Our client is one of biggest semiconductor manufacturers. The main idea is to implement a ‘smart manufacturing’ approach and this project is one of the first steps in this direction. As part of the solution, we should apply complex processing using cutting-edge Big Data technologies to process the original data to produce comprehensive block information.

responsibilities

Increase production line throughput
Improve manufacturing equipment utilization
Relate the events during the processing to the resulting product quality
Identify the reasons for delays in the processing pipeline and quantifying their impact
Manufacture control systems capture process data in various formats including relational storage, files, and system snapshots
Build a technology platform to bring all the data together in one place, process it, and map to a unified data model, handling the proprietary unstructured data formats
Analyze the data to get the big picture on manufacturing delays across the fab to potentially optimize the manufacturing process and increase manufacturing capacity and cost effectiveness

requirements

Java
Scala
Python
Apache Spark
Apache Parquet
Apache EMR
Jupyter
NoSQL
AWS
Intermediate or higher English level, both verbal and written (B1+)

senior big data engineer for beauty retailer

Big Data

Java, Scala, Python.Core

40 hrs/week

12+ months

Armenia, Kazakhstan,

and others

Armenia, Georgia, Kazakhstan, Poland, Uzbekistan

Learn more

We are looking for a Senior Big Data Enginee with 3+ years of experience in Java/Scala or Python, experience with HIVE, Spark will be a plus, Apache Hadoop will be a plus, SQL, Hadoop, Kafka, Spark, to join our team remotely.

Today we are a global team of technologists and thinkers, who help transform the world with the power of software, enabling our customers to be competitive and disruptive in the marketplace through innovative technology solutions. As a remote Senior Big Data Engineer , you will live on the front lines of the products we create, and build features used by millions of people every day. Our customer is a leader in global prestige retail, teaching and inspiring clients to play in a world of beauty. The company has earned its reputation all over the world as a beauty trailblazer with its expertise, innovation, and entrepreneurial spirit. Our project is about developing a full-stack e-Commerce application for the world's biggest cosmetics retailer chain of stores.

requirements

3+ years of experience in Java/Scala or Python
Experience with HIVE, Spark will be a plus
Experience with Apache Hadoop will be a plus
English level: Intermediate+ (B1+)

Big Data Engineer Jobs

Other skills