senior big data engineer for a video game development company
We are looking for a remote Senior Big Data Engineer with 3+ years of experience in Data Intelligence and Big Data to help build and manage big data streaming technologies.
The customer is an American video game and software developer and publisher, developing commercially available game engine which also powers their internally developed video games.
The main pool of tasks is associated with optimizing our existing streaming services, evaluating new technologies, managing cloud infrastructure, and more. The ingestion pipelines peak at over 90GB per minute.
- Building and supporting highly available streaming services, written in Java/Scala using Apache Flink
- Operating and developing applications on distributed NoSQL key-value stores such as HBase, DynamoDB or FoundationDB
- Managing cloud infrastructure launched in AWS via Terraform
- Participating in an on-call rotation for Epic Games streaming tech stack
- Evaluating new technologies and components of the big data ecosystem that are not yet part of our environment
- 3+ years of experience in Data Intelligence and Big Data
- Experience building high throughput Java/Scala streaming services using technologies such as Flink or Spark
- Strong understanding of distributed message brokers such as Kafka and Kinesis
- Experience with performance tuning of distributed computing technologies at massive scale
- Experience working with AWS services such as Kinesis, EC2, EMR, RDS via infrastructure as code technologies
- Experience working with high throughput, distributed NoSQL databases
- Capable of independently performing root cause analysis on high throughput systems