senior data engineer

Data Software Engineering, Amazon Web Services, Apache Spark, Apache Hadoop, Databricks, Scala

Georgia

Sorry the job is no longer available.

Senior Data Engineer

We are looking for an experienced remote Senior Data Engineer to join our team.

You will be responsible for expanding and optimizing our data and data pipeline design, as well as optimizing data flow and collection for cross-functional teams. The ideal candidate is an experienced data pipeline builder and data wrangler who enjoys optimizing data systems and building them from the ground up.

responsibilities

Create and maintain optimal data pipeline design
Assemble large, complex data sets that meet functional/non-functional business requirements
Identify, design, and implement internal process improvements
Build the infrastructure required for optimal ETL process of data from a wide variety of data sources using SQL and AWS Big Data technologies or it can be Databricks as well
Work with stakeholders to assist with data-related technical issues and support their data infrastructure needs

requirements

3+ years of experience in a Data Engineer role
Good experience in Apache Spark and Hadoop technologies
Hands-on experience in Databricks platform
Must have experience in PySpark or Scala programming languages
Experience with AWS cloud services: EC2, EMR, RDS, Aurora DB services
Advanced working SQL knowledge and experience working with relational databases
B2+ English level

nice to have

Good to have Java and related programming language experience
Experience building and optimizing big data data pipelines
Experience supporting and working with cross-functional teams in a dynamic environment