senior big data developer
Our company is seeking an experienced remote Senior Big Data Developer with expertise in Data Software Engineering, specifically Snowflake and AWS Spark.
The candidate must demonstrate practical experience with AWS technologies such as API Gateway, Lambda, SQS, Glue, RDS, etc. The role involves building ingestion pipelines from multiple sources to a Raw data area, implementing initial data quality checks, and transforming raw data into Data Products based on business logic and requirements. The candidate must have excellent communication skills and experience working in a team-oriented environment.
- Build ingestion pipelines from multiple sources to the Raw data area, using (micro)batch, streaming, and API calls
- Implement initial data quality checks, logging, and monitoring processes
- Transform raw data into Data Products based on business logic and requirements
- Create a number of data feeds to external systems using (micro)batch, streaming
- Work collaboratively within a team to troubleshoot and solve problems
- At least 3 years of relevant experience in Data Software Engineering with practical hands-on experience in Snowflake and Spark in AWS
- Practical experience in AWS technologies such as API Gateway, SQS, RDS, etc.
- Experience with Apache Spark
- Experience with Python and its application in Big Data development
- Expertise in designing, building, and maintaining big data pipelines
- Upper-Intermediate English proficiency (B2+)
nice to have
- Knowledge of Amazon Web Services (AWS) Glue and Lambda