We are currently looking for a remote Lead Big Data Engineer with strong experience with ETL, knowledge of Spark and familiarity with Databricks to join our team.
The customer is a provider of high-quality business services in such areas as IT, Procurement, HR, Product Lifecycle Services, Financial Reporting and Analytics, and Real Estate and Facility Services.
- Understand requirements, clarify with BA and data analysts when necessary
- Automating data ingestion processes
- Enhance data sets by creating automated match rules
- Automating data pre-processing, cleansing and verifying the integrity
- Developing ETL transformations for data from various data sources
- Working in a feature team of developers and test engineers, deliver results every sprint in accordance with sprint and release plans; report about activities via agreed project tools, timely report about risks/issues/dependencies, apply efforts to resolve and move the work forward
- 5+ years of hands-on experience in a Data Engineer (or similar) role
- Strong experience with ETL
- Knowledge of Spark
- Familiarity with Databricks
- Experience with Redshift
- S3 knowledge
- Knowledge of Python
- Experience with SQL
- Proficient working in AWS cloud environment
- 1+ year of relevant leadership experience
- Proficient English (written and spoken) B2
looking for something else?
Find a vacancy that works for you. Send us your CV to receive a personalized offer.