senior data software engineer
We are seeking a remote Senior Data Software Engineer with experience in PySpark, Azure Data Factory, and advanced SQL for the project.
The ideal candidate must have at least 2 years of solid/hands-on experience in Data Software Engineering with primary skills in Big Data and a strong data background.
- Conduct data analysis and troubleshootin
- Plan and implement new requirements/data entities on ED
- Provide support for integration testing
- Make sure data pipelines are scalable and efficient
- 3+ years of relevant work experience
- Must have skills in DSE Python and Azure Databricks
- Familiarity with EDL changes in DB Views/Stored procedures and integration testing support
- Advanced knowledge of PySpark, Azure Data Factory, and SQL
- Ability to collaborate effectively with the team
- Excellent communication skills with an upper-intermediate level of English
nice to have
- Experience with HDInsight, Azure Data Lake, Data API, Spark, Scala, and Kafka will be an added advantage