senior big data engineer for a healthcare company
We are currently looking for a remote Senior Big Data Engineer with 3+ years of experience in Data Engineering to join our team.
The customer is an American multinational company working in the combined industries of health information technology and clinical research.
- Data transformation and warehousing
- Analyze and correlate dataset to user profile
- Create the required technical tasks in backlog and update status regularly
- Provide Production Support for any issues raised by the Business and experience on Production/Integration release management
- Recommend changes to client software based on information such as client feedback, language changes and new technology
- developing Shell scripts to orchestrate execution of all other scripts and move the data files within and outside of HDFS
- Participate in all Agile ceremony meetings and update task status and adhere to timelines and quality standards for the task
- Working with business and peers to define, estimate, and deliver functionality
- 3+ years of Big Data Engineering experience
- In-depth understanding/knowledge of Hadoop & Spark Architecture and its components such as HDFS, Job Tracker, Task Tracker, executor cores and memory parameters
- Experience in Hadoop development, working experience on SPARK, SCALA is mandatory and Database exposure is must
- Hands-on experience in Spark and Spark Streaming creating RDD's, applying operations -Transformation and Actions
- Experience in code optimize to fine tune the applications
- Expertise in writing Hadoop/spark Jobs for analyzing data using Spark, Scala, Hive, Kafka and Python
- Experience with streaming workflow operations
- Experience with developing large-scale distributed applications and developing solutions to analyze large data sets efficiently
- Experience in Data Warehousing and ETL processes
- Strong database, SQL, ETL and data analysis skills
- Experienced in developing scripts for doing transformations using Scala
- Experience in Using Kafka on publish-subscribe messaging as a distributed commit log, have experienced in its fast, scalable and durability
- Able to understand existing business logic & implement the business changes
- Should have gone through entire development life cycle including system integration testing
- Right communication skills to be able to work with Business
- Good logical reasoning, analytical abilities
- Experience working with remote teams
- Experienced in Agile development methodology
- Proficient English (written and spoken) B2
looking for something else?
Find a vacancy that works for you. Send us your CV to receive a personalized offer.