lead big data engineer for an insurance company
Currently, we are looking for a remote Lead Big Data Engineer with 5+ years of Big Data engineering experience, and Hands-on experience with Cloudera Hadoop stack to join our team.
Our customer offers automobile, property, liability, agriculture, and surety insurance.
- Contribute to designing, developing, and monitoring systems and solutions for collecting, storing, processing, and analyzing large data sets
- Work in a team made up of other Data Engineers
- Understand and implement certain platform capabilities
- Understand coding standards and implement them within code being developed
- 5+ years of Big Data engineering experience
- Hands-on experience with Cloudera Hadoop stack including Hive and Impala performance optimization
- Data Quality management knowledge is highly desirable
- Experience with real-time data processing technologies: Flume, Message Queues (preferably IBM WebSphere MQ)
- 3+ years' experience with Hadoop stack: HDFS, Hive, Impala
- 3+ years' experience with Spark Scala
- 2+ year of ETL tools experience, preferably Pentaho
- Experience with Cloudera 6+
- Knowledge of SQL/HQL
- Experience with scripting (bash preferably or Perl)
- Knowledge of RDBMS technologies (Oracle preferably and experience with SQL Server, SSRS - reporting, SSIS - ETL)
- Experience with AWS or Cloud technologies
- Experience in messaging tech – MQ or Kafka or both and general cloud awareness would be extremely helpful
- Knowledge of Insurance field and Guidewire system
- Data Modelling CA Erwin/Sparx
- Experience in OOZIE workflow
- Proficient English (written and spoken) B2
looking for something else?
Find a vacancy that works for you. Send us your CV to receive a personalized offer.