We are seeking a talented remote Senior Data Engineer with strong experience in Data Software Engineering.
The ideal candidate will have Big Data technologies, primarily Spark, work with cross-functional teams to design, develop, and deploy high-performance data pipelines.
responsibilities
Design, develop and deploy high-performance data pipelines
Work with cross-functional teams to deliver complex projects
Develop, maintain and optimize ETL processes
Troubleshoot and optimize data pipeline processes
Create and maintain data models and databases
requirements
3+ years of software engineering experience
Good knowledge of Big Data technologies, primarily Spark
We are seeking a remote Senior Data Software Engineer to join our team.
The right candidate will have a strong technical background with expertise in Apache Spark, Microsoft Azure, and Python. This position offers a unique opportunity to work on a high-impact project with one of the world's most recognized brands.
responsibilities
Collaborate with cross-functional teams to build and maintain the data integration solution
Develop, build, and optimize data pipelines using Apache Spark, Microsoft Azure, and Python
Ensure data pipelines are scalable, maintainable, and reliable
Take data science models and make them production ready
Develop and maintain forecasting models to support business decisions
Ensure data quality and consistency across all data sources
Monitor and optimize data pipelines to ensure efficient and effective data processing
Collaborate with data scientists to develop and implement machine learning models
Work with the team to continuously improve and optimize the data integration solution for Fedex
Stay current with emerging technologies and trends in data software engineering
requirements
At least 3+ years of experience in data software engineering or similar roles
Expertise in Apache Spark, Microsoft Azure, and Python
Strong knowledge of forecasting models, data science, and MLOps
Experience working with Databricks to build, maintain, and optimize data pipelines
Ability to take data science models and make them production ready
Experience with Git for version control
Understanding of basic Azure concepts including clouds, regions, ADLS, and compute
Strong analytical and problem-solving skills with the ability to think critically and creatively
Experience working in Agile development environments
Excellent English communication skills, both written and verbal (B2+ level)
nice to have
Experience with Panda for data manipulation and analysis
Strong knowledge of SQL and relational tables
Understanding of statistical models and the ability to develop models utilizing Python, Spark, etc.
Knowledge of ML tools space
get job alerts in your inboxHundreds of open jobs for Software Engineers, QA, DevOps, Business Analysts and other tech professionals
We are seeking a highly skilled Senior Data Software Engineer to join our remote team, working on a tech-enabled healthcare delivery system focused on improving the health of diverse populations.
We are seeking a highly skilled Senior Data Software Engineer to join our remote team, working on an integrated, holistic, and tech-enabled healthcare delivery system focused on improving the health of diverse populations. As a Senior Data Software Engineer, you will be responsible for designing and maintaining data integration pipelines and ETL/ELT workflows, implementing end-to-end data solutions. You will work closely with stakeholders to analyze data requirements and collaborate with the data science team to identify optimal solutions for problems related to data quality, reliability, and performance.
responsibilities
Design and maintain data integration pipelines and ETL/ELT workflows
Develop and implement end-to-end Data solutions to meet business requirements
Ensure that data solutions are scalable, secure, and optimized for performance
Work with stakeholders to analyze data requirements and implement data models that support business objectives
Collaborate with the data science and engineering team members to identify optimal solutions for problems related to data quality, reliability, and performance
requirements
3+ years of experience in Data Software Engineering, demonstrating your expertise in AWS, Apache Spark, PySpark
Demonstrated experience in designing and maintaining data integration pipelines and ETL/ELT workflows
Proficient in implementing best practices for data management, data governance, quality control, and data security
Ability to work independently and manage stress effectively, maintaining a high level of performance even under pressure
Fluent spoken and written English at a B2 level or higher
Currently, we are looking for a Remote Senior Python Developer to join our team.
The customer is a premier asset management firm, focused on delivering unique insight and partnership for the most sophisticated global institutional investors.
responsibilities
Query tuning, performance tuning, troubleshooting, and debugging Spark or other big data solutions
Build use cases that use data - such as risk modeling, fraud detection, customer life-time value
Experience with design and implementation of big data technologies such as Spark/Delta, Hadoop, and NoSQL
Maintain and extend production data systems to evolve with complex needs
requirements
3+ years of experience with Python.Core
Production programming experience in Python or Scala
Deep Expertise with Spark Structured Streaming
Apache Spark experience
Big Data experience
nice to have
Experience scaling big data workloads that are performant and cost-effective
Experience with Development Tools for CI/CD, Unit and Integration testing, Automation and Orchestration, REST API, BI tools and SQL Interfaces. E.g. GitLab
Experience designing data solutions on cloud infrastructure and services, such as AWS, Azure, or GCP using best practices in cloud security and networking
0
latest insights
01
career advicehow to become a senior software engineer: a guideLearn how to grow to a senior software engineer role, how long does it take, and whether you have the skills to become one.read more
02
career advicecreative Salesforce portfolio project ideasExplore our dynamic collection of Salesforce portfolio projects and get inspiration from ideas to win your next top developer job.read more
03
tips & toolsrebase vs. merge: a comprehensive guideUnderstand the techniques of rebase vs merge, discover the pros, cons, and when best to use git rebase vs git merge in this comprehensive guide.read more
