Edtech.com's Summary

Elsevier is hiring a Senior Data Engineer II. The role leads the design and operation of scalable data pipelines for Libra, a team within Elsevier’s Search and AI Platform that is building an evaluation tool for subject matter experts to review AI-generated medical content. The engineer also sets engineering standards, mentors other engineers, and works with product managers and architects on delivery roadmaps.

Highlights

Lead the design, prototyping, and build of scalable data pipelines that ingest content into a centralized content storage system and generate reports across Libra services, setting clean-code standards for the team.
Build with Python (FastAPI), Java (Spring Boot), Spark, Airflow, Snowflake, Kafka, Iceberg, and ETL/ELT tooling, with SQL and data warehouse design as core skills.
Own the AWS cloud infrastructure that hosts, monitors, and scales the services, automating routine tasks and driving cost and performance optimisation.
Partner with product managers, architects, and engineering teams to turn product requirements into technical solutions, implementation plans, and delivery roadmaps.
Mentor and coach engineers through design reviews, pair programming, and code review to raise the team’s technical bar.
Drive data quality, observability, governance, and reliability practices across Libra data products, including SLAs, lineage, and monitoring.
Contribute to architectural decisions for the wider Search and AI Platform and influence technical direction beyond the immediate team.
At least 6 years of professional data engineering experience building scalable, reliable data platforms, including deep SQL and data warehouse experience, preferably with Snowflake, and end-to-end technical delivery of non-trivial projects.
Familiarity with AWS Glue, CI/CD, testing strategy, and architectural design patterns is expected; experience with large-scale content management systems, infrastructure as code (GitHub Actions, Terraform), and search or graph databases is a plus.

Senior Data Engineer II Full Description

Are you a Senior Data Engineer II looking for an innovative role?

Do you enjoy being part of a team that works with a diverse range of technology?

About our Team: -

The Search and AI Platform is Elsevier’s agentic data platform which powers products such as LeapSpace, Scopus, ScienceDirect, Clinical Key, Reaxys, Libra and our next-generation AI powered search systems.

The platform uses agentic services to interrogate our rich knowledge graphs, search and recommendation systems, and our unparalleled collection of research data to deliver insights to the scientific community so they can collaborate more effectively, work smarter, and deliver quality research more quickly.

We’re looking for an innovative, passionate Senior Data Engineer II in the Libra team to lead the design, build and operation of scalable pipelines that ingest content into a centralized content storage system. The Libra systems are in early-stage implementation and will handle massive volumes of content, so we are looking for someone who can take technical ownership of the architecture, build and maintenance of scalable and reliable data pipelines and services for various kinds of report generation.

About the Role: -

The Libra Team is a core part of Elsevier’s Search and AI Platform. This team works on building an Evaluation Platform for Subject Matter Experts (SMEs) to review AI-generated medical content item-by-item, provide ratings/feedback based on predefined metrics, track team progress, and export results for quality assurance and model improvement purposes. As a Senior Data Engineer II, you will lead the technical design and delivery of robust, scalable and reliable data pipelines and services for report generation across Libra services, set engineering standards within the team, and mentor and grow other engineers.

Responsibilities: -

Leading the design, prototyping and build of robust and scalable data pipelines, setting the standard for clean code and best-practice software engineering principles across the team.
Working with technologies including Python (Fast API), Java (Spring Boot), Data Warehouse, Spark, Airflow, Snowflake, ETL/ELT, Iceberg, Kafka and RDBMS.
Owning cloud infrastructure in AWS to host, monitor and scale the services, automating common tasks mercilessly and driving cost and performance optimisation.
Partnering with Product Managers, architects and engineering teams to translate product requirements into scalable technical solutions, implementation plans and delivery roadmaps.
Mentoring and coaching engineers through design reviews, pair programming and code review, raising the technical bar of the team.
Driving data quality, observability, governance and reliability practices across Libra data products, including SLAs, lineage and monitoring.
Contributing to architectural decisions for the wider Search and AI Platform and influencing technical direction beyond your immediate team.

Requirements: -

6+ years of professional experience in data engineering, with a strong track record of building scalable, high-performance and reliable data platforms and services.
Strong core skill set including SQL, Python, Airflow, Data Warehousing, Snowflake, AWS, Git, ETL/ELT and familiarity with AWS Glue.
Deep experience with SQL and data warehouses, preferably Snowflake, including data modelling, performance tuning and cost management at scale.
An advocate and enforcer of best practices for rapid development of quality software, including clean code, architectural design patterns, testing strategy and CI/CD.
Comfortable learning new technologies, languages and tools on the job, and helping others do the same, to ensure the product is not left behind in a rapidly evolving ecosystem.
Demonstrated experience leading the technical delivery of non-trivial projects end to end, from design through to production support.
A desire to operate in a senior data engineering role, including deputising for the technical lead, owning design and implementation of data systems and pipelines, and setting engineering standards.
An inclusive, positive, collaborative mindset and a desire to deliver real business value to our customers.

Nice to have

Experience with large-scale content management systems and big data platforms.
Experience leading architectural decisions within a team or across teams.
Familiarity with CI/CD pipelines and infrastructure as code (GitHub Actions, Terraform).
Exposure to search, graph, or large-scale analytical databases.
Strong ownership mindset and proven ability to drive outcomes in ambiguous problem spaces.

Work in a way that works for you: -

Working flexible hours - flexing the times when you work in the day to help you fit everything in and work when you are the most productive.

We are an equal opportunity employer: qualified applicants are considered for and treated during employment without regard to race, color, creed, religion, sex, national origin, citizenship status, disability status, protected veteran status, age, marital status, sexual orientation, gender identity, genetic information, or any other characteristic protected by law.

Let me find your next job.

Thanks - you're signed up.

Senior Data Engineer II

Edtech.com's Summary

Senior Data Engineer II Full Description

Sr Data Research Analyst II

Principal Analytics Engineer - Global Reporting & Analytics

Data Engineer, Lead

Senior Data Engineering

Data Analyst I