Edtech.com's Summary

MagicSchool is hiring a Senior Backend Engineer- Evaluations to join their Trust, Safety and Quality team and enhance their Generative AI Evaluations capabilities. The role involves designing and building scalable, resilient backend systems, performing data modeling, and collaborating cross-functionally to ensure the safety, speed, and quality of the evaluation platform.

Highlights

Build and maintain tools and processes for evaluating AI output at scale.
Design, architect, and write high-quality, maintainable Python code using modern features like typehints, asyncio, and multiprocessing.
Data modeling experience with SQL (PostgreSQL), relational databases, data warehouses, and object stores.
Debug complex cloud-based applications and deploy Dockerized applications.
Collaborate closely with product managers, data scientists, and engineers for feature delivery.
Assess and integrate appropriate third-party solutions when necessary.
Qualifications include 7+ years engineering experience, with at least 2 years at senior level, startup background, and experience with LLMs and Generative AI APIs.
Preferred skills include evaluating generative AI systems, TypeScript/React, and AWS technologies (bedrock, E(C|K)S).
Company offers flexible remote work, unlimited time off, stock options vested over 4 years, health insurance, 401k match, and monthly wellness stipend.
Emphasizes values such as educator empowerment, innovation, responsibility, diversity, community, and excellence in education technology.

Senior Backend Engineer- Evaluations Full Description

WHO WE ARE: MagicSchool is the premier generative AI platform for teachers. We're just over 2 years old, and more than 6 million teachers from all over the world have joined our platform. Join a top team at a fast growing company that is working towards real social impact. Make an account and try us out at our website and connect with our passionate community on our Wall of Love.

As a Senior Backend Engineer- Evaluations, you will work with the Trust, Safety and Quality team as a strong individual contributor to expand our Generative AI Evaluations capabilities. This work involves both application and data engineering. You will feel a sense of ownership over the space far beyond just taking tickets. You will obsess over safety, quality, speed and user impact. You will know when it's time to build versus time to buy and integrate. You will be passionate about building resilient scalable systems to ensure our users have the best experience possible.

What You Will Do:

Ensure we're building the safest product MagicSchool can build.
- Build tools and processes for evaluating MagicSchool output at scale
- Stay up to date on the latest tools and literature
- Passionate about solving education problems with technology
Work closely with Evaluations product manager, team lead, data scientist and engineer and deliver high quality features
Design, architect, and write high quality code to expand our Evaluations framework's capabilities
- Debug complex code and applications in a cloud environment
- Build software that is easy for others to understand and easy to maintain
Data modeling in a relational database, data warehouse, and object stores.
Assess and help the team choose appropriate third party solutions to problems when building doesn't make sense
Work cross-functionally to integrate MagicSchool's evaluations platform with the larger application

Qualifications/Competencies/Skills:

Working knowledge of SQL (PostgreSQL) and database design / data modeling
Expert knowledge of Python
- Modern Python features like typehints and asyncio
- Python Multiprocessing to support evaluation jobs in the 10s to 100s of thousands per run.
- Familiarity with Pydantic
Strong communication skills: team-first mindset, highly collaborative, can articulate decisions within team's context
Experience working with and deploying docker applications
Builds relationships easily: emotionally intelligent, communication, warm
Gets a lot done: Works hard, resourceful, do whatever it takes
Adaptable: Smart, learns fast, curious
Nice to Have:
- Experience with evaluating generative AI systems
- TypeScript and React (useful for internal tooling and cross-team contributions)
- AWS experience (bedrock, E(C|K)S)

Experience:

7+ years as an engineer, 2 of which were at senior level or above
Start up experience
Experience with LLMs and Generative AI APIs and systems

Why Join Us?

Work on cutting-edge AI technology that directly impacts educators and students.
Join a mission-driven team passionate about making education more efficient and equitable.
Flexibility of working from home, while fostering a unique culture built on relationships, trust, communication, and collaboration with our team - no matter where they live.
Unlimited time off to empower our employees to manage their work-life balance. We work hard for our teachers and users, and encourage our employees to rest and take the time they need.
Choice of employer-paid health insurance plans so that you can take care of yourself and your family. Dental and vision are also offered at very low premiums.
Every employee is offered generous stock options, vested over 4 years.
Plus a 401k match & monthly wellness stipend

Our Values:

Educators are Magic: Educators are the most important ingredient in the educational process - they are the magic, not the AI. Trust them, empower them, and put them at the center of leading change in service of students and families.
Joy and Magic: Bring joy and magic into every learning experience - push the boundaries of what’s possible with AI.
Community: Foster community that supports one another during a time of rapid technological change. Listen to them and serve their needs.
Innovation: The education system is outdated and in need of innovation and change - AI is an opportunity to bring equity, access, and serve the individual needs of students better than we ever have before.
Responsibility: Put responsibility and safety at the forefront of the technological change that AI is bringing to education.
Diversity: Diversity of thought, perspectives, and backgrounds helps us serve the wide audience of educators and students around the world.
Excellence: Educators and students deserve the best - and we strive for the highest quality in everything we do.

Original Job Description

What You Will Do:

Ensure we're building the safest product MagicSchool can build.
- Build tools and processes for evaluating MagicSchool output at scale
- Stay up to date on the latest tools and literature
- Passionate about solving education problems with technology
Work closely with Evaluations product manager, team lead, data scientist and engineer and deliver high quality features
Design, architect, and write high quality code to expand our Evaluations framework's capabilities
- Debug complex code and applications in a cloud environment
- Build software that is easy for others to understand and easy to maintain
Data modeling in a relational database, data warehouse, and object stores.
Assess and help the team choose appropriate third party solutions to problems when building doesn't make sense
Work cross-functionally to integrate MagicSchool's evaluations platform with the larger application

Qualifications/Competencies/Skills:

Working knowledge of SQL (PostgreSQL) and database design / data modeling
Expert knowledge of Python
- Modern Python features like typehints and asyncio
- Python Multiprocessing to support evaluation jobs in the 10s to 100s of thousands per run.
- Familiarity with Pydantic
Strong communication skills: team-first mindset, highly collaborative, can articulate decisions within team's context
Experience working with and deploying docker applications
Builds relationships easily: emotionally intelligent, communication, warm
Gets a lot done: Works hard, resourceful, do whatever it takes
Adaptable: Smart, learns fast, curious
Nice to Have:
- Experience with evaluating generative AI systems
- TypeScript and React (useful for internal tooling and cross-team contributions)
- AWS experience (bedrock, E(C|K)S)

Experience:

7+ years as an engineer, 2 of which were at senior level or above
Start up experience
Experience with LLMs and Generative AI APIs and systems

Why Join Us?

Work on cutting-edge AI technology that directly impacts educators and students.
Join a mission-driven team passionate about making education more efficient and equitable.
Flexibility of working from home, while fostering a unique culture built on relationships, trust, communication, and collaboration with our team - no matter where they live.
Unlimited time off to empower our employees to manage their work-life balance. We work hard for our teachers and users, and encourage our employees to rest and take the time they need.
Choice of employer-paid health insurance plans so that you can take care of yourself and your family. Dental and vision are also offered at very low premiums.
Every employee is offered generous stock options, vested over 4 years.
Plus a 401k match & monthly wellness stipend

Our Values:

Educators are Magic: Educators are the most important ingredient in the educational process - they are the magic, not the AI. Trust them, empower them, and put them at the center of leading change in service of students and families.
Joy and Magic: Bring joy and magic into every learning experience - push the boundaries of what’s possible with AI.
Community: Foster community that supports one another during a time of rapid technological change. Listen to them and serve their needs.
Innovation: The education system is outdated and in need of innovation and change - AI is an opportunity to bring equity, access, and serve the individual needs of students better than we ever have before.
Responsibility: Put responsibility and safety at the forefront of the technological change that AI is bringing to education.
Diversity: Diversity of thought, perspectives, and backgrounds helps us serve the wide audience of educators and students around the world.
Excellence: Educators and students deserve the best - and we strive for the highest quality in everything we do.

Let me find your next job.

Thanks - you're signed up.

Senior Backend Engineer- Evaluations

Edtech.com's Summary

Senior Backend Engineer- Evaluations Full Description

Staff Software Engineer, Gen AI

Senior Software Engineer (AI Applications)

Software Engineer I

Senior Engineering Manager - Martech

Senior Platform Engineer