Software Engineer I (Coding)

Edtech.com's Summary

Handshake is hiring a Software Engineer I (Coding) to build and maintain data infrastructure and pipelines that support AI coding models. The role involves designing scalable data pipelines, developing evaluation and quality assurance systems, and collaborating with cross-functional teams to advance AI-powered software development.

Highlights

Design and build scalable data pipelines for large-scale coding datasets generation and validation
Develop automated and human-in-the-loop evaluation workflows and quality assurance systems
Integrate with developer ecosystems like GitHub and work with containerized environments such as Docker
Build backend systems and APIs for dataset delivery and model evaluation
Collaborate with machine learning researchers, product managers, and engineers to improve dataset quality
Implement automated grading, benchmarking, and assessment systems for coding tasks
Experience required: 2+ years in software engineering, strong programming skills in Python or TypeScript, and familiarity with data pipelines, ETL, containerization, and cloud infrastructure (AWS, GCP, or similar)
Preferred skills include working with machine learning datasets, coding agents, workflow orchestration tools, and developer tooling APIs
Compensation includes equity, competitive pay, 401(k) match, paid parental leave, health benefits, wellness stipend, learning stipend, and flexible PTO
Collaborate across engineering, ML research, and product teams supporting AI-driven coding model benchmarks

Software Engineer I (Coding) Full Description

About Handshake

Handshake is the career network for the AI economy. 20 million knowledge workers, 1,600 educational institutions, 1 million employers (including 100% of the Fortune 50), and every foundational AI lab trust Handshake to power career discovery, hiring, and upskilling, from freelance AI training gigs to first internships to full-time careers and beyond. This unique value is leading to unparalleled growth; in 2025, we tripled our ARR at scale.

Why join Handshake now:

Shape how every career evolves in the AI economy, at global scale, with impact your friends, family and peers can see and feel

Work hand-in-hand with world-class AI labs, Fortune 500 partners and the worldâs top educational institutions

Join a team with leadership from Scale AI, Meta, xAI, Notion, Coinbase, and Palantir, among others

Build a massive, fast-growing business with billions in revenue

About the Role

As a Software Engineer on our Coding Pod, you will build the data infrastructure and pipelines that power frontier AI coding models. This team sits at the intersection of applied machine learning, data engineering, and product developmentâcreating large-scale, high-quality benchmark datasets that evaluate how models perform on real-world, economically valuable coding tasks.

Youâll own end-to-end data pipelinesâfrom task generation and dataset construction to quality assurance and deliveryâhelping define how next-generation coding models are trained and evaluated. Youâll also work with human-in-the-loop evaluation systems, agentic coding tools, and automated assessment frameworks to ensure dataset quality and model performance.

This role is ideal for engineers who enjoy working across backend systems, data pipelines, and developer tooling, and want direct impact on the future of AI-powered software development.

Location: San Francisco, CA | 5 days/week in-office

What Youâll Do

Design and build scalable data pipelines for generating, transforming, and validating large-scale coding datasets
Develop systems for task generation, dataset curation, and quality assurance, including automated and human-in-the-loop evaluation workflows
Integrate with developer ecosystems (e.g., GitHub) and build tooling that supports real-world coding environments
Work with containerized environments (e.g., Docker) to safely execute and evaluate code at scale
Build backend systems and APIs that power dataset delivery and model evaluation pipelines
Collaborate closely with ML researchers, product managers, and other engineers to define evaluation methodologies and improve dataset quality
Implement automated grading, benchmarking, and assessment systems for coding tasks
Debug and optimize pipeline performance, reliability, and scalability across distributed systems
Contribute to architectural decisions around data infrastructure, evaluation systems, and pipeline orchestration

Desired Capabilities

2+ years of professional software engineering experience, with exposure to backend systems or data engineering
Strong programming skills (e.g., Python, TypeScript, or similar)
Experience building or maintaining data pipelines, ETL systems, or distributed systems
Familiarity with containerization (e.g., Docker) and cloud infrastructure (AWS, GCP, or similar)
Understanding of databases and data modeling (SQL or NoSQL)
Comfort working across systems boundaries, from infrastructure to developer-facing tools
Strong problem-solving skills and attention to detail, especially in data quality and correctness
Effective communication skills and ability to collaborate cross-functionally

Extra Credit

Experience working with machine learning datasets, evaluation frameworks, or benchmarking systems
Familiarity with coding agents, developer tools, or AI-assisted programming systems
Experience integrating with APIs like GitHub or building developer platform tooling
Exposure to workflow orchestration tools (e.g., Airflow, Temporal) or distributed job systems
Experience designing automated testing or grading systems for code
Background in high-growth or infrastructure-heavy engineering environments
Interest in the intersection of AI, developer productivity, and real-world software engineering workflows

Perks

Handshake delivers benefits that help you feel supportedâand thrive at work and in life.

The below benefits are for full-time US employees.

ð¯ Ownership: Equity in a fast-growing company

ð° Financial Wellness: 401(k) match, competitive compensation, financial coaching

ð¼ Family Support: Paid parental leave, fertility benefits, parental coaching

ð Wellbeing: Medical, dental, and vision, mental health support, $500 wellness stipend

ð Growth: $2,000 learning stipend, ongoing development

ð» Remote & Office: Internet, commuting, and free lunch/gym in our SF office

ð Time Off: Flexible PTO, 15 holidays + 2 flex days

ð¤ Connection: Team outings & referral bonuses

Explore our mission, values, and comprehensive US benefits at joinhandshake.com/careers.

Original Job Description