Edtech.com's Summary

Weekday AI is hiring an AI Rater Guidelines Writer (Linguist / Instructional Designer). The role involves creating and refining clear, precise rating guidelines and evaluation rubrics for human evaluators to ensure high-quality training data for advanced AI systems across multiple domains. The writer converts complex requirements into structured instructions, collaborates with experts, and improves documentation clarity to support consistent AI evaluations.

Highlights

Create and refine rater guidelines, evaluation rubrics, and reviewer instructions for AI training quality.
Translate complex and ambiguous program requirements from diverse domains such as finance, retail, insurance, legal, and sports into clear guidance.
Review and revise documentation to eliminate ambiguities, inconsistencies, and coverage gaps.
Collaborate with research teams, program managers, and subject matter experts to maintain guideline consistency.
Develop documentation addressing edge cases and complex evaluation scenarios to minimize reviewer escalation.
Required experience includes minimum 3 years in Linguistics, Instructional Design, Technical Writing, or related fields, with direct AI evaluation or annotation guideline experience.
Strong written communication and analytical skills with the ability to clarify nuanced concepts.
Commitment of approximately 35 hours per week, fully remote, independent contractor role.
Compensation ranges from $45 to $65 per hour.
Preferred candidates have background in AI model training, RLHF, cross-functional collaboration, and structured documentation standards.

AI Rater Guidelines Writer (Linguist / Instructional Designer) Full Description

This role is for one of our clients

Compensation: $45-$65 per hour

Join a cutting-edge AI initiative focused on building the next generation of foundational AI models. We are seeking experienced Linguists, Instructional Designers, Technical Writers, and AI Content Specialists who excel at transforming complex, ambiguous program requirements into clear, structured, and actionable guidance for human evaluators.

In this role, you will play a critical part in improving AI training quality by creating precise rating guidelines, evaluation rubrics, and reviewer instructions across diverse domains including finance, retail, insurance, legal, and sports. Your expertise will help ensure consistency, accuracy, and reliability in human evaluations that directly shape advanced AI systems.

This is a full-time, fully remote engagement requiring a commitment of approximately 35 hours per week during standard weekdays.

Requirements

Key Responsibilities

Translate complex and ambiguous program requirements into clear, concise, and easy-to-follow rater guidelines.
Design comprehensive evaluation rubrics and scoring frameworks that enable consistent decision-making across a variety of AI tasks.
Review existing documentation to identify ambiguity, inconsistencies, contradictions, and coverage gaps, then revise guidelines to improve clarity and usability.
Convert subject-matter requirements from domains such as finance, retail, insurance, legal, and sports into structured evaluation instructions that non-domain raters can apply confidently.
Collaborate with research teams, program managers, and subject matter experts to maintain consistency across multiple guideline sets.
Develop documentation that addresses edge cases, exceptions, and complex evaluation scenarios while minimizing reviewer escalation.

Required Qualifications

Minimum 3 years of professional experience in Linguistics, Instructional Design, Technical Writing, AI Content Development, or a closely related field.
Direct experience creating, refining, or maintaining evaluation guidelines, rating rubrics, or reviewer instructions within Generative AI, RLHF, human evaluation, or AI data annotation environments.
Demonstrated ability to work across multiple subject areas and convert domain-specific knowledge into clear, structured documentation.
Proven experience resolving ambiguity and improving written specifications with measurable before-and-after improvements.
Strong analytical thinking with exceptional attention to detail.
Demonstrated career progression and professional growth.
Ability to commit reliably to 35+ hours per week during weekdays.
Outstanding written communication skills with the ability to explain nuanced concepts in a precise, consistent, and easy-to-understand manner.

Preferred Qualifications

Experience supporting AI model training, evaluation, RLHF, or large-scale annotation programs.
Background working with cross-functional teams including researchers, engineers, product managers, and subject matter experts.
Familiarity with structured documentation standards, quality assurance methodologies, and guideline governance.
Experience designing documentation that supports scalable, high-quality human evaluation processes.

Why Join

Help define how human evaluators assess next-generation AI systems.
Influence the quality and consistency of AI training data across multiple industries.
Collaborate with multidisciplinary experts working on advanced Generative AI initiatives.
Solve complex language and reasoning challenges while improving AI evaluation standards at scale.
Enjoy the flexibility of a fully remote engagement while contributing to high-impact AI research.

Equal Opportunity

We are committed to creating an inclusive workplace and welcome applications from qualified professionals of all backgrounds. Reasonable accommodations are available throughout the application and engagement process.

Contract & Engagement Details

Independent contractor engagement.
Fully remote with flexible working arrangements.
Expected commitment of approximately 35 hours per week during weekdays.
Project duration may vary depending on business requirements and individual performance.
Work does not require access to confidential or proprietary information from any current or former employer.
Payments are issued weekly based on approved work completed.
At this time, we are unable to support H1-B or STEM OPT candidates.

Original Job Description

This role is for one of our clients

Compensation: $45-$65 per hour

This is a full-time, fully remote engagement requiring a commitment of approximately 35 hours per week during standard weekdays.

Requirements

Key Responsibilities

Translate complex and ambiguous program requirements into clear, concise, and easy-to-follow rater guidelines.
Design comprehensive evaluation rubrics and scoring frameworks that enable consistent decision-making across a variety of AI tasks.
Review existing documentation to identify ambiguity, inconsistencies, contradictions, and coverage gaps, then revise guidelines to improve clarity and usability.
Convert subject-matter requirements from domains such as finance, retail, insurance, legal, and sports into structured evaluation instructions that non-domain raters can apply confidently.
Collaborate with research teams, program managers, and subject matter experts to maintain consistency across multiple guideline sets.
Develop documentation that addresses edge cases, exceptions, and complex evaluation scenarios while minimizing reviewer escalation.

Required Qualifications

Minimum 3 years of professional experience in Linguistics, Instructional Design, Technical Writing, AI Content Development, or a closely related field.
Direct experience creating, refining, or maintaining evaluation guidelines, rating rubrics, or reviewer instructions within Generative AI, RLHF, human evaluation, or AI data annotation environments.
Demonstrated ability to work across multiple subject areas and convert domain-specific knowledge into clear, structured documentation.
Proven experience resolving ambiguity and improving written specifications with measurable before-and-after improvements.
Strong analytical thinking with exceptional attention to detail.
Demonstrated career progression and professional growth.
Ability to commit reliably to 35+ hours per week during weekdays.
Outstanding written communication skills with the ability to explain nuanced concepts in a precise, consistent, and easy-to-understand manner.

Preferred Qualifications

Experience supporting AI model training, evaluation, RLHF, or large-scale annotation programs.
Background working with cross-functional teams including researchers, engineers, product managers, and subject matter experts.
Familiarity with structured documentation standards, quality assurance methodologies, and guideline governance.
Experience designing documentation that supports scalable, high-quality human evaluation processes.

Why Join

Help define how human evaluators assess next-generation AI systems.
Influence the quality and consistency of AI training data across multiple industries.
Collaborate with multidisciplinary experts working on advanced Generative AI initiatives.
Solve complex language and reasoning challenges while improving AI evaluation standards at scale.
Enjoy the flexibility of a fully remote engagement while contributing to high-impact AI research.

Equal Opportunity

Contract & Engagement Details

Independent contractor engagement.
Fully remote with flexible working arrangements.
Expected commitment of approximately 35 hours per week during weekdays.
Project duration may vary depending on business requirements and individual performance.
Work does not require access to confidential or proprietary information from any current or former employer.
Payments are issued weekly based on approved work completed.
At this time, we are unable to support H1-B or STEM OPT candidates.

Let me find your next job.

Thanks - you're signed up.

AI Rater Guidelines Writer (Linguist / Instructional Designer)

Edtech.com's Summary

AI Rater Guidelines Writer (Linguist / Instructional Designer) Full Description

Senior Instructional Designer

Learning Experience Designer

Instructional Systems Designer

Associate Instructional Designer

Instructional Design Consultant 1