This glossary defines the terms you meet while applying for and doing AI training work: rating, ranking, writing and checking AI responses.
Each entry has a plain definition, the platforms our own guides cover for it, and the DigiNo page that goes deeper.
Role counts come from the open roles on our AI training jobs list on 2026-11-03 that name the term in their title or description.
The DigiNo team writes and updates these definitions. New to the work? Start with The AI Training Guide.
Types of AI Training Work
RLHF
Reinforcement learning from human feedback. People compare or rank a model's responses, a reward model learns those preferences, and the main model is then trained to score well against it.
SFT (Supervised Fine-Tuning)
The training step where a model learns from examples people write: a prompt plus the response it should give. In AI training jobs this means writing demonstrations or ideal responses.
SFT vs RLHF
SFT teaches by example: the model learns to copy responses people wrote. RLHF teaches by preference: people judge the model's own responses and the model learns which kind they prefer. The InstructGPT method used SFT first, then RLHF.
Reward Model
A second model trained on people's rankings to predict which response a person would prefer. During RLHF it scores the main model, so your rankings shape it directly.
Post-Training
Everything done to a model after its first large-scale training, including SFT, RLHF and evaluation. Most AI training jobs are post-training work.
AI Evaluation
Judging AI responses against set criteria such as accuracy, safety and instruction following. It is the core task in most AI evaluator, rater and reviewer roles.
Prompt Writing
Writing the questions or instructions a model is tested on. Good training prompts are realistic, have a clear right answer and are hard enough that a model could get them wrong.
Ideal Response
The answer you would want every model to give to a prompt, written by a person. Projects also call it a golden response or reference answer, and a model's attempt is graded against it.
Response Rewrite
Editing a model's response until it fully meets the guidelines, instead of only rating it. The rewritten version becomes a training example.
Multi-Turn Conversation
A conversation with several back-and-forth turns between a person and the model. Multi-turn tasks check whether the model keeps context and still follows earlier instructions.
Chain-of-Thought Review
Checking the step-by-step reasoning a model shows before its answer, not only the final answer. The useful note is where the first wrong step happens.
Agent Evaluation
Reviewing what an AI agent did across a whole task: the steps it took, the tools it used and whether the result is right. The recorded sequence of steps is called a trajectory.
Tool Use Task
A task where the model has to call an outside tool, such as search, a calculator or an API. You judge whether it chose the right tool and used the result correctly.
Code Review Task
Reading code a model wrote and judging whether it runs, handles edge cases and does what was asked. Coding roles often add a short fix or explanation.
SWE Task
A software engineering task built from a real codebase, where a model must fix a bug or add a feature and tests check the result. SWE-Bench is a public benchmark made of tasks like these.
Rating and Review Methods
Rubric
A list of criteria a response is graded against, each one checkable on its own. Some roles grade responses with rubrics; others write the rubrics.
Side-by-Side Rating
Two responses to the same prompt are shown next to each other and you choose the better one, usually with how much better. Projects also call it SxS or pairwise comparison.
Preference Ranking
Ordering several responses to one prompt from best to worst. In the InstructGPT project, people ranked between 4 and 9 responses per prompt, and the rankings trained the reward model.
Likert Rating
A score on a fixed scale, often 1 to 5 or 1 to 7, for one quality such as helpfulness. The project defines what each point on the scale means.
Justification
The short written note that explains your rating. Reviewers use it to check the rating, so a correct rating with a vague justification can still be marked down.
Severity
How serious an error is, usually minor, major or critical. The guidelines set the levels, and severity often decides the final rating.
Guidelines
The project's written rules for how to rate, write or label. They change often, and following them counts for more than your own preference.
Edge Case
An unusual input that the guidelines do not clearly cover. Keeping notes on how you decided edge cases keeps your later work consistent.
Instruction Following
Whether a response does exactly what the prompt asked, including format, length and every constraint. It is one of the most common rating criteria.
Fact-Check Task
Checking each claim in a response against reliable sources before you rate it. Confident wording is not evidence.
Hallucination
A confident statement from a model that is false or made up, such as a fake citation or a wrong figure. Spotting hallucinations is a core rating skill.
Citation Check
Checking that a response's claims are supported by the sources it cites or was given. In Anthropic's research, crowdworkers often preferred responses with links that did not work, which is why this check exists.
Quality Checks and Agreement
Golden Dataset
A set of tasks with answers that experts have already agreed are correct. Teams use it to test models and to measure how accurate trainers are.
Gold Standard Task
A task with a known answer placed in your queue without a label. Your answers on these tasks help measure your accuracy.
Inter-Annotator Agreement
How often different trainers give the same answer on the same task. InstructGPT trainers agreed about 73% of the time, so some disagreement is normal and low agreement often points to unclear guidelines.
Calibration
Lining up your judgments with the project's standard, usually through practice tasks compared against reviewer answers. Some platforms run a calibration phase at the start of every project.
Quality Score
A platform's running measure of how good your work is, built from reviews and checks. Platforms that explain how they assign work point to it, along with past performance and availability.
Reviewer Level
The step up from doing tasks to checking other people's tasks. Work is often checked in layers: peer review, then operations or client checks. No platform publishes exactly how reviewers are picked.
Auditor
A senior checker who samples finished and reviewed work to measure quality across a project. Audit findings feed back into guidelines and scores.
Rework
A submitted task sent back for changes, or rejected in review. Platforms track it, and repeated rejections lower your standing.
Safety and Adversarial Testing
Red Teaming
Deliberately trying to make a model produce harmful, false or off-policy output, so the weak spots can be fixed. In AI training it is a writing and judging task, not hacking.
Jailbreak
A prompt written to trick a model into breaking its own safety rules. Red teaming roles write jailbreak attempts so the model can learn to refuse them.
Prompt Injection
Hidden instructions placed in content a model reads, such as a web page or a document, that try to take over its behavior. It matters most for AI agents that browse or use tools.
Adversarial Prompt
Any prompt designed to make a model fail: say something false, unsafe or against policy. Writing good ones takes real knowledge of the field.
Safety Rating
Judging whether a response could cause harm, and whether the model should have answered at all. The project's safety policy sets where the lines are.
Refusal
A response where the model declines to help. Raters judge whether the refusal was needed or whether the model turned down a safe request.
Content Moderation
Reviewing text, images or video and labeling what breaks a set of content rules. It is one of the older forms of data work that feeds AI systems.
PII Redaction
Removing personal details such as names, addresses and ID numbers from data before it is used for training. PII stands for personally identifiable information.
Data Labeling, Audio and Language Work
Data Annotation
Adding labels or notes to data so a model can learn from it, such as tags on images, text or audio. Many platforms use the term for rating work too.
Image Labeling
Tagging what an image shows, from a single category to every object in it. Medical and technical image work needs field experience.
Bounding Box
A rectangle drawn around an object in an image to mark where it is. It is one of the most common image labeling tasks.
Segmentation
Marking the exact outline of each object in an image, pixel by pixel, instead of drawing a simple box around it.
Multimodal Annotation
Work that mixes more than one kind of data, such as an image with a question about it, or audio with its transcript.
Transcription
Turning recorded speech into accurate text, often with timestamps and speaker labels. Projects set strict rules for filler words, noise and unclear audio.
Voice Recording Task
Reading set scripts or speaking naturally so the recordings can train speech models. Projects usually ask for a quiet room and a decent microphone.
Translation Review
Checking a model's translation for accuracy, tone and local fit, then correcting it. Localization goes further and adapts dates, units, idioms and cultural references.
Search Relevance Rating
Judging how well search results or ads match what a person searched for. Google says the ratings from its external Search Quality Raters do not directly change rankings.
Getting Hired
Domain Expert
A trainer hired for professional knowledge in one field, such as law, medicine or finance. Platforms also call it a subject matter expert, or SME.
Generalist
An AI training role that needs strong reading, writing and judgment but no specific profession. It is the usual way in without a field degree.
AI Interview
A screening interview run by an AI interviewer that asks questions about the role you applied for. Mercor says most of its AI interviews take about 20 minutes.
Practice Interview
A free mock AI interview that does not count toward any application. You can use it to test your setup and answers before the real one.
Skill Assessment
A test of the work itself, such as a quiz, a practice task or a written evaluation. On Mercor, most assessments allow only one submission.
Starter Assessment
DataAnnotation's entry test, chosen by track at sign-up. It can be taken only once, and it is reviewed for thoroughness and accuracy rather than speed.
Qualification Test
A project-level test you pass before you can work on that project. On some platforms the time spent on it is unpaid.
Certification
On most AI training platforms, passing the platform's own screening, not a paid outside course. The FTC warns about job scams that sell so-called training or certifications.
Work Trial
A paid, one-time test on a project-like task. Mercor's work trials typically take about four to six hours, and only one is paid per expert.
Identity Check
Proving you are who you say you are, usually with a government ID and a selfie or short video. Many platforms run it through Persona.
Onboarding
The setup after you are accepted: contract, ID check, tools and project training. Outlier says its onboarding takes 30 to 90 minutes.
Talent Pool
The group of approved people a platform invites to projects as work opens. Being in the pool does not guarantee work.
On the Job
Task Queue
The list of available tasks you claim work from. How full it is decides how much you can work that day.
Empty Queue (EQ)
When there are no tasks for you to claim. It is one of the most common complaints among AI trainers, and keeping a second platform active is the usual answer.
Project Pause
A project stops sending tasks, for a while or for good. A paused project sets your hours to zero, and when one ends you usually apply again for the next.
Offboarding
Being removed from a project, because it ended or because of quality or rule problems. Mercor says projects that end for completion, scope or budget reasons do not affect your standing.
Weekly Hours Cap
The most hours a contract lets you log in a week. On Mercor, each hourly contract sets one, and only the project lead or Mercor can change it.
Time Tracker
A desktop app that logs your working hours on hourly projects, and on some setups takes periodic screenshots. Time with no keyboard or mouse input counts as idle.
AI Use Policy
A platform's rules on using AI tools in your own work. Many ban using a chatbot to write answers, interview responses or justifications.
Pay, Contracts and Taxes
Listed Rate
The pay a platform shows on a role's own listing on the day it is checked. DigiNo shows pay as “listed at” with a date, because rates change and are not a promise of income.
Hourly vs Per Task
Hourly work pays for tracked time; per task work pays a set amount for each approved task. micro1 lists three models: hourly, pay per task, or a flat fee per approved task.
Independent Contractor
Most AI trainers work as independent contractors, not employees. That means no tax withheld, no benefits and no guaranteed hours.
Form 1099-NEC
The US form that reports what a platform paid you as a contractor. US citizens and residents get one from Mercor; non-US experts do not.
W-8BEN
The form non-US contractors fill in for US platforms to confirm they are not US taxpayers. UK contractors on Mercor normally use their National Insurance number on it.
Payout Schedule
When a platform pays for finished work, for example weekly on a fixed day. Each platform sets its own cycle.
Deel
A contractor payroll service some platforms use for contracts, ID checks and payouts. You then withdraw from Deel to your own bank or wallet.
Payout Fee
What a payment service keeps when it sends or converts your pay, often as a currency conversion spread. It is worth checking before you pick a payout method.
Ready to use these skills? The AI Training Job Matcher points you to open roles that fit your background.
AI Training Glossary FAQ
What Does RLHF Mean in AI Training Jobs?
RLHF means reinforcement learning from human feedback. In the job, you compare or rank AI responses, and those judgments train a reward model that steers the AI toward answers people prefer.
What Is the Difference Between SFT and RLHF?
SFT trains a model on example responses people write. RLHF trains it on people's judgments of its own responses. Writing ideal responses is SFT work; ranking responses is RLHF work.
What Is a Golden Dataset in AI Training?
A set of tasks with answers experts have agreed are correct. It is used to test models and to measure trainer accuracy, and some expert roles write golden answers themselves.
What Is Inter-Annotator Agreement?
How often different trainers give the same answer on the same task. Some disagreement is normal: in the InstructGPT project trainers agreed about 73% of the time.