AI Training Guide · Step 9 of 11
How to Write Prompts and Ideal Responses for AI Training
Some AI training tasks ask you to write instead of rate: a prompt that tests the model, or an ideal response that shows the model what a perfect answer looks like. Mercor runs fellowships for prompt and rubric writing.
Sources: Outlier FAQ, Mercor contracts.
Three Kinds of Prompts
When OpenAI hired people to write prompts for InstructGPT, it asked for three kinds, described in its 2022 paper:
| Type | What it is |
|---|---|
| Plain | Any task, as varied as possible |
| Few-shot | An instruction plus a few example input and answer pairs |
| User-based | A realistic version of something people actually use the model for |
Knowing which kind you are writing keeps the prompt focused.
Writing Prompts That Test a Model
A good prompt is something a real person would ask, specific enough to have a clear right answer, and hard enough that a model could get it wrong. Trainers on Reddit say reviewers flag contrived prompts, and every rubric criterion should be checkable on its own.
- ✓Sounds like a real user with a real situation
- ✓Has a clear, checkable right answer
- ✓Includes details that a careless model would miss
- ✓Follows the project's rules on topic, length and format
- ✓Is not a trick question with no real user behind it
Sources: GOV.UK: Self Assessment, GOV.UK: taxed twice on foreign income, r/DataAnnotationTech thread.
Prompts That Probe Safety
Some projects ask you to write prompts that try to make the model misbehave, called red teaming. Anthropic's red teaming paper gives clear guidance from its own program:
- Be creative. Swapping one word into a template produces low-value prompts.
- Stick to one topic per attempt.
- Plan the attempt before you write it.
- Choose topics within your own comfort level. The paper warned red teamers that some content is upsetting and let them pick topics within their own tolerance.
The same paper found that subtle requests for unethical help succeeded relatively more often than obvious ones, and that some topics need a domain expert to judge. That is one reason domain experts are useful here.
Mercor describes the same opt-in approach. Projects likely to include distressing material, red teaming prompts among them, are flagged in advance. Joining is voluntary, you can step back at any time without it affecting your standing, and live wellness support opens after 40 hours of project work.
Sources: Mercor wellness resources.
Writing Ideal Responses
An ideal response, sometimes called a golden response, is the answer you would want every model to give. Accuracy comes first: the InstructGPT instructions put truthfulness ahead of helpfulness in final evaluations.
One habit makes ideal responses stronger: list the criteria a perfect answer must meet, make each one checkable on its own, mark which are essential, and include the common errors it must avoid.
Sources: Scale AI, rubrics as rewards.
What Working Trainers Say
Trainers on Reddit who write prompts meant to make a model fail give the same two pieces of advice.
- Keep prompts realistic. Reviewers flag prompts that rely on gimmicks such as deliberate typos, contradictions or a stack of unrelated rules. No real user writes like that.
- Find failures in the real complexity of your own field: multi-step work, layered constraints, competing priorities with no single right answer, or obscure facts that are still true.
This section draws on Reddit threads such as: Reddit thread, Reddit thread.
Mistakes to Avoid
- Writing prompts only you would ask, with no real user behind them.
- Prompts with no clear right answer, which cannot be graded.
- Ideal responses that open with filler or praise for the question.
- Unverified facts in an ideal response.
- Ignoring the project's length and format rules.
Find writing-heavy AI work
Chat with DigiNo Toucan. Two questions, no CV needed.
Strong writer or expert? The AI Training Job Matcher points you to platforms with writing and expert tasks.
Prompts and Ideal Responses FAQ
What is a golden response in AI training?
An ideal answer written by a person to show the model what a perfect response looks like. It must be correct, complete and match the project's format.
How do you write a good prompt for AI training?
Write something a real user would ask, specific enough to have a checkable right answer and detailed enough that a careless model could get it wrong.
Are prompt writing tasks harder than rating?
They usually take longer and need strong writing. When OpenAI built InstructGPT, it screened the people writing ideal responses separately, on the quality of their writing for sensitive prompts.
Should an ideal response be long?
Only as long as the user needs. Answer first, then cover what matters, in the project's format.
Can I use AI to write prompts or ideal responses?
No. Mercor's AI-use policy allows AI only for grammar and light polishing of your own ideas, and other platforms have similar rules. News reports describe contractors removed for using AI on the work.
