Understand the technology
From a broad need to useful evidence.
For May and November 2027. This lesson is for SL and HL students. Build criteria step by step using practical measures, worked thresholds and clear links to company needs.
01 | “WE NEED BETTER IMAGES” IS ONLY A START
Visionary Studios is investigating image generation for advertising, concept art and digital media. A manager might say, “We need high-quality images quickly.” That expresses an ambition, but it leaves important questions unanswered.
What counts as high quality? Does quickly mean the first candidate or a finished, approved asset? An evaluation cannot answer these questions consistently until the company’s needs are made clearer.
An evaluation criterion is a stated basis for judging an option. This lesson shows how to move from a broad need to an observable test and a justified decision.
02 | DISTINGUISH A NEED, REQUIREMENT AND CRITERION
A need explains the problem or desired outcome: clients want recognisable characters across a campaign. A requirement states what the workflow must achieve: preserve specified character features across the requested scenes.
A criterion tells us how we will judge that achievement. For example, reviewers could check whether each selected scene retains the approved face, clothing and identifying accessory.
The terms sometimes overlap in practice. What matters is a clear connection between the business purpose, the expected behaviour and the evidence used to judge it.
03 | READ THE CASE BEFORE INVENTING A TEST
Start with needs actually supported by the booklet: useful image generation, output quality, consistency, resource considerations and responsible use. Connect your criteria to those needs.
When adding a precise target for a classroom example, label it as proposed. The booklet does not automatically establish a deadline or acceptance percentage just because your test needs one.
Keep three categories separate: information stated in the case, an inference you can justify and an assumption introduced for testing. This makes the evaluation easier to defend.
04 | ASK WHAT THE COMPANY WILL DO WITH THE OUTPUT
A concept image used in an internal discussion has a different purpose from a final product advertisement. An unusual detail might be acceptable in a rough idea but unacceptable in a published asset.
Identify the intended audience, production stage and decisions supported. Ask what would make an output unusable and who will approve it.
This context helps explain why a criterion matters. “Product details match the supplied reference” is meaningful when inaccurate details could mislead the client or audience.
05 | MAKE QUALITY OBSERVABLE
Quality is a broad label. Break it into relevant features such as visible artefacts, composition, requested detail and suitability for the brief.
A reviewer can then record whether an image has distorted objects or unwanted text, rather than simply declaring it “good”. Some judgements remain subjective, so describe the rubric and use informed reviewers.
Do not assume that larger pixel dimensions prove better visual quality. Resolution and suitability are different criteria.
06 | TURN STYLE ADHERENCE INTO A RUBRIC
Style adherence means matching the requested visual characteristics. For a fictional campaign, the brief might require muted colours, soft lighting and a hand-drawn appearance.
A rubric can define 0 as missing a characteristic, 1 as partly matching and 2 as clearly matching. Three characteristics would give a maximum of 6 points.
The scoring structure makes reasons more explicit but does not remove human judgement. Agree examples and discuss reviewer disagreements rather than treat a number as unquestionable truth.
07 | TEST CHARACTER CONSISTENCY ACROSS A SET
Consistency cannot be established from one image. Create a set of scenes requiring the same character, then inspect the identifying features across the selected outputs.
For a classroom pilot, the studio could propose six settings and require all selected images to retain an approved scarf, facial features and clothing design. These are illustrative conditions, not official booklet targets.
Allow changes the brief intends, such as pose or expression. Consistency means preserving relevant identity, not making every scene identical.
08 | DEFINE SPEED AT THE RIGHT ENDPOINT
“Generate in under a minute” measures one stage. If the business need is faster delivery, the more relevant endpoint may be an accepted, edited asset.
Record the start and finish conditions: does timing include setup, generation retries, review and editing? Use the same definition when comparing workflows.
A proposed criterion might be “reduce median production time per accepted asset against the baseline, while meeting the same quality checks”. Agree a target if the decision requires one.
09 | A WORKED CRITERIA TABLE
| Company need | Criterion and proposed condition | Evidence |
|---|---|---|
| Recognisable campaign character | All six selected scenes preserve agreed identity features | Feature checklist and reviewer notes |
| Requested visual style | Each selected image scores at least 5 of 6 on the agreed style rubric | Scores linked to descriptors |
| Faster production | At least 20% lower mean time per accepted asset than baseline | Complete production-time logs |
| Responsible approval | Required source, portrayal and disclosure checks completed; unresolved issues escalated | Review and approval records |
Numbers are useful only when the measure and target have a reason. An arbitrary percentage does not become a company requirement merely because it looks precise.
10 | WORK THROUGH A TIME TARGET
Suppose the existing workflow averages 30 minutes per accepted asset. A proposed 20% reduction means saving 30 × 0.20 = 6 minutes, giving a target of 24 minutes or less .
A pilot averaging 22.5 minutes meets that proposed time condition. However, it is not a complete success if the accepted assets fail the agreed quality or approval checks.
The threshold supports a decision about one measure. Use it alongside the other essential criteria and state the limits of the pilot.
11 | INCLUDE COMPUTING AND OPERATIONAL CONSTRAINTS
A workflow may produce good images but require hardware the studio does not have, an unaffordable subscription or a process that cannot meet expected demand.
A constraint limits the options available. Identify the relevant budget, hardware, staff skills, service availability and production volume before testing.
Measure resource requirements under representative conditions. A tool that works for one designer’s demonstration may behave differently when several colleagues need it at once.
12 | ASSESS RESPONSIBLE USE WITH EVIDENCE
“The tool must be ethical” is too vague to test directly. Break the concern into appropriate checks: source documentation, portrayal review, client communication and a route for unresolved issues.
Completing a checklist does not prove universal fairness or settle every rights question. It establishes whether the defined process has been followed and what remains unresolved.
Use honest wording: “required review completed with recorded decisions” is different from “all risks eliminated”.
13 | ESSENTIAL CONDITIONS AND PREFERENCES
Some criteria are essential gates, such as approval for publication under the agreed process. Others express preferences, such as an easier interface or a wider choice of editing controls.
Agree which are which before scoring options. Otherwise, a fast tool might receive a high total despite failing a condition the company considers necessary.
Weights can compare preferences among suitable options. They should not silently convert every essential condition into something that a speed score can compensate for.
14 | TRACE EVERY CRITERION TO A NEED
Traceability connects the original need, criterion, test and result. Give each criterion an identifier so the connection remains visible.
For example: N1 recognisable character → C1 preserve identifying features → T1 review six scenes → R1 five scenes meet the checklist. The result shows a shortfall against a six-scene condition.
If a criterion has no relevant need behind it, ask why it is included. If a major need has no test, the evaluation has a gap.
15 | CHOOSE REPRESENTATIVE TEST MATERIAL
Use briefs similar to the work the studio expects: different settings, compositions and appropriate character changes. Include demanding but relevant cases, not only prompts known to succeed.
Generate repeated candidates and preserve the selection record. This helps reveal whether success is dependable or required many rejected attempts.
The next lesson develops fair comparisons in more detail. Here the key point is that criteria need evidence from tasks that actually represent the company’s work.
16 | KNOW WHAT THE EVIDENCE CAN SUPPORT
A single selected image can illustrate a capability but cannot establish a reliable success rate. A small pilot may support further testing while leaving uncertainty about wider use.
Record model version, relevant settings, reviewer judgements and missing data. Report both the result and the limits of the measurement.
If the criterion is not met, distinguish a clear failure from insufficient evidence. “Not yet demonstrated” can be more accurate than either declaring success or claiming the tool can never work.
17 | REVISE CRITERIA OPENLY
A pilot may reveal an unclear descriptor or an unrealistic assumption. Criteria can be revised, but document the reason and version.
Do not quietly lower a target after seeing disappointing results and report the new test as if it were the original plan. Explain the change and, where needed, retest the options consistently.
18 | BUILD A COMPLETE EVALUATION CHAIN
What useful outcome does the company want?
What would count as success, and how will it be observed?
What happened, and what conclusion is justified?
In your book, propose three criteria for Visionary Studios. Include the linked need, a clear measure, a justified proposed condition and the evidence to collect.
Reveal a possible response
For character consistency, check agreed identifying features across six selected scenes and require all six to retain them. For production efficiency, compare complete time per accepted asset with a defined baseline and propose a justified improvement target. For responsible approval, require the agreed reviews and escalation decisions to be recorded before publication.
Label the thresholds as proposals, state who approves them and identify uncertainties. Use these independent notes alongside the official 2027 case-study booklet.
CHECK YOUR UNDERSTANDING | CLASSIFY THE EVALUATION ELEMENT