Evaluating Response Quality

Quality is multidimensional

A response can be correct but irrelevant, relevant but incomplete, clear but factually wrong, or detailed but instruction-breaking. Strong evaluation considers the dimensions the task actually asks you to judge.

Six foundational dimensions

• Instruction following — Did it do what was requested?
• Accuracy — Are factual claims correct?
• Relevance — Does it stay focused on the question?
• Clarity — Can the intended audience understand it?
• Completeness — Does it address all required parts?
• Support / calibration — Are uncertain or strong claims appropriately qualified?

Longer is not automatically better

Extra detail can reduce relevance, violate length constraints, or introduce more opportunities for factual errors.

Partial completion

If a user asks to compare two laptops on price and battery life, and the answer discusses only battery life, the answer may be accurate but incomplete.

Module summary

• Evaluate the dimensions required by the task.
• A single major instruction violation can outweigh polished writing.
• Accuracy alone is not enough.
• More words do not automatically mean higher quality.