Instruction Adherence
Instruction adherence measures whether a model actually did what it was told — followed the format, honored the constraints, stayed within the rules of the prompt. A model can give a high-quality answer that ignores half the instructions, and this is the metric that catches it.
Quality and obedience are different things. A model can produce a fluent, accurate answer that quietly ignores the instructions — returns prose when you asked for JSON, exceeds the length limit, skips a required field, or breaks a “don’t do X” rule. Instruction adherence scores that obedience directly: of the constraints the prompt set, how many did the output honor?
It matters most for production systems where output feeds other code — a malformed JSON or a missing field breaks the pipeline regardless of how good the content is. It’s usually checked with explicit validators where the rule is mechanical (schema, length, required fields) and with an LLM-as-a-judge for fuzzier instructions (“use a formal tone”). Rising instruction adherence is also a big part of why newer models feel more reliable to build on, even when raw knowledge is similar.