The bug that scored 0/10, and why we published it
Our fine-tuned model answered every question it should have refused. The cause was not the model, the data, or the fine-tune — it was a mismatch nobody looks for.
Campus GPT is meant to know what it does not know. Ask it something its documents do not cover and it should decline, rather than assembling a confident answer out of nothing. That behaviour is the entire point of the system — an assistant that invents institutional policy is worse than no assistant at all.
The first time we measured it, refusal accuracy came back at 0 out of 10. It answered everything. Every out-of-scope question got a fluent, plausible, completely unfounded response.
What it was not
The obvious suspects were all wrong, and ruling them out took longer than finding the actual cause.
- Not the fine-tune. The training run had converged. Loss curves were
clean, and the model reproduced the refusal behaviour correctly when probed against the training set.
- Not the training data. The refusal examples were there, correctly
labelled, in the proportion we intended.
- Not retrieval. The retriever was returning nothing for out-of-scope
questions, which is exactly right. The model was answering anyway.
That last point is what made it confusing. The pipeline was behaving. The model had been taught. And the output was still wrong.
What it was
The prompt the model saw in production was not shaped like the prompt it had been trained on.
Not the wording — the structure. Whitespace, the ordering of the system block against the retrieved context, the exact delimiters between sections. Small enough to be invisible in a diff read by a human, large enough that the fine-tuned behaviour never triggered. The model had learned to refuse when it saw a very specific arrangement of tokens, and production was handing it a slightly different one.
So it fell back on what the base model does with an unfamiliar prompt shape: it answered helpfully.
The fix, and the number
We aligned the serving prompt to the training prompt exactly — same delimiters, same ordering, same whitespace — and re-measured against the running system rather than a benchmark.
90% refusal accuracy on out-of-scope questions. 100% on grounded factual ones.
Ninety, not a hundred. The remaining ten percent are questions sitting close enough to the document boundary that a reasonable person would also hesitate. We would rather publish ninety and say what the gap is than round it up.
Why this is on our website
Because the 0/10 is the part that makes the 90% worth believing.
Anyone can put a number on a page. Very few will tell you what it was before they fixed it, or that the first version of their system was confidently wrong in exactly the way the marketing says it never is. A studio that only shows you the working version is showing you a filtered view, and you cannot tell how heavily filtered.
There is also something practically useful in it. If you are fine-tuning a model and the trained behaviour is not showing up in production, check that your serving prompt matches your training prompt byte for byte before you question the training. We lost time doing it in the other order.
What we took from it
Measure against the deployed system, not the notebook. The notebook was fine. The notebook was never the thing customers would talk to.