> Harness EngineeringHarness EngineeringHarness Engineering The engineering outside the model模型之外的工程模型之外的工程

5.6

Can a model check a model?模型能檢查模型嗎模型能检查模型吗

This chapter assumes you have read the independent check in Use subagents sparingly, and how its passes are counted in Make a pass count.

讀這一章,你需要讀過 慎用子 agent 裡的獨立檢查,以及 讓「通過」可信 裡它的通過怎麼計算。

读这一章之前,你需要读过慎用子 agent里的独立检查,以及让“通过”可信里它的通过是怎么计数的。

The Knowledge經驗分享经验分享

In one sentence

A model checking a model catches some errors, but the two share blind spots. Hiding the author's reasoning, or using another angle or model, helps; it does not remove them, and how much it helps has to be measured.

一句話

模型檢查模型,能抓到一些錯誤,但兩者有共同的盲點。藏起作者的推理,或換個角度、換個模型,都有幫助;但去不掉盲點,而且幫助有多大,必須去量。

一句话

模型检查模型,能抓到一部分错误,但两者有共同的盲点。藏起作者的推理,或者换个角度、换个模型,都有帮助,但去不掉盲点,而且帮助有多大,必须去测。

For leaders

  • The decision: decide how far to trust a model that reviews another model's work, and pay for a small measurement before relying on it for anything important.
  • The risk: the checker and the author share blind spots, so a pass means "a second reader found nothing", not "it is correct". A wrong FAIL also costs real work. I have not measured how often my checker fails good work or passes bad work. My one weak signal is that two checkers gave different verdicts on the same work.
  • The cost of doing it right: the cheap moves cost little: hide the author's reasoning, and ask a narrower question. A different model costs the most. The small measurement in the chapter takes about ten minutes: plant three errors, run the checker three times, and count.
  • Ask the team: "Have we ever planted known errors and counted how many our reviewing agent finds?"

給主管

  • 要做的決定:決定要多相信一個審查另一個模型的工作的模型,並且在把它用在重要的事情之前,花一點成本做一次小小的量測。
  • 風險:檢查者和作者有共同的盲點,所以一次通過的意思是「第二個讀者什麼都沒找到」,不是「它是對的」。錯誤的 FAIL 也會花掉實際的工作。我沒有量過我的檢查者多常讓好的工作失敗、多常讓壞的工作通過。我唯一的弱訊號是:兩個檢查者對同一份工作給出了不同的結論。
  • 做對的成本:便宜的做法花不了多少:藏起作者的推理,問一個更窄的問題。換一個不同的模型花得最多。這一章裡的小量測大約花十分鐘:放進三個錯誤,把檢查者跑三次,然後數。
  • 可以問團隊:「我們有沒有放進已知的錯誤,再數我們的審查 agent 找到幾個?」

写给管理者

  • 要拍板的事:决定对审查另一个模型的模型信到什么程度,并且在把它用到重要的事情上之前,花一点成本做一次小测量。
  • 风险:检查者和作者有共同的盲点,所以“通过”的意思是“第二个读者什么都没发现”,不是“它是对的”。错误的 FAIL 也会白费实际的工作。我没有测过我的检查者有多常把好活判为失败、多常放过坏活。我手上唯一的弱信号是:两个检查者对同一份工作给出了不同的结论。
  • 做对的代价:便宜的做法花不了多少:藏起作者的推理,问一个更窄的问题。换一个不同的模型花得最多。章里那个小测量大约要十分钟:放进三个错误,把检查者跑三次,然后数。
  • 可以问团队:“我们有没有放进已知的错误,再数我们的审查 agent 找出了几个?”

The problem

An independent check is a model reading another model's work. That should make you a little uneasy. If the author model has a blind spot, why would a second copy of the same model not have it too?

There is research behind the unease. Huang and colleagues tested whether a model can fix its own reasoning when nothing outside tells it what is wrong. Their finding: models "struggle to self-correct their responses without external feedback", and sometimes "their performance even degrades after self-correction" (Large Language Models Cannot Self-Correct Reasoning Yet, ICLR 2024). That paper is about a model revising its own answer, not about a second model checking. But it warns against the lazy reading of "ask the model to look again".

Two more papers are close to the checking case, though neither tests my checker. Panickssery and colleagues found that a model used as an evaluator can score its own outputs higher than others' ("self-preference"), and that this grows with how well it recognizes its own text (LLM Evaluators Recognize and Favor Their Own Generations). Zheng and colleagues, studying models as judges, list position, verbosity and self-enhancement biases (Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena, NeurIPS 2023). I use them as a reason to doubt a same-family checker, not as a measurement of one.

So the honest question is not "does a model check work?" It is "which errors does the checker see that the author did not, and which does it miss with the author?"

問題

獨立檢查是一個模型在讀另一個模型的工作。這應該讓你有點不安。如果作者模型有盲點,同一個模型的第二份副本,為什麼會沒有?

這份不安背後有研究。Huang 等人測試過:當外面沒有任何東西告訴模型哪裡錯了,模型能不能修正自己的推理。他們的發現是:模型「在沒有外部回饋的情況下,很難自己修正答案」,有時「自我修正之後,表現甚至變差」(Large Language Models Cannot Self-Correct Reasoning Yet,ICLR 2024)。那篇論文講的是模型修改自己的答案,不是第二個模型來檢查。但它提醒我們,不要對「叫模型再看一次」做偷懶的解讀。

還有兩篇論文和檢查的情況很接近,雖然沒有一篇測試過我的檢查者。Panickssery 等人發現,當作評估者的模型,給自己的輸出打的分,可以比給別人的高(「self-preference」),而且這和它辨認出自己文字的能力一起增長(LLM Evaluators Recognize and Favor Their Own Generations)。Zheng 等人研究把模型當評審,列出位置、冗長和自我偏好的偏誤(Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena,NeurIPS 2023)。我拿它們當作懷疑同一家族檢查者的理由,不是對檢查者的量測。

所以誠實的問題不是「模型檢查有沒有用?」而是:「檢查者看得到哪些作者沒看到的錯誤?又和作者一起漏掉哪些?」

问题

独立检查,是一个模型在读另一个模型的工作。这应该让你有点不踏实。如果作者模型有盲点,同一个模型的第二份副本,为什么就没有?

这份不踏实背后有研究。Huang 等人测过:外面没有任何东西告诉模型哪里错了,它能不能自己改对推理。他们的发现是:模型“在没有外部反馈的情况下,很难自我纠正自己的回答”,有时“自我纠正之后,表现甚至变差”(Large Language Models Cannot Self-Correct Reasoning Yet,ICLR 2024)。这篇论文讲的是模型修改自己的答案,不是第二个模型来检查。但它提醒我们,别把“让模型再看一遍”想得太省事。

还有两篇论文和检查的情形很接近,不过都没有测过我的检查者。Panickssery 等人发现,用作评估者的模型,给自己输出打的分可能比给别人的高(“self-preference”),而且这和它认出自己文字的能力一起增长(LLM Evaluators Recognize and Favor Their Own Generations)。Zheng 等人研究把模型当评审,列出了位置偏差、冗长偏差和自我增强偏差(Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena,NeurIPS 2023)。我拿它们当作怀疑同一家族检查者的理由,不是对某个检查者的测量。

所以实在的问题不是“模型检查有没有用”,而是:检查者能看出哪些作者没看出的错误,又和作者一起漏掉哪些?

How it works

Picture the errors in a piece of work as a set. The author misses some of them. The checker misses some of them. What you want to know is how much the two misses overlap.

errors the author misses errors the checker misses checker catches author catches missed by both A different angle or model shrinks the shared part. It does not remove it.
A checker catches what lies outside the author's blind spots. The overlap is what both miss.

Every method below tries to make the circle of the checker different from the circle of the author. None can make them fully apart.

1. Hide the author's reasoning

When you proofread your own essay, you read what you meant. An author's context is full of its own reasoning, and that reasoning explains away its own mistakes. A checker that has seen the reasoning is pulled along the same path.

So the first move is to give the checker only the request, the work and the diff (the list of changed lines), and keep the author's notes out on purpose (Use subagents sparingly explains why). This is cheap, and it should reduce the part of the overlap that comes from shared reasoning; I have not tested how much. It does not touch the part that comes from the shared model.

2. A different angle

The same model asked a different question looks at different things. "Is this right?" invites a general skim. A checklist tied to the requirement is narrower: for each item of the requirement, find the evidence in the work; for each conclusion in the work, find its source. The checker has to go and look, not to feel.

Two more rules push the angle: every finding names a file and line, and the checker points out but never writes the fix. Both are in Use subagents sparingly.

3. A different model

A checker from another model family, or at least another size, has different habits and different holes. This should shrink the shared circle the most, and it is also the one with the most cost: another model to set up, to pay for and to keep in step. It is a good candidate for a small trial before you commit to it. In my setup the checker runs on a model chosen separately from the main agent's, but I have not tested whether choosing a different family changes how many errors it finds.

4. A different kind of check

Some errors a model reading text will not find, by any means. A test that runs the code finds them. The same goes for the checker itself: if you give it tools to run the code and look at files, it stops guessing from the text and starts verifying. A person who knows the business finds others. A script finds anything that has one right answer. Put these next to the model checker, not behind it. Let the agent check its own work covers tests; Who decides covers which kind of judge fits which question.

Where I would start

My order, which I have not tested: first the two cheap moves (hide the reasoning, ask a narrower question), then a different kind of check that runs things, and only then a different model, because it costs the most.

How much does it catch?

Here I do not have an answer. I have not measured how often my checker fails good work or passes bad work. What I do have is one weak signal: two checkers gave different verdicts on the same work. That tells me a single verdict is not a fact.

The way to measure it is simple to state. Take work whose right answer you know, with some errors put in on purpose, and some "wrong-looking but fine" cases. Run the checker several times. Count the good work it failed, the bad work it passed, and how often its findings point at the right line. The measuring itself is the subject of the chapters on evidence; here the point is only that you should do it before you trust the checker with something important. A small version takes about ten minutes: put three errors into a piece of work you know, run the checker three times, and count how many it finds each time.

原理

把一份工作裡的錯誤想成一個集合。作者漏掉其中一些。檢查者也漏掉其中一些。你想知道的是,兩邊漏掉的有多少重疊。

作者看不到的錯 檢查者看不到的錯 檢查者能抓到 作者能抓到 兩邊都 看不到 換角度或換模型能縮小重疊的部分,但不能讓它消失。
檢查者抓得到作者盲點以外的錯。重疊的部分是兩邊都漏掉的。

下面每一種方法,都是想讓檢查者的圓,和作者的圓不一樣。沒有一種能讓兩個圓完全分開。

1. 藏起作者的推理

你校對自己的文章時,讀到的是你想寫的東西。作者的上下文裡全是它自己的推理,這些推理會把它自己的錯誤解釋掉。看過推理的檢查者,會被拉著走上同一條路。

所以第一步,是只給檢查者要求、工作和 diff(改動的行的清單),刻意把作者的筆記擋在外面(慎用子 agent 解釋了原因)。這很便宜,而且它應該能減少來自共同推理的那部分重疊;我沒有測過減少多少。它碰不到來自共同模型的那部分。

2. 換個角度

同一個模型被問不同的問題,會看不同的東西。「這樣對嗎?」會引來泛泛的掃一眼。綁在需求上的檢查清單比較窄:對需求的每一項,在工作裡找證據;對工作裡的每個結論,找它的來源。檢查者得去找,不是去感覺。

還有兩條規則會推動角度:每個發現都要指出檔案和行號,檢查者只指出問題,絕不寫修正。兩條都在 慎用子 agent 裡。

3. 換個模型

來自另一個模型家族、或至少是另一種大小的檢查者,有不同的習慣和不同的洞。這應該最能縮小共同的圓,但它的成本也最高:多一個模型要設定、要付費、要保持同步。在你投入之前,它很適合先做一個小試驗。在我的設定裡,檢查者跑在一個和主 agent 分開挑選的模型上,但我沒有測過換成不同家族,找到的錯誤數量會不會改變。

4. 換一種檢查

有些錯誤,讀文字的模型怎麼樣都找不到。執行程式碼的測試找得到。檢查者本身也一樣:如果你給它能執行程式碼、能看檔案的工具,它就不再憑文字猜測,開始驗證。懂業務的人會找到另一些。腳本能找到任何只有一個正確答案的東西。把這些放在模型檢查者旁邊,不要放在它後面。讓 agent 能檢查自己的活 講測試;誰來判斷:程式、便宜模型、主模型、人 講哪一種評審適合哪一種問題。

我會從哪裡開始

我的順序,我沒有測過:先做兩個便宜的做法(藏起推理,問更窄的問題),然後是會執行東西的另一種檢查,最後才是換個模型,因為它花得最多。

它能抓到多少?

這裡我沒有答案。我沒有量過我的檢查者多常讓好的工作失敗,或讓壞的工作通過。我有的只是一個弱訊號:兩個檢查者對同一份工作給出了不同的結論。這告訴我,單一的結論不是事實。

量測的辦法,說起來很簡單。拿一份你知道正確答案的工作,故意放進一些錯誤,再放一些「看起來不對、其實沒問題」的情況。把檢查者跑幾次。數它讓幾份好的工作失敗、讓幾份壞的工作通過,以及它的發現多常指到對的那一行。量測本身是關於證據的那幾章的主題(固定的任務,用腳本打分);這裡的重點只是:在你把重要的事交給檢查者之前,應該先做這件事。小版本大約十分鐘:在一份你熟悉的工作裡放進三個錯誤,把檢查者跑三次,數它每次找到幾個。

原理

把一份工作里的错误想成一个集合。作者漏掉其中一些,检查者也漏掉其中一些。你想知道的是,两边漏掉的有多少重叠。

作者看不到的错 检查者看不到的错 检查者能抓到 作者能抓到 两边都 看不到 换个角度或换个模型能缩小重叠的部分,但不能让它消失。
检查者能抓到作者盲点以外的错。重叠的部分是两边都漏掉的。

下面每一种方法,都是想让检查者的圆和作者的圆不一样。没有一种能让两个圆完全分开。

1. 藏起作者的推理

自己校对自己的文章时,你读到的是你想写的。作者的上下文里满是它自己的推理,这些推理会把它自己的错误解释过去。看过这些推理的检查者,会被带着走上同一条路。

所以第一招,是只给检查者请求、成果和 diff(改动行的列表),有意把作者的笔记挡在外面(原因见慎用子 agent)。这一招很便宜,应该能减少“共同推理”造成的那部分重叠,但减少多少,我没有测过。它碰不到“共用同一个模型”造成的那部分。

2. 换个角度

同一个模型,换一个问题,看的地方就不同。“这对不对?”会引来泛泛的扫读。贴着需求的检查清单要窄得多:需求里的每一条,都到成果里找证据;成果里的每一个结论,都找出它的来源。检查者得去查,而不是凭感觉。

还有两条规则能推动这个角度:每条发现都要写明文件和行号;检查者只指出问题,不写修复。这两条都在慎用子 agent里。

3. 换个模型

来自另一个模型家族、至少是另一种规格的检查者,习惯不同,漏洞也不同。这应该最能缩小共同的那个圆,但成本也最高:要多配一个模型,多付一份钱,还要让它跟上其余部分的变化。在决定采用之前,它很适合先做个小试验。在我的配置里,检查者用的模型和主 agent 是分开选的,但换成另一个家族会不会改变它找到的错误数量,我没有测过。

4. 换一种检查

有些错误,模型读文字怎么读都读不出来。跑代码的测试能找出来。检查者自己也一样:给它工具,让它能运行代码、查看文件,它就不再凭文字猜测,而是开始验证。懂业务的人能找出另一些错误。脚本能找出任何只有一个正确答案的问题。把这些放在模型检查者旁边,而不是后面。让 agent 能检查自己的活讲测试;谁来判断:代码、便宜模型、主模型、人讲哪种判断者适合哪种问题。

我会从哪里开始

我的顺序,没有测过:先用两招便宜的(藏起推理,问更窄的问题),再换一种会运行东西的检查,最后才换模型,因为它最贵。

它能抓住多少?

这里我没有答案。我没有测过我的检查者有多常把好活判为失败、多常放过坏活。我手上只有一个弱信号:两个检查者对同一份工作给出了不同的结论。这说明单次的结论不是事实。

测量的办法说起来很简单。拿一份你知道正确答案的工作,故意放进一些错误,再放进一些“看着不对其实没问题”的情形。把检查者跑几次。数一数:被它判为失败的好活有几个,被它放过的坏活有几个,它的发现有多常指向正确的那一行。测量本身是证据那几章的主题;这里要说的只是,在你把重要的事交给检查者之前,应该先做一遍。小版本大约要十分钟:往一份你熟悉的工作里放进三个错误,把检查者跑三次,数每次找出几个。

Costs and traps

A pass lowers the risk. It does not remove it. Treat it as "a second reader found nothing", not "it is correct".

A wrong FAIL costs real money. Each false FAIL sends the author back to fix a problem that is not there, and may make the work worse. The rule that findings must point to a real file and line cuts this, but only for the form of a finding, not for its truth.

Agreement is not independence. If both models are the same one and have seen the same reasoning, two passes may be one opinion twice. That is why the checker is a new one every time, with a clean context.

A model checker does not replace a person. It narrows what a person has to read. Approval of the final result stays with a person (Two approvals).

代價和陷阱

一次通過會降低風險,不會消除它。把它當成「第二個讀者什麼都沒找到」,不是「它是對的」。

錯誤的 FAIL 要花真的錢。每個假的 FAIL,都會讓作者回去修一個不存在的問題,還可能讓工作變得更糟。要求發現必須指向真實的檔案和行號,這條規則能減少這種情況,但只管發現的形式,不管它是不是真的。

意見一致不等於獨立。如果兩個模型是同一個,看過同樣的推理,兩次通過可能只是同一個意見說了兩次。這就是為什麼檢查者每次都是新的,上下文是乾淨的。

模型檢查者不能代替人。它縮小的是人必須讀的範圍。最後結果的批准留給人(兩道門:人在哪裡批准)。

代价和坑

通过只是降低风险,不是消除风险。把它当作“第二个读者什么都没发现”,而不是“它是对的”。

错误的 FAIL 要花真金白银。每一个误报的 FAIL 都会让作者回去修一个并不存在的问题,还可能把活改得更糟。“发现必须指向真实的文件和行号”这条规则能减少这类情况,但只管发现的格式,管不了它是否属实。

意见一致不等于独立。如果两个模型是同一个,又看过同样的推理,两次通过可能只是同一个意见说了两遍。所以每次的检查者都是新的,上下文是干净的。

模型检查者替代不了人。它缩小的是人要读的范围。最终结果由人审批(见两道门:人在哪里审批)。

When to delete it

If you measure your checker and find that it catches nothing the author's own run of the tests did not, drop it and keep the tests. If the platform later offers a way to see how well a checker does on your own tasks, use that instead of a feeling.

什麼時候可以刪掉

如果你量了你的檢查者,發現它抓到的,作者自己執行測試時都已經抓到了,那就拿掉它,留下測試。如果平台之後提供一種辦法,能看到檢查者在你自己的任務上表現得多好,就用那個,不要靠感覺。

什么时候可以删掉

如果你测过自己的检查者,发现它抓不到任何作者自己跑测试抓不到的东西,就把它去掉,留下测试。如果平台以后提供办法,能在你自己的任务上看出检查者做得怎么样,就用那个,别凭感觉。

Go deeper

延伸閱讀

延伸阅读

You can stop here: that is the whole idea. The next part shows where it runs, so you can look inside.讀到這裡就夠了,這就是整個想法。下一部分告訴你它在哪裡運行,可以進去看看。读到这里就够了,核心思路就是这些。下一部分告诉你它在哪里跑,可以进去看看。

In my practice (Copilot Harness)實作实战

Copilot Harness is my open-source harness for VS Code Copilot.

Repository: https://github.com/Smars-Bin-Hu/copilot-agentic-harness

Get started: clone it, open it in VS Code, and type /harness-rookie-get-me-ready in Copilot Chat. You need Python 3.9 or later and git.

Copilot Harness 是我開源的 VS Code Copilot harness。

倉庫: https://github.com/Smars-Bin-Hu/copilot-agentic-harness

開始:clone 下來,用 VS Code 打開,在 Copilot Chat 裡輸入 /harness-rookie-get-me-ready。需要 Python 3.9 以上和 git。

Copilot Harness 是我开源的 VS Code Copilot harness。

仓库: https://github.com/Smars-Bin-Hu/copilot-agentic-harness

上手:把仓库 clone 下来,用 VS Code 打开,在 Copilot Chat 里输入 /harness-rookie-get-me-ready。需要 Python 3.9 及以上版本和 git。

Explore it in the harness

The checker that cannot see the author's reasoning, the format it must follow, and the script that drops findings without a real line all run in Copilot Harness. The harness can teach them to you from the inside, with the real agent file. After the setup above, type:

/harness-rookie-wondering I have read the chapter "Can a model check a model?". Teach me how this harness builds its independent checker (topic 11): what it is and is not allowed to see, how its answer is shaped, and what the honest limits are. Open the checker's agent file with me and draw how the pieces fit together.

It teaches one small piece at a time, stops to ask if you are following, and opens the files with you.

在 harness 裡探索

看不到作者推理的檢查者、它必須遵守的格式,以及丟掉沒有真實行號的發現的腳本,都在 Copilot Harness 裡執行。harness 可以從裡面教你這些,附上真實的 agent 檔案。照上面的說明準備好之後,輸入:

/harness-rookie-wondering 我讀完了「模型能檢查模型嗎」這一章。請教我這個 harness 怎麼建立它的獨立檢查者(主題 11):它能看什麼、不能看什麼,它的答案是什麼形狀,以及誠實的限制是什麼。帶我打開檢查者的 agent 檔案,並畫出這些部分怎麼組合在一起。

它一次教一小塊,停下來確認你跟得上,然後和你一起打開檔案。

在 harness 里探索

看不到作者推理的检查者、它必须遵守的格式,以及丢掉没有真实行号的发现的脚本,都在 Copilot Harness 里跑着。harness 可以从内部教你,配上真实的 agent 文件。按上面的说明准备好之后,输入:

/harness-rookie-wondering 我读完了“模型能检查模型吗”这一章。请教我这个 harness 是怎么搭它的独立检查者的(主题 11):它能看什么、不能看什么,它的回答是什么格式,以及诚实的局限有哪些。和我一起打开检查者的 agent 文件,并画出各部分是怎么拼在一起的。

它一次只教一小块,停下来确认你跟上了,再和你一起打开文件。