8.2
The toolkit: scripts that draw a map of the codeToolkit:用腳本畫出程式碼的地圖Toolkit:用脚本画出代码的地图
This chapter follows A knowledge base. It is step one and two of the road.
這一章接在 知識庫:程式碼的誠實索引 後面。它是路線的第一步和第二步。
这一章接在知识库:代码的诚实索引后面,是路线的第一步和第二步。
The Knowledge經驗分享经验分享
In one sentence
Which objects exist, and who uses whom, has one right answer. So a script computes it, the same way every time, at no token cost. The scripts are called a toolkit, and it is code the team owns, written for your own technology stack.
一句話
有哪些物件、誰用誰,只有一個正確答案。所以用腳本來算,每次都一樣,也不花 token。這些腳本叫 toolkit,是團隊自己擁有的程式碼,為你自己的技術堆疊而寫。
一句话
有哪些对象、谁用谁,只有一个正确答案。所以由脚本来算,每次结果一样,也不花 token。这些脚本叫 toolkit,是团队自己拥有的代码,针对你自己的技术栈来写。
For leaders
- The decision: have the team own a small set of scripts that list which objects exist and who uses whom, instead of asking a model to work it out each time.
- The risk: a model asked for the map reads by judgment, can skip an object, and gives two different maps if asked twice. A script gives the same map every time and says what it could not read. On a small open-source project my sample found 58 objects in 35 files and all 6 known uses; that shows it runs, not that it works on your code.
- The cost of doing it right: the model writes the scripts and a person approves the design; the scripts then cost no tokens. They are code to maintain when the stack changes. I have not tested a very large code base.
- Ask the team: "Who owns the scripts that map our code, and which uses do we already know they must find?"
給主管
- 要做的決定:讓團隊擁有一小組腳本,列出有哪些物件、誰用誰,不要每次都請模型去推算。
- 風險:被要求畫地圖的模型,是憑判斷在讀,可能漏掉一個物件,問兩次會給出兩張不同的地圖。腳本每次給同一張地圖,並說出它讀不到的部分。在一個小的開源專案上,我的範例在 35 個檔案裡找到 58 個物件,6 個已知的使用全都找到了;這只證明它跑得起來,不證明它在你的程式碼上行得通。
- 做對的成本:模型寫腳本,人批准設計;之後腳本不花 token。堆疊改變時,它們是要維護的程式碼。我沒有測過很大的程式碼庫。
- 可以問團隊:「畫我們程式碼地圖的腳本誰負責?我們已經知道它們必須找到的使用有哪些?」
写给管理者
- 要拍板的事:让团队拥有一小组脚本,列出有哪些对象、谁用谁,而不是每次都让模型去推算。
- 风险:让模型出地图,它凭判断去读,可能漏掉对象,问两次会给出两张不同的地图。脚本每次给同一张地图,还会说明它哪里没读懂。在一个小的开源项目上,我的样例找到了 35 个文件里的 58 个对象,以及已知的 6 处使用全部找到;这只说明它能跑,不说明它在你的代码上管用。
- 做对的代价:脚本由模型来写,设计由人审批;之后脚本不花 token。技术栈变了,它们是要维护的代码。我没有在非常大的代码库上测过。
- 可以问团队:“谁负责给我们的代码画地图的那些脚本?我们已经知道哪些使用是它必须找到的?”
The problem
Before an agent can write a card about an object, someone has to answer two plain questions. Which objects are there? And for each one, what does it use?
Two words first.
- An object is a named thing in the code that other things can use (see A knowledge base: an honest index of the code).
- A use is one object depending on another: a procedure calls another procedure, a class reads a table, a job runs a package.
Together, all the objects and all the uses are the map of the code (named in the previous chapter). It is the same thing as a dependency graph, if you know that term.
You could ask a model to produce the map by reading the code. It would do something reasonable. But it decides what to read next from what it just read. It can skip an object, follow a name that means two things, or stop early. It cannot say how much it missed. And ask it twice, and the two maps differ.
A script that parses every file does not have these problems. It gives the same map every time, for no tokens. And it can say what it could not read. That is the whole case for this chapter.
問題
在 agent 能為一個物件寫卡片之前,得有人回答兩個簡單的問題。有哪些物件?每一個物件,它用了什麼?
先說兩個詞。
- 物件是程式碼裡一個有名字、別的東西可以使用的東西(見 知識庫:程式碼的誠實索引)。
- 使用是一個物件依賴另一個:一個預存程序呼叫另一個預存程序,一個類別讀一張資料表,一個工作執行一個套件。
所有物件加上所有使用,合起來是程式碼的地圖(上一章已經提過這個詞)。如果你知道這個詞,它就是依賴圖。
你可以請模型讀程式碼來畫地圖。它會做出合理的東西。但它是靠剛讀到的,決定下一步讀什麼。它可能漏掉一個物件,追到一個有兩種意思的名字,或太早停下來。它說不出自己漏了多少。而且問它兩次,兩張地圖不一樣。
解析每個檔案的腳本沒有這些問題。它每次給同一張地圖,不花 token。它還能說出自己讀不到的部分。這一章的全部理由就是這個。
问题
agent 要给一个对象写卡片,得先有人回答两个朴素的问题。有哪些对象?每个对象用到了什么?
先交代两个词。
- 对象是代码里有名字、别的东西可以使用的东西(见知识库:代码的诚实索引)。
- 使用是一个对象依赖另一个对象:一个存储过程调用另一个存储过程,一个类读一张表,一个任务运行一个包。
所有的对象和所有的使用合在一起,就是代码的地图(上一章起的名字)。如果你知道依赖图这个说法,它们是一回事。
你可以让模型读代码来出这张地图。它会做出一份像样的东西。但它是根据刚读到的内容,决定下一步读什么。它可能漏掉一个对象,追到一个有两种意思的名字,或者太早停下来。它说不出自己漏了多少。而且问它两次,两张地图会不一样。
一个解析每个文件的脚本没有这些问题。它每次给出同样的地图,不花 token。它还能说出哪些东西它没读懂。这一章的理由就是这些。
How it works
What the toolkit is
How a procedure, a job or a class looks inside a file depends on the stack. A call in one stack is CALL name, in another it is a method on an injected object. So there is no ready-made parser that covers every convention of your code. Where a language server or a code-search service already gives you objects and uses, use it first (see When to delete it). You write extractors for what they do not cover.
The toolkit is a small set of programs plus one config file, all inside the knowledge base:
- An extractor reads one kind of file and prints the objects it defines and the uses it finds.
- The config file says what an object is in your code, what kinds of use count, which types of object are entry points (the places the outside world calls in: a screen, an API, a scheduled job), which folders are retired, and which words to ignore in names.
A runner applies the extractors to each repository's main branch and saves the map in a cache. On the next run, only the files that changed since the last run are read again.
- The code is the truth. Objects and uses are written in it, in the forms your stack uses.
- Extractors are small scripts. Each reads one kind of file and prints the objects and uses it finds. The same input gives the same output.
- The output is the map: every object that exists, and who uses whom.
- A few uses that a person knew are written down as a test. Every run must find all of them.
- On the next run, only the files that changed are read again.
Who writes it: the model designs, a person approves, scripts run
You might expect to write the extractors yourself. In my setup the model writes them, in four moves:
- Survey. A script counts the kinds of file in each repository. The model reads a few samples of each kind.
- Design. The model writes down what objects and uses look like in this stack, with examples from the code. A person reads it and approves it. This is the moment to catch "that is not a use, that is a comment".
- Build. The model writes the extractors.
- Run until the known uses are found. See the next section.
The model is good at writing a parser once. It should not be the parser every time. After the build, only scripts touch the code.
A test that stays: the uses a person knew
When the map is first made, how do you know it is not nonsense? A person writes down 5 to 10 uses they know are there, for example "procedure A calls procedure B". These go into a file in the toolkit. It is better to write them before the extractor exists, and not to show them to the model while it writes, so the extractor is not tuned to just these. Every run of the map must find all of them. If a later change to an extractor loses one, the run fails.
It is a regression test: a test that guards what already worked. It is small, and I do not pretend it measures the whole map. Whenever you meet a use the map missed, add it. Chapter 8.3 measures how complete the map is.
The runner also reports, on every run:
- the files it could not read, and the files it read and found nothing in (both are suspicious);
- the names that are used but defined nowhere: they may be other systems, deleted code, or a form the extractor misreads. Look at a few.
Two samples, only as examples
The harness ships no toolkit of its own, only two samples that show the shape.
- SQL Server: stored procedures, scheduled agent jobs, and integration packages.
- Spring: a Spring Boot service, with controllers, repositories, entities and tables.
I built the Spring sample on the open-source spring-petclinic project. On that code it found 58 objects in 35 files, and all 6 of the known uses; no file failed to read and none came back empty. That is a smoke test: it shows the toolkit runs on one small, tidy project. It does not show it works on yours. The hard cases (SQL built at run time, calls driven by configuration) are not in that project. Your stack, your conventions and your odd cases are what you write.
The real files of the Spring sample show the shape. First the known uses a person wrote down for it:
Five of the sample's known uses: one object, the kind of use, the other object.
{"from": "controller:PetController", "kind": "uses", "to": "repository:PetTypeRepository"}
{"from": "controller:VetController", "kind": "uses", "to": "repository:VetRepository"}
{"from": "repository:OwnerRepository", "kind": "reads", "to": "entity:Owner"}
{"from": "entity:Owner", "kind": "maps", "to": "table:owners"}
{"from": "entity:Pet", "kind": "uses", "to": "entity:Visit"}.harness/examples/kb-toolkits/spring/facts.jsonl L4–L8 @ 1.5.50
Then the function that reads one Java file and prints its objects and uses:
The extractor for a Java file: find the class, its role, and the types it uses.
def java(text: str, context: Dict[str, Any]) -> List[Dict[str, Any]]:
clean = blank_comments(text)
match = TYPE.search(clean)
if not match:
return []
name, head = match.group(2), clean[:match.start()]
role = next((role for mark, role in ROLE if mark in head), "class")
repository = REPOSITORY.search(match.group(3))
if repository:
role = "repository"
refs: List[Dict[str, Any]] = []
if repository:
refs.append({"kind": "reads", "name": repository.group(1), "as": "entity", "line": line_at(clean, match.start(3))})
parent = EXTENDS.search(match.group(3))
if parent and not repository:
refs.append({"kind": "extends", "name": parent.group(1), "line": line_at(clean, match.start(3))})
table = TABLE.search(head)
if table:
refs.append({"kind": "maps", "name": table.group(1), "as": "table", "line": line_at(clean, table.start(1))})
body = clean[match.end():]
offset = line_at(clean, match.end()) - 1
for used in used_types(body):
if used != name:
refs.append({"kind": "uses", "name": used, "line": offset + 1})
found = [{"id": f"{role}:{name}", "type": role, "name": name, "line": line_at(clean, match.start(2)), "dynamic": False, "refs": refs}]
if role == "controller":
found += endpoints(clean, head, body, offset, name)
return found.harness/examples/kb-toolkits/spring/extract_spring.py L50–L77 @ 1.5.50
And the sample's own record of how it was built, including the first run that missed one use and what it cannot see:
Two rounds of fixing the extractor, and the blind spots left.
## Rounds
1. 9 of 10 facts: `new ModelAndView("owners/ownerDetails")` not known. Also `welcome.html` at the top of `templates/` was not matched by `templates/**/*.html`: fixed in the harness (`**/` stands for no folder too).
2. 10 of 10. An added fact on a view returned through a constant failed; constants are now read. 11 of 11.
## Blind spots
- Join tables named only in `@JoinTable` (`vet_specialties`) have no use.
- Beans wired by the framework (`@Configuration`, formatters) show as `unused`: the framework calls them..harness/examples/kb-toolkits/spring/DESIGN.md L22–L30 @ 1.5.50
Why a general harness does not include a toolkit
I did not ship parsers for the common stacks. A parser that is almost right fails silently, and each team writes code a little differently. A toolkit the team owns and audits (next chapter) can be fixed when it misses, and it lives in the knowledge base, so upgrading the harness never overwrites it.
Test your own setup
- Pick one repository. Write down 5 to 10 uses you know are there, each with a file and line.
- Ask an agent to draft what objects and uses look like in your stack, with examples from your code. Read it. Correct it.
- Have the agent write the extractors. Run them, and check that all your known uses appear.
- Run them twice. The two outputs must be identical.
原理
toolkit 是什麼
一個預存程序、一個工作或一個類別在檔案裡長什麼樣,取決於堆疊。一種堆疊裡的呼叫是 CALL name,另一種是注入物件上的一個方法。所以沒有現成的解析器涵蓋你程式碼的每一種慣例。語言伺服器或程式碼搜尋服務已經給你物件和使用的地方,先用它們(見 什麼時候可以刪掉)。你為它們沒涵蓋的部分寫抽取器。
toolkit 是一小組程式,加上一個設定檔,全都放在知識庫裡面:
- 抽取器讀一種檔案,印出它定義的物件和它找到的使用。
- 設定檔說明在你的程式碼裡什麼是物件、哪些種類的使用算數、哪些類型的物件是入口(外面的世界呼叫進來的地方:畫面、API、排程工作)、哪些資料夾已經退役,以及名字裡哪些詞要忽略。
執行器把抽取器用在每個倉庫的主分支上,並把地圖存進快取。下一次執行時,只重讀上次之後改過的檔案。
- 程式碼才是真相。物件和「用」就寫在裡面,形式取決於你的技術棧。
- 抽取器是小腳本。每支讀一種檔案,印出它找到的物件和「用」。同樣的輸入,同樣的輸出。
- 輸出就是地圖:有哪些物件,誰用誰。
- 幾個人本來就知道的「用」寫下來當測試。每次執行都必須全部找到。
- 下一次,只重讀改過的檔案。
誰寫:模型設計,人批准,腳本執行
你可能預期要自己寫抽取器。在我的設定裡,是模型寫,分四步:
- 調查。一個腳本數出每個倉庫裡各種檔案的數量。模型讀每一種的幾個樣本。
- 設計。模型寫下在這個堆疊裡,物件和使用長什麼樣,附上程式碼裡的例子。人讀它並批准。這是抓出「那不是使用,那是註解」的時刻。
- 建造。模型寫抽取器。
- 執行,直到已知的使用都找到。見下一節。
模型擅長把解析器寫一次。它不該每一次都當解析器。建好之後,只有腳本碰程式碼。
一個一直留著的測試:人知道的那些使用
地圖第一次做出來時,你怎麼知道它不是胡說?人寫下 5 到 10 個他知道存在的使用,例如「預存程序 A 呼叫預存程序 B」。這些放進 toolkit 裡的一個檔案。最好在抽取器出現之前就寫,並且在模型寫的時候不給它看,這樣抽取器就不會只對這幾個調校。地圖的每次執行,都必須找到所有這些。如果之後對抽取器的改動弄丟了一個,這次執行就失敗。
這是迴歸測試:守住已經行得通的東西的測試。它很小,我不假裝它量得出整張地圖。每次你遇到地圖漏掉的使用,就把它加進去。第 8.3 章 量地圖有多完整。
執行器每次執行也會回報:
- 它讀不了的檔案,以及它讀了卻什麼都沒找到的檔案(兩者都可疑);
- 被使用卻沒在任何地方定義的名字:它們可能是別的系統、被刪掉的程式碼,或抽取器讀錯的寫法。看幾個。
兩個範例,只當例子
harness 沒有附帶自己的 toolkit,只有兩個範例,讓你看到它的形狀。
- SQL Server:預存程序、排程的 agent 工作,和整合套件。
- Spring:一個 Spring Boot 服務,有 controller、repository、entity 和資料表。
我在開源專案 spring-petclinic 上建了 Spring 範例。在那份程式碼上,它在 35 個檔案裡找到 58 個物件,6 個已知的使用全都找到了;沒有檔案讀取失敗,也沒有檔案是空的。這是冒煙測試:它證明 toolkit 在一個小而整齊的專案上跑得起來。它不證明在你的專案上行得通。難的情況(執行時才組出來的 SQL、由設定驅動的呼叫)不在那個專案裡。你的堆疊、你的慣例和你那些奇怪的情況,要由你來寫。
Spring 範例的真實檔案,讓你看到形狀。先是人為它寫下的已知使用:
範例的五個已知使用:一個物件、使用的種類、另一個物件。
{"from": "controller:PetController", "kind": "uses", "to": "repository:PetTypeRepository"}
{"from": "controller:VetController", "kind": "uses", "to": "repository:VetRepository"}
{"from": "repository:OwnerRepository", "kind": "reads", "to": "entity:Owner"}
{"from": "entity:Owner", "kind": "maps", "to": "table:owners"}
{"from": "entity:Pet", "kind": "uses", "to": "entity:Visit"}.harness/examples/kb-toolkits/spring/facts.jsonl L4–L8 @ 1.5.50
然後是讀一個 Java 檔案、印出它的物件和使用的函式:
Java 檔案的抽取器:找出類別、它的角色,和它用到的型別。
def java(text: str, context: Dict[str, Any]) -> List[Dict[str, Any]]:
clean = blank_comments(text)
match = TYPE.search(clean)
if not match:
return []
name, head = match.group(2), clean[:match.start()]
role = next((role for mark, role in ROLE if mark in head), "class")
repository = REPOSITORY.search(match.group(3))
if repository:
role = "repository"
refs: List[Dict[str, Any]] = []
if repository:
refs.append({"kind": "reads", "name": repository.group(1), "as": "entity", "line": line_at(clean, match.start(3))})
parent = EXTENDS.search(match.group(3))
if parent and not repository:
refs.append({"kind": "extends", "name": parent.group(1), "line": line_at(clean, match.start(3))})
table = TABLE.search(head)
if table:
refs.append({"kind": "maps", "name": table.group(1), "as": "table", "line": line_at(clean, table.start(1))})
body = clean[match.end():]
offset = line_at(clean, match.end()) - 1
for used in used_types(body):
if used != name:
refs.append({"kind": "uses", "name": used, "line": offset + 1})
found = [{"id": f"{role}:{name}", "type": role, "name": name, "line": line_at(clean, match.start(2)), "dynamic": False, "refs": refs}]
if role == "controller":
found += endpoints(clean, head, body, offset, name)
return found.harness/examples/kb-toolkits/spring/extract_spring.py L50–L77 @ 1.5.50
還有範例自己對建造過程的紀錄,包括漏了一個使用的第一次執行,以及它看不到的東西:
修抽取器的兩輪,和留下的盲點。
## Rounds
1. 9 of 10 facts: `new ModelAndView("owners/ownerDetails")` not known. Also `welcome.html` at the top of `templates/` was not matched by `templates/**/*.html`: fixed in the harness (`**/` stands for no folder too).
2. 10 of 10. An added fact on a view returned through a constant failed; constants are now read. 11 of 11.
## Blind spots
- Join tables named only in `@JoinTable` (`vet_specialties`) have no use.
- Beans wired by the framework (`@Configuration`, formatters) show as `unused`: the framework calls them..harness/examples/kb-toolkits/spring/DESIGN.md L22–L30 @ 1.5.50
為什麼通用的 harness 不附 toolkit
我沒有為常見的堆疊附上解析器。幾乎正確的解析器會無聲地失敗,而且每個團隊寫程式碼的方式都有點不同。團隊自己擁有、自己抽查的 toolkit(下一章),漏掉東西時可以修,而且它放在知識庫裡,所以升級 harness 不會覆蓋它。
測你自己的設定
- 挑一個倉庫。寫下 5 到 10 個你知道存在的使用,每個附上檔案和行號。
- 請 agent 草擬在你的堆疊裡物件和使用長什麼樣,附上你程式碼裡的例子。讀它,修正它。
- 讓 agent 寫抽取器。執行它們,檢查你所有已知的使用都出現了。
- 執行兩次。兩次的輸出必須完全一樣。
原理
toolkit 是什么
一个存储过程、一个任务或者一个类在文件里长什么样,取决于技术栈。同样是一次调用,在一种栈里是 CALL name,在另一种栈里是注入对象上的一个方法。所以没有哪个现成的解析器能覆盖你代码里的所有写法。如果语言服务器或代码搜索服务已经给了你对象和使用,先用它们(见什么时候可以删掉)。它们覆盖不到的部分,你来写抽取器。
toolkit 是一小组程序加一个配置文件,都放在知识库里:
- 抽取器读一种文件,打印它定义的对象和找到的使用。
- 配置文件写明在你的代码里什么算对象,哪几种使用算数,哪几类对象是入口(外部调用进来的地方:一个页面、一个 API、一个定时任务),哪些文件夹已经停用,以及名字里要忽略哪些词。
一个运行器把抽取器用到每个仓库的主分支上,把地图存进缓存。下一次运行,只重新读上次以来改过的文件。
- 代码才是真相。对象和使用就写在里面,形式取决于你的技术栈。
- 抽取器是小脚本。每个读一种文件,打印它找到的对象和使用。同样的输入,同样的输出。
- 输出就是地图:有哪些对象,谁用谁。
- 几处人本来就知道的使用,写下来当测试。每次运行都必须全部找到。
- 下一次,只重读改过的文件。
谁来写:模型设计,人审批,脚本运行
你可能以为抽取器要自己写。在我的配置里是模型来写,分四步:
- 调研。一个脚本统计每个仓库里各种文件的数量。模型每种读几个样本。
- 设计。模型写下在这个技术栈里,对象和使用长什么样,附上代码里的例子。人来读,并审批。这时正好可以抓出“那不是使用,那是注释”。
- 构建。模型写抽取器。
- 反复运行,直到已知的使用都找到。见下一节。
模型擅长把一个解析器写一次。它不该每次都充当解析器。构建完成之后,碰代码的只有脚本。
一个一直留着的测试:人知道的那些使用
地图刚做出来时,你怎么知道它不是胡说?由人写下 5 到 10 处他们知道确实存在的使用,比如“存储过程 A 调用存储过程 B”。这些放进 toolkit 里的一个文件。最好在抽取器出现之前就写好,模型写抽取器时也不给它看,这样抽取器就不会只为这几处调出来。每次运行地图,都必须找到全部这些使用。以后改抽取器时丢了一处,这次运行就失败。
这是一个回归测试:保护已经好使的东西的测试。它很小,我也不假装它衡量了整张地图。每当你发现地图漏掉一处使用,就把它加进去。第 8.3 章衡量地图有多完整。
运行器每次运行还会报告:
- 它读不了的文件,以及读了却什么都没找到的文件(两种都可疑);
- 被用到但哪里都没定义的名字:它们可能是别的系统、已删除的代码,或者抽取器读错的写法。挑几个看看。
两个样例,只当例子
harness 自己不带 toolkit,只带两个样例,让你看到它的样子。
- SQL Server:存储过程、定时的 agent 任务,以及集成包。
- Spring:一个 Spring Boot 服务,有 controller、repository、实体和表。
我在开源项目 spring-petclinic 上做了 Spring 样例。在那份代码上,它找到了 35 个文件里的 58 个对象,已知的 6 处使用也全部找到;没有文件读失败,也没有文件读出来是空的。这是一次冒烟测试:它说明 toolkit 能在一个小而整洁的项目上跑起来。它不说明它在你的项目上管用。难的情况(运行时拼出来的 SQL、由配置驱动的调用)不在那个项目里。你的技术栈、你的约定、你那些古怪的情况,要你自己来写。
Spring 样例的真实文件能看出它的样子。先是人为它写下的已知使用:
样例已知的五处使用:一个对象、使用的类型、另一个对象。
{"from": "controller:PetController", "kind": "uses", "to": "repository:PetTypeRepository"}
{"from": "controller:VetController", "kind": "uses", "to": "repository:VetRepository"}
{"from": "repository:OwnerRepository", "kind": "reads", "to": "entity:Owner"}
{"from": "entity:Owner", "kind": "maps", "to": "table:owners"}
{"from": "entity:Pet", "kind": "uses", "to": "entity:Visit"}.harness/examples/kb-toolkits/spring/facts.jsonl L4–L8 @ 1.5.50
再是读一个 Java 文件、打印它的对象和使用的函数:
Java 文件的抽取器:找出类、它的角色,以及它用到的类型。
def java(text: str, context: Dict[str, Any]) -> List[Dict[str, Any]]:
clean = blank_comments(text)
match = TYPE.search(clean)
if not match:
return []
name, head = match.group(2), clean[:match.start()]
role = next((role for mark, role in ROLE if mark in head), "class")
repository = REPOSITORY.search(match.group(3))
if repository:
role = "repository"
refs: List[Dict[str, Any]] = []
if repository:
refs.append({"kind": "reads", "name": repository.group(1), "as": "entity", "line": line_at(clean, match.start(3))})
parent = EXTENDS.search(match.group(3))
if parent and not repository:
refs.append({"kind": "extends", "name": parent.group(1), "line": line_at(clean, match.start(3))})
table = TABLE.search(head)
if table:
refs.append({"kind": "maps", "name": table.group(1), "as": "table", "line": line_at(clean, table.start(1))})
body = clean[match.end():]
offset = line_at(clean, match.end()) - 1
for used in used_types(body):
if used != name:
refs.append({"kind": "uses", "name": used, "line": offset + 1})
found = [{"id": f"{role}:{name}", "type": role, "name": name, "line": line_at(clean, match.start(2)), "dynamic": False, "refs": refs}]
if role == "controller":
found += endpoints(clean, head, body, offset, name)
return found.harness/examples/kb-toolkits/spring/extract_spring.py L50–L77 @ 1.5.50
还有样例自己的构建记录,包括第一次运行漏掉一处使用,以及它看不到什么:
抽取器修了两轮,以及还剩下的盲区。
## Rounds
1. 9 of 10 facts: `new ModelAndView("owners/ownerDetails")` not known. Also `welcome.html` at the top of `templates/` was not matched by `templates/**/*.html`: fixed in the harness (`**/` stands for no folder too).
2. 10 of 10. An added fact on a view returned through a constant failed; constants are now read. 11 of 11.
## Blind spots
- Join tables named only in `@JoinTable` (`vet_specialties`) have no use.
- Beans wired by the framework (`@Configuration`, formatters) show as `unused`: the framework calls them..harness/examples/kb-toolkits/spring/DESIGN.md L22–L30 @ 1.5.50
为什么通用的 harness 不带 toolkit
我没有为常见的技术栈提供解析器。差不多对的解析器会悄悄出错,而且每个团队写代码的方式都略有不同。团队自己拥有、自己审计的 toolkit(下一章)漏了就能修,而且它放在知识库里,所以升级 harness 永远不会把它覆盖掉。
在你自己的环境里测一测
- 挑一个仓库。写下 5 到 10 处你知道确实存在的使用,每处带文件和行号。
- 让 agent 起草在你的技术栈里对象和使用长什么样,附上你代码里的例子。读一遍,改正它。
- 让 agent 写抽取器。运行它们,检查你所有已知的使用都出现了。
- 运行两次。两次的输出必须完全一样。
Costs and traps
- The toolkit is code to maintain. When your stack changes, so does it. That is the price of owning it.
- It is only as good as its reading of the forms. An extractor that handles three ways of writing a call and misses a fourth looks fine on the known uses. That is why we audit (8.3).
- Some things no script can see. A query built as a string at run time, or a call driven by a config table, is invisible to any parser. These are marked, not guessed (8.3).
- I have not tested the toolkit on a very large code base. The petclinic run is a small case.
代價和陷阱
When to delete it
If a tool you trust, such as your language server or a code-search service, already gives you objects and uses with a measured accuracy, feed its output into the map and drop your extractors. The rest of the road does not care where the map came from.
什麼時候可以刪掉
如果一個你信任的工具,例如你的語言伺服器或程式碼搜尋服務,已經給你物件和使用,並且有量過的準確度,就把它的輸出餵進地圖,放掉你的抽取器。路線的其餘部分不在乎地圖從哪裡來。
什么时候可以删掉
如果你信任的某个工具,比如你的语言服务器或代码搜索服务,已经给出了对象和使用,而且准确度测量过,就把它的输出喂给地图,丢掉你自己的抽取器。路线上的其余部分不关心地图是从哪来的。
Try it on your own setup
No harness needed. Open your agent in your own repository and paste:
Scan this repository for the kinds of dependency that a script reading the code cannot see: queries or commands built as text at run time, calls chosen by a variable or a configuration row, and names that resolve to something outside this repository. Give a rough count of each kind, with three example locations (file and line). Say what share of the code looks hard to map by script. Do not change any files.在你自己的環境試試
不用裝 harness。在你自己的倉庫裡打開 agent,貼上:
掃描這個倉庫,找出讀程式碼的腳本看不見的那幾種依賴:執行時組成文字的查詢或指令、由變數或設定列選擇的呼叫,以及解析到這個倉庫之外的名字。每一種給一個大略的數量,附三個例子的位置(檔案和行號)。說明程式碼中看起來難以用腳本畫地圖的比例。不要改任何檔案。在你自己的环境里试试
不用装 harness。在你自己的仓库里打开 agent,粘贴:
扫描这个仓库,找出读代码的脚本看不到的那几类依赖:运行时拼成文本的查询或命令、由变量或配置行决定的调用、以及解析到这个仓库之外的名字。每一类给出大致数量,并各举三个位置(文件和行号)。说明代码中看起来难以用脚本映射的部分占多大比例。不要修改任何文件。Go deeper
- A knowledge base: an honest index of the code
- The audit: how to measure the map you just made.
延伸閱讀
- 知識庫:程式碼的誠實索引
- 地圖對不對:盲抽查:怎麼量你剛畫出來的地圖。
延伸阅读
- 知识库:代码的诚实索引
- 地图对不对:盲抽查:怎么衡量你刚做出来的地图。
You can stop here: that is the whole idea. The next part shows where it runs, so you can look inside.讀到這裡就夠了,這就是整個想法。下一部分告訴你它在哪裡運行,可以進去看看。读到这里就够了,核心思路就是这些。下一部分告诉你它在哪里跑,可以进去看看。
In my practice (Copilot Harness)實作实战
Copilot Harness is my open-source harness for VS Code Copilot.
Repository: https://github.com/Smars-Bin-Hu/copilot-agentic-harness
Get started: clone it, open it in VS Code, and type /harness-rookie-get-me-ready in Copilot Chat. You need Python 3.9 or later and git.
Copilot Harness 是我開源的 VS Code Copilot harness。
倉庫: https://github.com/Smars-Bin-Hu/copilot-agentic-harness
開始:clone 下來,用 VS Code 打開,在 Copilot Chat 裡輸入 /harness-rookie-get-me-ready。需要 Python 3.9 以上和 git。
Copilot Harness 是我开源的 VS Code Copilot harness。
仓库: https://github.com/Smars-Bin-Hu/copilot-agentic-harness
上手:把仓库 clone 下来,用 VS Code 打开,在 Copilot Chat 里输入 /harness-rookie-get-me-ready。需要 Python 3.9 及以上版本和 git。
Explore it in the harness
Copilot Harness has the commands that survey the code, build the map and run the checks, and the two sample toolkits. After the setup above, type:
/harness-rookie-wondering I have read the chapter "The toolkit: scripts that draw a map of the code". Teach me topic 16, parts 16.1 and 16.2: how the map is built and cached. Open the Spring sample toolkit and its known-uses file with me, and show me what an extractor looks like.It teaches one small piece at a time, stops to ask if you are following, and opens the files with you.
在 harness 裡探索
Copilot Harness 有調查程式碼、建地圖和執行檢查的指令,也有那兩個範例 toolkit。照上面的說明準備好之後,輸入:
/harness-rookie-wondering 我讀完了「Toolkit:用腳本畫出程式碼的地圖」這一章。請教我主題 16 的 16.1 和 16.2:地圖是怎麼建出來和快取的。和我一起打開 Spring 範例 toolkit 和它的已知使用檔案,讓我看抽取器長什麼樣。它一次教一小塊,停下來確認你跟得上,然後和你一起打開檔案。
在 harness 里探索
Copilot Harness 里有调研代码、构建地图、运行检查的命令,还有两个样例 toolkit。按上面的说明准备好之后,输入:
/harness-rookie-wondering 我读完了“Toolkit:用脚本画出代码的地图”这一章。请教我主题 16 的 16.1 和 16.2:地图是怎么构建和缓存的。带我打开 Spring 样例 toolkit 和它的已知使用文件,让我看看抽取器长什么样。它一次只教一小块,停下来确认你跟上了,再和你一起打开文件。