Will Cook

Stale descriptions

Their work
Glean is Meta’s open-source program that indexes code and answers questions such as which functions call a given function. The three Python files in this test are Glean’s own example.
My work
I wrote sentences that describe those three files and marked each one true or false after each change to the code. A script asks Glean which functions call or refer to the changed code and lists the sentences about those functions.

I ask AI coding assistants to keep a few sentences about each file inside that file. Editing a function in one file can make a sentence in another file false. In the test, one function is changed from doubling a number to tripling it, and a sentence about its caller in another file, that it returns twice the value, becomes false. Glean returns that caller, and the script lists its three sentences for rereading.

Limits: Three files from Glean’s own Python example, with sentences and true-or-false verdicts that I wrote by hand. It shows the mechanism. How often this happens in a real project is untested.

Code on GitHub

I use AI to make AI-written maths and code easier to understand.

All twenty-one projects