Naturally, the LLM can generate interpretations and probabilities which are not correct. Yet, we believe that an LLM is a good-enough measure of semantic ambiguity for our needs.

You’re asking an LLM to invent a probability distribution over meanings. Embedding models are explicitly trained to represent semantic similarity geometrically. Why not use the right tool for the job over "slop"?

1. Download you an embedding model — you can run it on garbage, or pay for an API if you want.
2. Embed the identifier
3. Embed the symbol’s implementation summary (or docstring)
4. cosine(name_embedding, description_embedding)
5. Have LLM suggest alternatives and repeat
6. Rank names by semantic similarity.

ETA:

I present: https://gitlab.com/MagickShel/namer

It doesn't generate the list of names to test, though. I leave that to the imagination of the user.

ETAA:

There is a class of naming it doesn't work great on. I probably overstepped and was a bit of an ass. This just gave me the inspiration to do basically the thing you did here, the way I have told myself for months it should be done.

Also, there is a class of naming - jargon and acronyms - where your approach works better. I still prefer something more deterministic, but credit where credit is due.