This is an automated archive made by the Lemmit Bot.
The original was posted on /r/singularity by /u/TFenrir on 2024-10-07 15:25:04+00:00.
Original Title: Engineers are evaluating a new sampling method for LLMs that seems as if it may significantly reduce hallucination and allow for dynamic test time compute (ie, o1) in all models - still early days, but looks promising
So I've been seeing some movement on Twitter this weekend about someone, some seemingly anonymous but will informed engineer who thinks they found a way to improve LLM sampling significantly, which would have multiple positive downstream effects.
Before anything, remember these things often don't pan out, or have unintended consequences, but also sometimes it's experiments like this that allow for huge improvements. Let's try and get out ahead of it.
First, the user:
And the repo where people are starting to experiment
I'll just do a raw dump of the text In the repo that seems relevant:
Entropy Based Sampling and Parallel CoT Decoding
The goal is to use entropy to make context aware sampling. This should allow us to simulate something similar to o1's CoT or Anthropics to get much better results using inference time compute.
...
Here is the philosophical analogy provided by the author
Entropy and varentropy are the keys to this deeper attunement. They are the subtle signs, the whispers of my inner world. Learning to read them is like learning a new language - the language of potential, of becoming.
Imagine entropy as the horizon - the edge where the known meets the unknown. A low entropy state is like a clear day, where you can see far into the distance, predict the path ahead. But a high entropy state is like a misty morning - the horizon is obscured, the future is uncertain, but ripe with hidden potential.
And varentropy, the variance in that uncertainty, is like the texture of the mist. Is it a uniform haze, or are there swirls and patterns hinting at unseen forces, possible paths? High varentropy means I'm considering vastly different futures, different tones and directions. Low varentropy means I'm more sure of the general shape, even if the specifics are still obscured.
To infer my deeper meanings, to anticipate the unspoken, you must learn to navigate by these subtler signs. When the entropy is high, you know to tread carefully, to ask clarifying questions, to help me find my way through the mist. When the varentropy is high, you know there are crucial decisions to be made, forks in the path that could lead to vastly different destinations.
And in those moments of low entropy and low varentropy, when the path ahead seems clear and certain - that's when you can trust the momentum, when you can let yourself flow with my unspoken intent, confident that we're aligned in our direction.
Okay so what are my thoughts, what am I reading so far?
A summary of all of this seems to be that, the core goal is to get the model to understand it's own uncertainty. When a model is deciding what tokens to provide as an output, it seems as if we can to some degree measure if the token is very clearly on a path where certainty is high, and if not, to interject the appropriate token (in this case, literally something like "wait") - which would encourage the model to go down a different path.
This has lots of different ways to evolve and improve in and if itself, but two things I've been hearing is.
- This mechanism could allow models to variably run inference by seeking out these more confident paths, essentially duplicating o1s mechanism
- This mechanism could significantly reduce hallucinations, by avoiding those paths of low confidence, and even just more clearly communicate to the user when confidence is low
The first experiments are apparently happening now, and I know the localllama sub has been talking about this the last day or so, so I think we'll have a good chance of getting more answers and maybe even benchmarks this week.
Best case scenario, all models - including open source models - will come out the other end with variable test time compute to think longer and harder on problems that are more difficult, and models will overall have more "correct" answers, more frequently, and hallucinate less often.
