Technology

Yeah they call that artificial data and tout it as a good thing

That was a problem that's been basically solved. So no, it's not actually an issue anymore.

That's why this is happening. You basically want to train a model that's going to train the next model and that's going to train the next model.

Doing a bunch of other fancy s*** means that each consecutive level improves the quality instead of reducing it.

So we're at the point where it's more effective to let your opponent do all the heavy, more expensive training of the less effective model. Then train the model that you want to make on that because paying them to run their finished model is cheaper than making it in the first place to train the next one.

In a sense, it's sort of like compression. The harder you compress the longer and more expensive and more difficult it becomes. So ideally you want to be the last guy in the chain to do the final compression so you get a reap all of the rewards without any of the overhead.