7

I should start with a confession: almost every interesting result in this project showed up first as a bug.
The value function looked wrong before it looked right, the defensive zones came out more valuable than the penalty box, which is the kind of thing that should make you stop and distrust everything downstream of it. The headline policy comparison flipped sign between two runs of the same code with a different random seed, which meant the number I nearly wrote down as “the effect” was, for a while, just noise wearing a confidence interval as a costume. None of this is a special property of football, or of me being particularly careless. It’s what happens whenever you try to build something quantitative that continuously simulates itself: every layer you add is a new place for a plausible-looking number to be wrong in a way that doesn’t announce itself.

top 1 comments
sorted by: hot top new old
[-] CombatWombat@feddit.online 2 points 2 weeks ago

I find this to be an important insight about statistical models -- they do not, and possibly cannot, account for the fact that there is another team on the field who will notice and react to changes you make

There’s a limit to how far any of this should be trusted, and it’s worth stating plainly. The odds this model learned were produced by real defenses reacting to what teams actually do. If a team almost never crosses from deep, opposing full-backs don’t have to respect that threat and can tuck inside, which is baked into the data the model learned from. Simulate a version of that team that suddenly crosses a lot more, while keeping those same learned odds fixed, and the simulation is quietly assuming the opposition doesn’t notice or adjust.

this post was submitted on 18 Oct 2026
7 points (100.0% liked)

Football

564 readers
13 users here now

Rules

founded 2 years ago
MODERATORS