Yeah, I think if AI was good at translations, we'd see a lot more localizations of media, like manga/mangua/manghwa, video games, etc.

But the reality is that when people say LLMs are great at translation, it's more of a "they can make things understandable (some portion of the time)" not "they make it sound as if a native of the other language is saying the exact same thing in that language that you said in yours".

LLMs make token associations, there is no understanding. They'll need bigger neural nets and even better training data to model understanding. They seem to be making progress in programming but that's because the whole pipeline can be better automated: a program can be compiled to find syntax or grammar errors, and it can be run to check the behaviour. There is no automatic feedback mechanism for things like translation or image generation.

It's all subjective, too, so someone who wants the models to be perceived as "good" will be more likely to perceive it as good themselves and miss or forgive errors, while skeptics will be looking closely for them. Neutrals won't be looking for errors but will recognize the uncanny valley (which bad translations can also fall into, though they are usually perceived more as funny than creepy).