LLM 360 probably deserves a mention there, trying to be open, reproducible and publishing intermediate training checkpoints.

meaning they’re not made up of unethically sourced training material

I am old enough to remember the days before 'piracy' was considered unethical. I still stand by that. Authors need to be compensated but also "publication" is supposed to give some rights to the "public". Sharing non-commercially things you love over internet used to be a gray zone that should have been the subject of a political debate. Instead, right holders decided that the letter of the law should strictly apply to conditions it was never designed for. I have little empathy for them moaning now that AI training, while not formally illegal, should be considered unethical.

A copyright reform is long overdue. Let's discuss it seriously instead of playing the game of "who can lobby the most?". In the meantime, enjoy the gray zone that refusing to update 19th century laws to the century of internet causes.

https://eurollm.io/ ?

Looks like they go over their data sets here:

https://www.sciencedirect.com/science/article/pii/S1877050925006210

https://huggingface.co/togethercomputer

Hopefully someone else knows more, I don't really care, I just want Europe to catch up to China/America otherwise it won't matter how ethical the LLM is because everyone will be using Chinese/American anyway like 99% of people already are

I see Olmo by AllenAI frequently used in research. The publish all the checkpoints of training and all training sources. https://allenai.org/olmo / https://huggingface.co/collections/allenai/olmo-3

Here's the link to the dataset used to train Olmo 3: https://huggingface.co/datasets/allenai/dolma3_mix-6T

It's fairly modern. Olmo 3.1 was released January 2026.

I recall OpenAI saying to a courtroom judge that modern AI requires training off copyrighted material. And obviously the AI labs cannot just release those.

Distlled models (aka models that train themselves off the responses from other AI models) could probably release all the AI responses they used for training, since afaik AI generated content is not copyrightable..

However I suspect that most modern models are built from previous models. For example Claude Fable is probably based on Claude Sonnet or something. And for a downstream model to be fully "open source" and reproducible, you would need to reproduce the upstream model too. So even if a downstream model didn't use copyrighted data (eg it only used distills), one of the upstream models likely did.

https://huggingface.co/spaces/Soofi-Project/Pretraining-Tech-Report

Many of the Chinese LLMs have open source models, even frontier models.

Nemotron is a fairly significant model family in this space, IIRC. e.g. here's one of the recent releases: https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16

Edit: Not fully open -- see thread below.