10

cross-posted from: https://slrpnk.net/post/24729225

Are there any open datasets that track who is federated / defederated with who? I have the Lemmyverse datasets but it has nothing on node relationships.

lemmy-federate.com appears to have the info I am after, but it only appears to be reachable in a GUI webpage. Any way to get the dataset?

you are viewing a single comment's thread
view the rest of the comments
[-] iso@lemy.lol 2 points 1 year ago

You can easily fetch through the API. You can either check API routes from the repo or I can give you some examples if you want to?

[-] activistPnk@slrpnk.net 2 points 1 year ago* (last edited 1 year ago)

I don’t have full time Internet access. I pop into a library periodically. So I have found it useful to fetch the lemmyverse dataset and import it. Takes just a few seconds to fetch the whole dataset. Then when I am offline I search the DB to decide where to post, and write my posts offline.

I’m trying to grasp what you are suggesting. Do you mean I write a program to reach out to each instance and harvest the data? I think there are around 150 lemmy instances. I’m not sure how quickly that can be done, or if it would be welcome. When I visit slrpnk.net using my browser, there is an anti-bot check which takes a few seconds. I don’t imagine cURL or the like would get past that, which I suspect is some javascript code that must be executed.

In your repo I see routes/community.ts and nodes.ts. But therein I do not see a list of instances or communities to visit. Is ts for typescript? That lang is unknown to me.

[-] iso@lemy.lol 2 points 1 year ago

Here you go:

https://lemmy-federate.com/api/community.find?input=%7B%22skip%22%3A0%2C%22take%22%3A10%7D
https://lemmy-federate.com/api/instance.find?input=%7B%22search%22%3A%22%22%2C%22skip%22%3A0%2C%22take%22%3A10%2C%22enabledOnly%22%3Afalse%7D

You can change the skip and take values to paginate through list.

[-] activistPnk@slrpnk.net 1 points 1 year ago* (last edited 1 year ago)

thanks!

The instance list is tiny. But perhaps that doesn’t matter since instances can be derived from the community list.

this post was submitted on 15 Jul 2025
10 points (100.0% liked)

[moved to lemmyfederate@lemmy.world]

120 readers
1 users here now

Updates and questions about lemmy-federate.com

founded 2 years ago
MODERATORS