Disregard the API completely and just scrape the web interface of specific subreddits. It really doesn't matter that things won't update in real time.
this post was submitted on 16 Jun 2023
16 points (100.0% liked)
Reddit Migration
37 readers
2 users here now
### About Community Tracking and helping #redditmigration to Kbin and the Fediverse. Say hello to the decentralized and open future. To see latest reeddit blackout info, see here: https://reddark.untone.uk/
founded 1 year ago
I was thinking about that. In theory if I can see it, what is stopping a bot from grabbing it. I notice when I use search engines they have no problem seeing what is inside reddit - so there has to be a way to create a bot that does this. I'm just not programmer savvy in that way to know how it works.
Don't platforms like Facebook or Instagram have detection against web scraping and they ban you really fast for that or something similar? I'd imagine Reddit would have some protection against that as well, no?
I'm not expert, for me u are blocked when u exceed an request limits. Don't code ur scrapper directly on google or the site but with an copy of the page.
I've been blocked when I coded , never at using it. But I'm not an expert, just amateur so it's a pure guess
My point of view is : Just take their contents manually or with not, to discuss about it here. With the indexation by search engines, fediverse will be the new reference.
For ur last point the solution is reddit choosing to transfers to the fediverse system. There is (my POV) no interest to Dev an solution to make reddit connected to Lemmy/kbin
I don't mind doing it manually (I was planning to do it for now anyway), but it would be nice to have an automated process for that. Plus, I don't think Reddit will ever allow content migration to Fediverse, considering they were censoring stuff related to Kbin/Lemmy a few days ago...
You bring up a good point in the reddit API pricing, in that the fediverse would then become like a giant third party reddit client. One advantage being that the API calls could be much fewer in total, because any given post or comment only has to be requested once, and after that, they'd be on the fediverse, available to all.
Also it might have copyright implications.
I don't think it has copyright implications, as the other user has shown, unless it's some original work. I wouldn't mind the posts on the Fediverse mentioning the original authors on Reddit and even linking back to the post just in case.
Except platforms often claim ownership over user generated content. Legally, reddit could absolutely declare that copying stuff off their platform is stealing. That's kind of why this whole API thing is happening, as least they claim, that they are doing it to monetize the usage of reddit as training data for AI.
If we start taking that and making it accessible outside reddit, I am almost certain they will retaliate in some way.
You can simply add .json to the end of any sub or account to get the data, so it shouldn’t be too hard to set something up, assuming there is a POST API in kbin. Example: https://www.reddit.com/r/voidlinux.json
This is good to know, will it still work with the API changes?
Guess time will tell
Yep, saw another user mention this as well. I never knew this was possible, makes things so much easier haha.
I haven't checked Kbin's API documentation, but I assume (and hope) there is a post API endpoint.
edit: Just checked and it seems like you're only able to do GET requests at the moment (link to docs). Will be waiting for updates from @ernest in this case.
Keep in mind that Reddit has rate limits, and they may throttle your requests if you make many. There's some (probably out of date) documentation here.
They'd probably try DMCA takedowns I imagine.
They can't do that one under their terms of use, at least not directly:
By submitting user content to reddit, you grant us a royalty-free, perpetual, irrevocable, non-exclusive, unrestricted, worldwide license to reproduce, prepare derivative works, distribute copies, perform, or publicly display your user content in any medium and for any purpose, including commercial purposes, and to authorize others to do so
So OP retains copyrights, Reddit has a nonexclusive license to it forever, but only OPs DMCA would be valid.
view more: next ›