this post was submitted on 10 Jun 2023
520 points (97.6% liked)
Asklemmy
43952 readers
729 users here now
A loosely moderated place to ask open-ended questions
Search asklemmy π
If your post meets the following criteria, it's welcome here!
- Open-ended question
- Not offensive: at this point, we do not have the bandwidth to moderate overtly political discussions. Assume best intent and be excellent to each other.
- Not regarding using or support for Lemmy: context, see the list of support communities and tools for finding communities below
- Not ad nauseam inducing: please make sure it is a question that would be new to most members
- An actual topic of discussion
Looking for support?
Looking for a community?
- Lemmyverse: community search
- sub.rehab: maps old subreddits to fediverse options, marks official as such
- !lemmy411@lemmy.ca: a community for finding communities
~Icon~ ~by~ ~@Double_A@discuss.tchncs.de~
founded 5 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
I hate reddit. But it feels like the library of Alexandria burning down (yea I know). All those google search results and educational subreddits that are shutting down forever, and because they are too small reddit won't force open them again.
A lot are in the pushshift archive, but that cuts of at 2022. Also, it doesn't include a lot of the smaller subreddits.
I have had my PC running 24/7 with multiple VPNs to avoid rate limits downloading as much as I can before the API dies, but with some blackouts moving forward a day I have already missed a few.
Like many others, I would often add "reddit" to the end of my searches to get better results, half the websites on web searches now are either AI generated, copies or on completely AD ridden websites that ask you to turn off your AD blocker.
I think this is honestly the biggest issue. Web search has been garbage for years, with legit the only saving grace being Reddit users sharing their knowledge. This is gonna have a horrible effect on producing good search results.
Totally agree. I feel like this is the equivalent, to some degree, of Stack Overflow just suddenly going away. The history needs to be preserved, somehow.
Is there anything in the Fediverse that is like a Stack Overflow clone? Might be time to start working on the backup plan for those big websites that do not show a sign of going away yet to avoid the rush when they inevitably do.
People have been joking lately about productivity suddenly increasing as a result of the Reddit blackout, but honestly? That loss of information is probably going to result in a loss of productivity in some cases.
Because yeah, in the nightmare scenario where both Reddit and Stack Overflow were to disappear, a lot of programmers would be at a complete loss.
It's valuable knowledge with how-to's that made me create an account there. I learned plentiful with the people that cared to share.
Most i implemented into my daily life & conditions have become favorable for me.
It's unfortunate that Reddit Company have refused to collaborate with its users, since years back. Otherwise we would have seen their web & mobile app develop/ innovate in great ways. But they have chosen one limitation after the other. Slowly over the years.
Reddit has answered almost every question I've ever had for years. The potential loss of all the knowledge is my greatest concern.
i getcha, but it was people who did that. it's kind of hard to shut us up, we'll answer more questions wherever we are
most knowledge has a shelf life anyway
how exaclty does this pushshift work? I downloaded some zsts from it but what do I do with them?
The file you downloaded is a compressed JSON file, it's not something you can really just look at. But it contains all the data needed to build a nice UI around.
I don't know what OS you are on but on linux you can run
zstd -d -c file.zst | jq .and it will print everything in the file. It's not really readable though. Also it doesn't have any of the media content, only the textThe Archive Team has been working on reddit for a while now. I've had six workers running since imgur slowed down. If you can spare a little bit of computing power we could use your help! You can learn more here:
https://wiki.archiveteam.org/index.php/ArchiveTeam_Warrior
Subreddits are still private but there data isnβt lost so the knowlege loss isnβt irretrievable mods from the subreddits will be able to transfer knowledge from Reddit to Lemmy
That assumes the mods do. I fear reddit will turn into another tinypic situation. even if it isn't an image host there will be pages of answers on forums and stack exchange pointing to dead reddit links. The fustration of finding a 10 year old forum post of someone having the same issue as you only to have the only answer point to a dead link is incredible.