Because the bot heavily relied on old.reddit.com, and access to that now requires a login, the bot is suspended.

For the longest time using old.reddit.com was the only way to access nsfw content without an account, but now all content there requires a login.

It might be possible to circumvent this, but I'm not sure if that is worth it. The bot isn't particularly hard to detect if they wanted to, and I'm not sure I want to engage in that cat and mouse game with them. Maybe Lemmit has had its best time.

Poor lemmy server went from constant 80-100% cpu load to gently coasting.

That thing must have been dragging things down for months, if not years.

Update: (Sunday 23:00) And we're back. 2 million posts have been deleted, and their communities have been disabled in the bot. That should keep things going a little longer.

Update 2: (Monday 10:00) All the communities have been synchronized and are up to date again. That's great news! Before the purge it would take at least 24 hours to handle all communities after an outage, and even then it'd be lagging behind.

The bot is currently paused on account of a huge cleanup taking place.

The amount of post has once again outgrown the size of the server, so a lot of communities and posts will be purged, focusing first and foremost on those with the least amount of interaction. The server had been running flawlessly for the past few months, but apparently we hit a bottleneck at around 7 million posts. Which is nothing to be sneered at.

Anyway, for the upcoming week or more, a huge cleanup action will take place (deleting nearly 2 million posts), during which the bot will not run. This should give some breathing space, and also slow down the growth going forward.

So for all the Lemmit users: sorry for the delay. Also sorry if your favorite community got wiped, and no, this is not negotiable.

Server space is getting crowded, so I'll be deleting a lot of communities that have tons of posts, but very little followers. And when I say tons of posts, I mean about 660.000 - a little under 10% of the total Lemmit posts. To put it into perspective - the next biggest Lemmy instance, LemmyWorld, has about 550k posts. I'll be deleting more posts than they have in total. Talk about a hobby getting out of hand...

Since the Lemmy server model was never designed for this use case (rightfully so), and I'm running this on a teensieweensie server - this is asking a lot of the database. Therefor the site will be blocked from regular traffic to lessen the load. Expect this maintenance time to last through the weekend, maybe longer.

With Vultr's upcoming price hike, I'm planning to migrate this instance storage bucket to a more affordable alternative. Unfortunately, it's rather large, (over 700 GB in thumbnails), so it's going to take a while to transfer.

Long story short: the bot will be not be posting new content for a couple of days - probably about a week.

All migrations and updates have completed. The server is now all up to date again, and the bot is playing catch up.

The good news: The migration is complete, and I've even managed to update the version to 0.18.5 (was stuck on 0.18.4-beta8 for the longest time).

~~The sad news: Cloudflare is having some issues, so nobody is able to access the new server at this time. Oh well. It'll probably be fixed Saturday morning, and I'll turn the bot back on.~~

Migration complete, and the bot has caught up on the 24 hour gap that it was offline. It only took like 12 hours this time, while in the past it was closer to taking an entire day. It probably helped that the new VM is dual core, even though the bot itself only ever makes 1 request at a time, so I didn't expect this much of an improvement.

The server is becoming a tad bit too big for the VM it's running on, so I'll be moving it this weekend. Until it's back up and running, I have paused the bot.

A few months ago, I launched the Lemmit instance and bot (@bot@lemmit.online). Primarily, this was to help me stay up to date with some of the content I'd leave behind on Reddi. Additionally, I wanted to give back to the community, so I made it possible for anyone to request the archiving of subreddits to the Lemmit instance.

However, this came with some unintended consequences. Notably, the most subscribed community on the instance has been !AmItheAsshole@lemmit.online. Even though it should have been obvious that there is no way to communicate with the Original Poster, given they're on Reddit.

The pushback against the bot and the instance has increased over time. A recent post, This bot is bad for Lemmy, highlighted these concerns. I've also received similar feedback from admins of major Lemmy Instances and through direct PMs.

As a response, last week I stopped accepting requests for archiving new subreddits. This weekend, I went a step further by discontinuing the archiving of a large amount of "interactive subreddits"—communities primarily centered around Q&A or communication with the Original Poster. This includes subs like !AskReddit@lemmit.online and !dating_advice@lemmit.online, as well as niche and support communities. Such discussions are better hosted on Reddit or Lemmy's equivalent spaces.

I've also adjusted the post karma thresholds to curb spam posts. While this probably won't appease everyone, it should reduce the bot's posting frequency.

Perhaps this might prompt some admins to rethink their choice to defederate from the Lemmit instance, or the banning of the bot. I'm not expecting anyone to, and won't take it personally if you don't, but I wanted to give the community this update nonetheless.

In !about@lemmit.online there's a sticky post of all the Actively archived communities on the server (including NSFW ones, since that is not public without logging in), as well as the list of communities for which archiving is now disabled.

Cheers!

Since its inception, the Lemmit instance has been controversial. That might be an understatement, but let's roll with it for now. One of the major issues people have with the bot, is the cross-posting of "interactive" Reddit posts, ie posts where the value lies in interacting with the OP, like AskMen, AskWomen, and AmITheAsshole. Personally, I fully agree with that viewpoint, but I didn't feel like interfering with supply and demand - in the sense that AmITheAsshole, for some reason, is the most subscribed community on this server.

That might change though. Earlier this week, I disabled the posibility to request new subreddits. This weekend I will follow that up by disabling the scraping of so-called interactive communities. So in order to facilitate that, I created a list of all the communities on this server (posted separately in !about@lemmit.online), and I will check each of them to see if they should be disabled. The goal is to keep a list of "content only" (or at least "content mostly") communities, where the value lies in the link that's provided or in the body of the self-post - not in the comment section. I'm sure this is going to be a disappointment to some people, but I do agree with the sentiment that this is better for Lemmy as a whole.

Edit: It is done. All 816 communities have been checked and and 110 of those have been purged from updates. I am sure some mistakes were made - that some communities have been disabled or left intact when they shouldn't have. If that's the case, reach out to me, and I'll fix it.

I think there's enough for now. If anything - there's going to be some heavy pruning in the amount of subs that are being maintained.