The next milestone release of lemmy is scheduled to include parallel processing per instance for incoming activities, which should resolve the issue.
I don't know what the timeframe on that is though
The next milestone release of lemmy is scheduled to include parallel processing per instance for incoming activities, which should resolve the issue.
I don't know what the timeframe on that is though
https://github.com/LemmyNet/lemmy/pull/4623 is on the 0.19.5 milestone, until parallel sending is implemented there won't be any benefit from parallel receiving.
0.19.4 will already have some improved logic for backgrounding some parts of the receiving logic to speed that up a little, but that won't be enough to deal with this.
stating that it's an issue on our end as our server isn't keeping up
this isn't exactly an issue in your end, unless you consider hosting the server in Australia as your issue. the problem is the latency across the world and lemmy not sending multiple activities simultaneously. there is nothing LW can do about this. as unfortunate as it is, the "best" solution at the time would be moving the server to Europe.
there are still some options besides moving the server entirely though. if you can get the activities to lemmy without as many delays am experience similar to being hosted in Europe can be achieved.
the “best” solution at the time would be moving the server to Europe
I believe the NZ instance basically set up a proxy right next to the LW server, which catches the ActivityPub messages and forwards them in batches to another proxy located in NZ, which translates back into a Lemmy-legal synchronous format. Rather a convoluted process, but it seems to work:

yes, that's about the second best option for the time being.
it's currently used by reddthat.com and lemmy.nz.
disclaimer: i wrote that software.
disclaimer: i wrote that software.
Oh, ha! Sorry for the lemmysplaining!
Is that software available for others to use? I've been busy lately, but Nath messaged me about this... I was traveling and forgot about it by the time I returned.
I'm open to the idea of setting up an EU VPS to proxy LW traffic temporarily. But the best fix is of course for Lemmy to operate in a way that doesn't cause this.
it's open source: https://github.com/Nothing4You/activitypub-federation-queue-batcher
I strongly recommend fully understanding how it works, which failure scenarios there are and how to recover from them before deploying it in production though. not all of this is currently documented, a lot of it has just been in matrix discussions.
I also have a script to prefetch posts and comments from remote communities before they'd get through via federation, which would make them appear without votes at least, and slightly improve processing speed while they're coming in through regular federation. this also doesn't require any additional privileges or being in a position to intercept traffic. it is however also not enough to catch up and stay caught up.
this script is not open source currently. while it's fairly simple and straightforward, i just didn't bother cleaning it up for publishing, as it's currently still partially integrated in an unrelated tool.
I previously tried offering to deploy this on matrix but one of my attempts to open a conversation was rejected and the other one never got accepted.
A side question...
What's gonna happen when this is finally fixed?
Will there be a flood of thousands of posts all at the same time/short period as we 'catch up'?
Kind of bumping this thread, what is the status? Seems like you still have a week delay with LW?
Discussion about the aussie.zone instance itself