Meta

630 readers

1 users here now

Discussion about the aussie.zone instance itself

founded 1 year ago

MODERATORS

admin@aussie.zone

lodion@aussie.zone

Ongoing federation issues (aussie.zone)

submitted 10 months ago* (last edited 10 months ago) by Zagorath@aussie.zone to c/meta@aussie.zone

25 comments fedilink hide all child comments

I'm still seeing cases where comments don't seem to be federating correctly. For example:

Other instance	aussie.zone
https://lemmy.world/comment/6786498	https://aussie.zone/comment/5886332
https://feddit.uk/post/6819925	https://aussie.zone/post/6020867
https://lemmy.ml/comment/7476880	https://aussie.zone/comment/5881947
https://lemmy.blahaj.zone/comment/5759107	https://aussie.zone/comment/5888459
https://midwest.social/comment/6313276	https://aussie.zone/comment/5889962

I'm not sure what the cause is, but the last few days it seems more of my comments are failing to federate than are succeeding.

you are viewing a single comment's thread
view the rest of the comments

[–] lodion@aussie.zone 5 points 10 months ago (1 children)

Ok, something is busted with the lemmy API endpoint that shows current federation state. It is currently showing nearly all remote instances as dead:

But "dead" instances are still successfully receiving content from AZ, and sending back to us.

[–] lodion@aussie.zone 5 points 10 months ago (2 children)

Ok, there's more to this than I first thought. It seems there is a back end task set to run at a set time every day, if the instance is restarting at that time the task doesn't run... this task updates the instances table to show remote instances as "seen" by AZ. With the memory leaks in 0.19.1, the instance has been restarting when this task is running... leading to this situation.

I've updated the server restart cronjob to not run around the time this task runs... and I've again manually updated the DB to flag all known instances as alive rather than dead.

Will keep an eye on it some more...

For anyone curious, two of the bugs that are related to this:
https://github.com/LemmyNet/lemmy/issues/4288
https://github.com/LemmyNet/lemmy/issues/4039

[–] Zagorath@aussie.zone 2 points 10 months ago

Thanks for the update!

[–] Zagorath@aussie.zone 1 points 10 months ago (1 children)

Looks like what caused this might have been resolved? https://github.com/LemmyNet/lemmy/pull/4377

[–] lodion@aussie.zone 2 points 10 months ago (1 children)

Excellent, hopefully that is included in 0.19.3 that came out overnight... will upgrade shortly.

[–] Zagorath@aussie.zone 1 points 10 months ago (1 children)

It is. I found the page I linked from the release notes.

[–] lodion@aussie.zone 2 points 10 months ago (2 children)

Cool, we're on 0.19.3 now.

[–] Zagorath@aussie.zone 1 points 10 months ago

Sweet

[–] maniacalmanicmania@aussie.zone 1 points 9 months ago (1 children)

How do things look after the past week?

[–] lodion@aussie.zone 2 points 9 months ago

0.19.3 still has the memory leak, I've left Lemmy to restart once a day to mitigate this. Federation seems to be fine for the last week after some minor changes and the upgrade.