this post was submitted on 31 Mar 2026
32 points (97.1% liked)

Selfhosted

61512 readers
709 users here now

A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.

Rules:

Detailed Rules Post

  1. Be civil.

  2. No spam.

  3. Posts are to be related to self-hosting.

  4. Don't duplicate the full text of your blog or readme if you're providing a link.

  5. Submission headline should match the article title.

  6. No trolling.

  7. Promotion posts require active participation, with an account that is at least 30 days old. F/LOSS without a paywall has exceptions, with requirements. See the rules link for details. Tags [CBH] or [AIP] are required, see the links in Rule 8 for details.

  8. AI-related discussions and AI-involved promotional posts have additional requirements for tagging, as noted in Rule 7 and the AI & Promotional Post Expanded Rules post, and find example disclosures here.

Resources:

Any issues on the community? Report it using the report flag.

Questions? DM the mods!

founded 3 years ago
MODERATORS
 

This may sound like a weird thing to do, but I realised that many crawlers and bots are somehow still able to get past my Anubis. I presume they have gotten smarter and are capable of using JavaScript.

To counter this, I want to link my Anubis to an Iocane setup such that:

Internet > nginx reverse proxy > Anubis > Iocane > my site/app

My hope is that two different filtering mechanisms (one of which will actively poison and waste the bot's resourced) will protect my system better.

I thought I'd ask before actually trying out something like this.

top 12 comments
sorted by: hot top controversial new old
[–] tal@lemmy.today 17 points 4 months ago

Context:

https://en.wikipedia.org/wiki/Anubis_(software)

Anubis is an open source software program that adds a proof of work challenge to websites before users can access them in order to deter web scraping. It has been adopted mainly by Git forges and free and open-source software projects.[4][5]

https://lib.rs/crates/iocaine

Iocaine is a defense mechanism against unwanted scrapers, sitting between upstream resources and the fronting reverse proxy.

[–] db0@lemmy.dbzer0.com 15 points 4 months ago (1 children)

Iocaine expects you know how to detect it the bots, if they can get past anubis do you have another detection process?

[–] algernon@lemmy.ml 1 points 3 months ago (1 children)

I'm very late to the party, but: no, iocaine does not expect you to detect the bots. It used to, but it does its own detection for quite a while now (you can replace the detection mechanism, though).

[–] db0@lemmy.dbzer0.com 2 points 3 months ago (1 children)

What do you mean? Where does it do it's own detection?

[–] algernon@lemmy.ml 1 points 3 months ago (1 children)

Around here. In the default configuration, it is using the built-in handler. The script can be replaced with something like Nam-Shub of Enki (used by pretty much everything I host, and by Codeberg too, for example).

[–] db0@lemmy.dbzer0.com 1 points 3 months ago (1 children)

Ah that wasn't there when I deployed it.

[–] algernon@lemmy.ml 1 points 3 months ago

Scriptability has been a thing since 2.2.0, released on 2025-06-16, but the built-in script appeared in 3.0 (2025-11-14).

[–] ResistingArrest@lemmy.zip 11 points 4 months ago

This may sound like a weird thing to do, but I realised that many crawlers and golems are somehow still able to get past my Anubis. I presume they have gotten smarter and are capable of using Runes.

To counter this, I want to ethereally link my Anubis to an locane setup such that:

Ether > mage tower > Anubis >Iocane > my orb

My hope is that two different filtering mechanisms one of which will actively poison and waste the golem’s energy) will protect my realm better.

I thought I'd ask before actually trying out something like this.

[–] Black616Angel@discuss.tchncs.de 10 points 4 months ago

Have you tried fucking with the status codes?

There is a great defcon talk about that:

Slides

Video on Youtube

So you could e.g. return a 401 and still show the page. Most automated systems will probably ignore the response of an 'unauthorized' message.

[–] ki9@lemmy.gf4.pw 5 points 4 months ago

Idk why you'd keep Anubis, just give the bots unrestricted access to the poison.

[–] Fizz 4 points 4 months ago (1 children)

Does anubis actually catch anything?

[–] nemecle@jlai.lu 1 points 4 months ago

Yes, bots are starting to get around it so you need to keep it up to date but it turned two of my services from inaccessible to users to usable (not just for a few hours, been running it for months)