46
Announcing list2regexp - Generate regular expressions to match lists of strings (codeberg.org)
submitted 4 days ago by trevor@lemmy.ml to c/opensource@lemmy.ml
23 comments fedilink hide all child comments
top 23 comments
sorted by: hot top new old
[-] trevor@lemmy.ml 10 points 4 days ago

Here is an example. Given the following list of strings:

1.2.3.4
127.0.0.1
127.0.0.2
127.0.0.11
128.0.0.1
125.0.0.*
200.0.0.1
2001:0db8:85a3:0000:0000:8a2e:0370:1300
2001:0db8:85a3:0000:0000:8a2e:0370:1330
2001:0db8:85a3:0000:0000:8a2e:0370:1337
あいうえお
あいうえこ
😃😁😆😐
😃😁😆
😃😁🤣

list2regexp will return the following pattern:

^(?:1(?:(?:\.(?:2(?:\.(?:3(?:\.(?:4))))))|(?:2(?:(?:5(?:\.(?:0(?:\.(?:0(?:\.(?:.*)))))))|(?:7(?:\.(?:0(?:\.(?:0(?:\.(?:(?:1(?:|(?:1)))|(?:2))))))))|(?:8(?:\.(?:0(?:\.(?:0(?:\.(?:1)))))))))))|(?:2(?:0(?:0(?:(?:\.(?:0(?:\.(?:0(?:\.(?:1))))))|(?:1(?::(?:0(?:d(?:b(?:8(?::(?:8(?:5(?:a(?:3(?::(?:0(?:0(?:0(?:0(?::(?:0(?:0(?:0(?:0(?::(?:8(?:a(?:2(?:e(?::(?:0(?:3(?:7(?:0(?::(?:1(?:3(?:(?:0(?:0))|(?:3(?:(?:0|7))))))))))))))))))))))))))))))))))))))))))|(?:あ(?:い(?:う(?:え(?:(?:お|こ))))))|(?:😃(?:😁(?:(?:😆(?:|(?:😐)))|(?:🤣))))$
[-] rbn@sopuli.xyz 18 points 4 days ago 
.*

... should do the trick.

[-] leanleft@lemmy.ml 3 points 3 days ago* (last edited 3 days ago)

it should(could) have newlines for readability. and is genererally an underappreciated feature that leads to people thinking that regex is crazy and unreadable.

[-] trevor@lemmy.ml 2 points 3 days ago* (last edited 3 days ago)

I didn't know about this so I tried testing it out. Yes, this is possible, but requires enabling a specific flag (x when it comes to regexp2) assuming the engine even supports this feature.

[-] leanleft@lemmy.ml 3 points 3 days ago* (last edited 3 days ago)

also comments. which can be very simple..like metadata about boundary,membership,grouping, or hierarchy.
its supported in most languages except javascript and plain grep.
but it doesnt serve a programmatic use.. so even if its a hella useful single char or digit, that massively boosts readability.. some people would argue that it's extra/junk

[-] helix@feddit.org 6 points 4 days ago

Why? 🤔

[-] trevor@lemmy.ml 5 points 4 days ago

In my case, to efficiently match against a large number of IP addresses and IP address ranges:

https://codeberg.org/tslocum/sriracha/src/commit/5bf24b0daabd058c2714427da17c7ea16b7cd43a/internal/server/server.go#L447

[-] helix@feddit.org 3 points 4 days ago

It's called "allowlist" and "denylist" nowadays as it's easier to understand for non-native speakers and not racist.

I feel like there's an easier and faster way to match IP addresses than regex, e.g. the search tree apparia suggested. At least use bytes to match 😄

[-] trevor@lemmy.ml 1 points 3 days ago* (last edited 3 days ago)

The term's connection with racism, as well as the value in avoiding its use has been disputed.

https://en.wikipedia.org/wiki/Blacklist_%28computing%29#Controversy_over_terminology

(See the section that follows for more context.)

[-] helix@feddit.org 2 points 3 days ago

Yes, but does it really hurt you to make your software more accessible?

What good arguments do you have for the term blacklist?

"Denylist" is easier to understand and doesn't spark discussions. Nobody will be offended, whether it's technically correct to be offended or not.

I'm not a person of colour but your answer still insults me, because it's so tone deaf.

[-] apparia@discuss.tchncs.de 5 points 4 days ago

Holy overengineering, Batman. IP addresses of all things are trivially parseable into fixed-length byte sequences. Just do that and use a search tree. Or, IDK; this is an extremely solved problem, there are probably even better solutions out there -- but I promise you text regex is involved in none of the good ones.

Even for actual string matching, any decent regex engine will compile and optimise an alternation a|b|c|... better than this text wrangling will. Unless you have actual benchmarks showing this gives you an advantage with the engine you're using, you're just introducing complexity for no gain.

[-] MadPsyentist 3 points 3 days ago

Also of note. Regexlicensing.org has issued no licences to date for use of regex in production environments

https://regexlicensing.org/license/

[-] trevor@lemmy.ml 2 points 3 days ago

Not much substance to their argument, mostly just whining. Funny website, though.

[-] Courantdair@jlai.lu 1 points 4 days ago

I think the list of strings with pipes between each element would be shorter than this huge unreadable blob

[-] trevor@lemmy.ml 4 points 4 days ago

Being pretty or short is not a goal of this project, nor do I see the value in such goals.

[-] helix@feddit.org 2 points 4 days ago

Pretty and short probably also have better performance.

Did you manually come up with a prettier regex to compare against?

[-] thingsiplay@lemmy.ml 3 points 4 days ago

Is the goal to match ONLY the given haystack? What would be the use case? Because usually for dynamic usage you need the opposite site, find matches not the pattern. This is basically a pattern finder. Don't get me wrong, from technical level this is impressive and interesting. But I don't see a use case right now.

Well I could imagine a scenario to create test cases for your software that uses regex. But the resulting pattern is unreadable and probably not very efficient too. I don't have a better solution to this, so don't take my reply as a poison; I just analyze with interest.

[-] GenderNeutralBro@lemmy.sdf.org 3 points 4 days ago

I have lots of places I can use regex to match strings, but otherwise have no easy way to say "match any in list".

Is it stupid? Sure. But I can't fix the umpteen web consoles I manage; I have to work with them as they are.

[-] HotChickenFeet@sopuli.xyz 3 points 3 days ago* (last edited 3 days ago)

(?: strings|stringb|stringc)

Is valid regex given a list of strings to match, no?

Is the advantage here some performance gain or potential to use fewer characters depending on the list?

Edit: I see elsewhere it's specifically so you can match against a long list of specific IPs - so maybe a bit of both.

[-] thingsiplay@lemmy.ml 2 points 4 days ago

I see. Even if I have no use case for it personally, I want you to thank you for sharing. Its probably a niche use case, but always interesting to see what problems others try to solve and how they did it. The code is actually smaller than I expected.

[-] arran4@aussie.zone 1 points 4 days ago

Does it do exclusions? Can it merge regexp?

[-] trevor@lemmy.ml 1 points 4 days ago* (last edited 4 days ago)

Those would be interesting features, but it currently only operates on plain text lists, or lists with wildcards (*) which are converted into regexp (.*), or lists with CIDR suffixes (127.0.0.0/24). You can disable escaping, but list2regexp doesn't understand the individual segments of regular expressions, so the result will probably be wrong.

With that said, the library being used to handle escaping and testing the generated patterns (regexp2) does expose the syntax tree so the information is available to do interesting things with, such as what you suggest.

[-] arran4@aussie.zone 1 points 4 days ago

Merging probably has more value as people would understand the edge cases, with generated regexp it would be harder for someone to confidently understand the edges without having a deeper understanding of the algorithm and data set.

this post was submitted on 17 Sep 2026
46 points (97.9% liked)

Open Source

49130 readers
977 users here now

All about open source! Feel free to ask questions, and share news, and interesting stuff!

Useful Links

Rules

Related Communities

Community icon from opensource.org, but we are not affiliated with them.

founded 7 years ago
MODERATORS
Cloak@lemmy.ml
kevincox@lemmy.ml
CrypticCoffee@lemmy.ml
Lettuceeatlettuce@lemmy.ml