Remote
Alun Jones
@pengfold@social.ty-penguin.org.uk
Interested in programming, electronics, science, photography, retrocomputing. Bit of a geek, you could say.
This incarnation of me is self-hosted using GotoSocial.
377 Followers
601 Following
10 Posts
Joined November 03, 2022
Ham radio / Locator:
GW1URF / IO72XJ
Pronouns:
He/Him
Boasting:
It had to happen, eventually. My AI crawler antagoniser, https://www.ty-penguin.org.uk/~auj/spigot/ has been seeing sustained traffic of between 300 and 500 thousand hits per hour. I've not been particularly bothered by that, but a couple of days ago, my provider, @bitfolk@hostux.social, sent me a bandwidth warning: I'm on track to hit 2TBytes of outbound bandwidth this month and end up paying for the excess.
So I've added firewalling - if more than 5% of machines in a /23 network hit spigot within an hour, then the entire network gets a temporary block until it completely stops hitting my server. Hopefully that will cut things back enough to avoid charges.
The thing that amazes me is that the list has already accumulated nearly 10,000 entries. Put another way, I'm already blocking 0.12% of the whole IPV4 address space because it's being used for web crawling.
Replying to
@emf do/will day tickets exist? We suspect we'd love it, but it would be nice to be able to dip our toes in this year so we know what to expect.
Replying to
@emf that's fine, thanks for the quick response. Maybe we'll just go for it next time...
Replying to
@zaire I'm already compressing (and sending "interestingly tweaked" gzip, png and jpeg payloads for their pleasure). It's worth remembering, though, that decompression is typically less CPU-heavy than compression. If I'm doing the compressing and they're doing the decompressing (and they're also able to deploy hundreds of thousands of machines to doing it) then they're always going to be able to exhaust my resources, be they bandwidth or CPU.
Replying to
@darkling @dan @simontatham Years ago, we noticed that someone on our network had changed their MAC address to c0ffeed00d1e (we found out because he put in a support request about how he could no longer get a DHCP lease for his registered machine).
Replying to
@ojensen@hachyderm.io @briankrebs@infosec.exchange my website generates time-limited email addresses. The key for generating them is the time to millisecond accuracy i.e. I'm accumulating 1,000 new addresses every second. About 4 billion since I put this system in place.
Replying to
@seachaint@masto.hackers.town @GigaByte4711@whitespashe.uk @rra@post.lurk.org You piqued my curiosity with this comment. First off, I'll say I think that aesthetic looks really nice, irrespective of whether it's doing a good job.
Out of interest, I had a look at my blog, where I normalise most images to fit inside 1280x1024, and to have whatever quality is needed for them to be under 300kbytes. I've got 469 images and the effective bits per pixel (i.e. total size in bits / total pixels) is 1.9 after JPEG compression.
The magazine says it's using 4 level greyscale images, so that's 2 bits per pixel before PNG compression. They're really not going to need a lot of compression to beat the average across my site. Of course, I could drop the quality further, but I came to the conclusion that 300k was a reasonable compromise between size and visible artifacts.
Replying to
@mike@social.chinwag.org I've got prototype code which will read a Python program, and make a parse tree out of it. You can then ask it (repeatedly) to make random changes to the tree and output the resulting code. The output is valid Python and looks plausibly like the input, but is filled with nasty bugs.
At some point soon, that's getting fed a big bunch of Python and released on an unsuspecting crawler. They seem so very very eager to get hold of source code, so I feel the need to give them some.