I have made some editorial adjustments to this post after Jay Allen - the author of MT Blacklist - showed up in the comments and corrected me on some points. Thanks Jay.

I mentioned in a comment to Helen's post about MT Blacklist that I couldn't see just what Blacklist was up to because I could no longer view the Activity Log. And the reason for that is that there were over 500,000 entries in the log, which is kept in a SQL table.*

So I dumped the table and cleared it out and restarted everything and found that it looked something like this:

Date	IP Address	Log Entry
2004.12.22 04:24:03	212.87.234.1	MT-Blacklist comment denial on Hold The Mayo: discounts.8bit.co.uk
2004.12.22 04:24:09	212.87.234.1	MT-Blacklist comment denial on trying to grok: discounts.8bit.co.uk
2004.12.22 04:24:30	212.87.234.1	MT-Blacklist comment denial on Ambient Irony: discounts.8bit.co.uk
2004.12.22 04:25:22	212.87.234.1	MT-Blacklist comment denial on Hold The Mayo: discounts.8bit.co.uk
2004.12.22 04:25:37	212.87.234.1	MT-Blacklist comment denial on Hold The Mayo: discounts.8bit.co.uk
2004.12.22 04:25:49	212.87.234.1	MT-Blacklist comment denial on angelweave: discounts.8bit.co.uk
2004.12.22 04:26:05	212.87.234.1	MT-Blacklist comment denial on Read My Lips - the blog: discounts.8bit.co.uk
2004.12.22 04:26:17	212.87.234.1	MT-Blacklist comment denial on Read My Lips - the blog: bmw.com
2004.12.22 04:26:51	212.87.234.1	MT-Blacklist comment denial on Hoppings of Roxette Bunny: discounts.8bit.co.uk
2004.12.22 04:26:52	212.87.234.1	MT-Blacklist comment denial on trying to grok: discounts.8bit.co.uk
2004.12.22 04:27:54	212.87.234.1	MT-Blacklist comment denial on Rocket Jones: bmw.com
2004.12.22 04:27:56	212.87.234.1	MT-Blacklist comment denial on Publius & Co.: discounts.8bit.co.uk
2004.12.22 04:28:04	212.87.234.1	MT-Blacklist comment denial on Rocket Jones: discounts.8bit.co.uk
2004.12.22 04:28:10	212.87.234.1	MT-Blacklist comment denial on The Nap Room: discounts.8bit.co.uk
2004.12.22 04:28:40	212.87.234.1	MT-Blacklist comment denial on Mookie Riffic: discounts.8bit.co.uk
2004.12.22 04:28:40	212.87.234.1	MT-Blacklist comment denial on Being American in T.O.: discounts.8bit.co.uk
2004.12.22 04:28:52	212.87.234.1	MT-Blacklist comment denial on trying to grok: discounts.8bit.co.uk
2004.12.22 04:28:55	212.87.234.1	MT-Blacklist comment denial on Semi-Intelligent Thoughts: comeback.com
2004.12.22 04:28:58	212.87.234.1	MT-Blacklist comment denial on trying to grok: discounts.8bit.co.uk
2004.12.22 04:29:02	212.87.234.1	MT-Blacklist comment denial on Semi-Intelligent Thoughts: discounts.8bit.co.uk
2004.12.22 04:29:02	212.87.234.1	MT-Blacklist comment denial on Semi-Intelligent Thoughts: comeback.com
2004.12.22 04:29:05	212.87.234.1	MT-Blacklist comment denial on Hold The Mayo: bmw.com
2004.12.22 04:29:13	212.87.234.1	MT-Blacklist comment denial on Semi-Intelligent Thoughts: discounts.8bit.co.uk
2004.12.22 04:29:15	212.87.234.1	MT-Blacklist comment denial on Read My Lips - the blog: discounts.8bit.co.uk
2004.12.22 04:29:47	212.87.234.1	MT-Blacklist comment denial on trying to grok: discounts.8bit.co.uk
2004.12.22 04:29:53	212.87.234.1	MT-Blacklist comment denial on Hoppings of Roxette Bunny: discounts.8bit.co.uk
2004.12.22 04:30:01	212.87.234.1	MT-Blacklist comment denial on Rocket Jones: discounts.8bit.co.uk
2004.12.22 04:30:23	212.87.234.1	MT-Blacklist comment denial on Rocket Jones: discounts.8bit.co.uk
2004.12.22 04:30:25	202.57.35.130	MT-Blacklist comment denial on Snooze Button Dreams: hermosa
2004.12.22 04:30:31	212.87.234.1	MT-Blacklist comment denial on Publius & Co.: discounts.8bit.co.uk
2004.12.22 04:30:47	212.87.234.1	MT-Blacklist comment denial on Being American in T.O.: discounts.8bit.co.uk
2004.12.22 04:30:49	212.87.234.1	MT-Blacklist comment denial on Semi-Intelligent Thoughts: discounts.8bit.co.uk
2004.12.22 04:31:05	212.87.234.1	MT-Blacklist comment denial on angelweave: comeback.com
2004.12.22 04:31:14	212.87.234.1	MT-Blacklist comment denial on The Nap Room: discounts.8bit.co.uk
2004.12.22 04:31:15	212.87.234.1	MT-Blacklist comment denial on Civilization Calls: discounts.8bit.co.uk
2004.12.22 04:31:17	212.87.234.1	MT-Blacklist comment denial on Read My Lips - the blog: discounts.8bit.co.uk
2004.12.22 04:31:26	212.87.234.1	MT-Blacklist comment denial on Rocket Jones: bmw.com
2004.12.22 04:32:07	212.87.234.1	MT-Blacklist comment denial on Rocket Jones: discounts.8bit.co.uk
2004.12.22 04:32:11	212.87.234.1	MT-Blacklist comment denial on Rocket Jones: comeback.com
2004.12.22 04:32:13	212.87.234.1	MT-Blacklist comment denial on Rocket Jones: discounts.8bit.co.uk
2004.12.22 04:32:28	212.87.234.1	MT-Blacklist comment denial on Rocket Jones: discounts.8bit.co.uk
2004.12.22 04:32:33	212.87.234.1	MT-Blacklist comment denial on Frank J. Fan Club: discounts.8bit.co.uk
2004.12.22 04:32:34	68.93.224.4	Search: query for 'poltergeist'
2004.12.22 04:32:40	212.87.234.1	MT-Blacklist comment denial on Mookie Riffic: discounts.8bit.co.uk
2004.12.22 04:32:40	212.87.234.1	MT-Blacklist comment denial on Mookie Riffic: discounts.8bit.co.uk
2004.12.22 04:32:46	212.87.234.1	MT-Blacklist comment denial on Rocket Jones: discounts.8bit.co.uk
2004.12.22 04:33:14	212.87.234.1	MT-Blacklist comment denial on Mookie Riffic: discounts.8bit.co.uk
2004.12.22 04:33:47	212.87.234.1	MT-Blacklist comment denial on Rocket Jones: discounts.8bit.co.uk
2004.12.22 04:33:49	212.87.234.1	MT-Blacklist comment denial on The Nap Room: discounts.8bit.co.uk
2004.12.22 04:33:49	212.87.234.1	MT-Blacklist comment denial on Auterrific: discounts.8bit.co.uk
2004.12.22 04:33:49	68.93.224.4	Search: query for 'poltergeist'
2004.12.22 04:34:08	212.87.234.1	MT-Blacklist comment denial on angelweave: comeback.com
2004.12.22 04:34:19	212.87.234.1	MT-Blacklist comment denial on Rocket Jones: discounts.8bit.co.uk
2004.12.22 04:34:28	212.87.234.1	MT-Blacklist comment denial on angelweave: discounts.8bit.co.uk
2004.12.22 04:35:44	212.87.234.1	MT-Blacklist comment denial on Semi-Intelligent Thoughts: discounts.8bit.co.uk
2004.12.22 04:36:04	212.87.234.1	MT-Blacklist comment denial on Hoppings of Roxette Bunny: discounts.8bit.co.uk
2004.12.22 04:36:11	212.87.234.1	MT-Blacklist comment denial on Semi-Intelligent Thoughts: discounts.8bit.co.uk
2004.12.22 04:36:13	212.87.234.1	MT-Blacklist comment denial on Semi-Intelligent Thoughts: discounts.8bit.co.uk
2004.12.22 04:36:18	212.87.234.1	MT-Blacklist comment denial on Being American in T.O.: comeback.com
2004.12.22 04:36:24	212.87.234.1	MT-Blacklist comment denial on Semi-Intelligent Thoughts: discounts.8bit.co.uk
2004.12.22 04:36:33	212.87.234.1	MT-Blacklist comment denial on Rocket Jones: discounts.8bit.co.uk
2004.12.22 04:36:36	212.87.234.1	MT-Blacklist comment denial on Snooze Button Dreams: discounts.8bit.co.uk
2004.12.22 04:36:51	212.87.234.1	MT-Blacklist comment denial on Ramblings' Journal: discounts.8bit.co.uk
2004.12.22 04:37:02	212.87.234.1	MT-Blacklist comment denial on trying to grok: bmw.com
2004.12.22 04:37:40	212.87.234.1	MT-Blacklist comment denial on Read My Lips - the blog: discounts.8bit.co.uk
2004.12.22 04:37:44	212.87.234.1	MT-Blacklist comment denial on Read My Lips - the blog: discounts.8bit.co.uk
2004.12.22 04:37:44	212.87.234.1	MT-Blacklist comment denial on The Nap Room: discounts.8bit.co.uk
2004.12.22 04:37:52	212.87.234.1	MT-Blacklist comment denial on Read My Lips - the blog: discounts.8bit.co.uk
2004.12.22 04:38:41	212.87.234.1	MT-Blacklist comment denial on Civilization Calls: discounts.8bit.co.uk
2004.12.22 04:38:43	212.87.234.1	MT-Blacklist comment denial on Rocket Jones: discounts.8bit.co.uk
2004.12.22 04:39:17	24.1.70.204	'JohnL' added entry #60439
2004.12.22 04:39:30	212.87.234.1	MT-Blacklist comment denial on Hoppings of Roxette Bunny: discounts.8bit.co.uk
2004.12.22 04:39:42	212.87.234.1	MT-Blacklist comment denial on Read My Lips - the blog: discounts.8bit.co.uk
2004.12.22 04:39:53	212.87.234.1	MT-Blacklist comment denial on angelweave: discounts.8bit.co.uk
2004.12.22 04:39:53	212.87.234.1	MT-Blacklist comment denial on trying to grok: discounts.8bit.co.uk
2004.12.22 04:40:06	212.87.234.1	MT-Blacklist comment denial on The Nap Room: discounts.8bit.co.uk
2004.12.22 04:40:15	212.87.234.1	MT-Blacklist comment denial on angelweave: discounts.8bit.co.uk
2004.12.22 04:40:27	212.87.234.1	MT-Blacklist comment denial on Rocket Jones: discounts.8bit.co.uk
2004.12.22 04:40:27	212.87.234.1	MT-Blacklist comment denial on Hold The Mayo: discounts.8bit.co.uk
2004.12.22 04:40:36	212.87.234.1	MT-Blacklist comment denial on Publius & Co.: discounts.8bit.co.uk
2004.12.22 04:40:42	212.87.234.1	MT-Blacklist comment denial on Rocket Jones: discounts.8bit.co.uk
2004.12.22 04:40:45	68.238.27.186	User 'Physics Geek' added 2 entries to the MT-Blacklist
2004.12.22 04:40:59	212.87.234.1	MT-Blacklist comment denial on The Nap Room: discounts.8bit.co.uk
2004.12.22 04:41:26	212.87.234.1	MT-Blacklist comment denial on Civilization Calls: discounts.8bit.co.uk
2004.12.22 04:41:40	212.87.234.1	MT-Blacklist comment denial on angelweave: discounts.8bit.co.uk
2004.12.22 04:42:00	212.87.234.1	MT-Blacklist comment denial on Mookie Riffic: discounts.8bit.co.uk
2004.12.22 04:42:44	212.87.234.1	MT-Blacklist comment denial on The Nap Room: discounts.8bit.co.uk
2004.12.22 04:42:45	212.87.234.1	MT-Blacklist comment denial on Rocket Jones Skunkworks: discounts.8bit.co.uk
2004.12.22 04:42:49	212.87.234.1	MT-Blacklist comment denial on Hoppings of Roxette Bunny: discounts.8bit.co.uk
2004.12.22 04:43:16	212.87.234.1	MT-Blacklist comment denial on Madfish Willie's Cyber Saloon: discounts.8bit.co.uk
2004.12.22 04:43:17	212.87.234.1	MT-Blacklist comment denial on angelweave: discounts.8bit.co.uk
2004.12.22 04:43:38	212.87.234.1	MT-Blacklist comment denial on angelweave: discounts.8bit.co.uk
2004.12.22 04:43:50	212.87.234.1	MT-Blacklist comment denial on Semi-Intelligent Thoughts: discounts.8bit.co.uk
2004.12.22 04:43:53	212.87.234.1	MT-Blacklist comment denial on Practical Penumbra: discounts.8bit.co.uk
2004.12.22 04:43:56	212.87.234.1	MT-Blacklist comment denial on The Nap Room: comeback.com
2004.12.22 04:44:37	212.87.234.1	MT-Blacklist comment denial on Ilyka Damen: discounts.8bit.co.uk
2004.12.22 04:45:09	68.93.224.4	Ping 'http://blog.mu.nu/cgi/mt-tb.cgi/260' failed: HTTP error: 500 read timeout
2004.12.22 04:45:38	212.87.234.1	MT-Blacklist comment denial on trying to grok: discounts.8bit.co.uk
2004.12.22 04:45:53	212.87.234.1	MT-Blacklist comment denial on Semi-Intelligent Thoughts: discounts.8bit.co.uk
2004.12.22 04:46:05	212.87.234.1	MT-Blacklist comment denial on Rocket Jones: discounts.8bit.co.uk
2004.12.22 04:46:14	212.87.234.1	MT-Blacklist comment denial on Rocket Jones: discounts.8bit.co.uk
2004.12.22 04:46:18	212.87.234.1	MT-Blacklist comment denial on Mookie Riffic: discounts.8bit.co.uk
2004.12.22 04:46:33	212.87.234.1	MT-Blacklist comment denial on Read My Lips - the blog: discounts.8bit.co.uk
2004.12.22 04:46:42	212.87.234.1	MT-Blacklist comment denial on Mookie Riffic: discounts.8bit.co.uk
2004.12.22 04:46:53	212.87.234.1	MT-Blacklist comment denial on The Nap Room: discounts.8bit.co.uk
2004.12.22 04:46:59	212.87.234.1	MT-Blacklist comment denial on Read My Lips - the blog: discounts.8bit.co.uk
2004.12.22 04:47:03	212.87.234.1	MT-Blacklist comment denial on trying to grok: discounts.8bit.co.uk
2004.12.22 04:47:12	212.87.234.1	MT-Blacklist comment denial on Hold The Mayo: discounts.8bit.co.uk
2004.12.22 04:47:17	212.87.234.1	MT-Blacklist comment denial on Ace of Spades HQ: discounts.8bit.co.uk
2004.12.22 04:49:26	134.159.132.4	'breanagh' added entry #60440
2004.12.22 04:56:46	68.217.68.204	'Queenie' added entry #60441

[Jay Allen, the author of MT Blacklist, has questioned me on this in the comments. I should explain that the problem the server suffered today was not a question of CPU utilisation but of memory - with 30 comments being handled at the same time, the system did not have enough memory to cope, and was constantly swapping tasks to and from disk, a condition known as "thrashing". This is a bad thing.]

For those who speak Linux, the load average was up around 70 during all of this. Which made it less than perfectly responsive when I was logged in and trying to fix it.

The point where the spam stops dead is where I got smart and simply added a .htaccess file that completely blocks access to the CGI directory for that particular spammer.

Meanwhile, Movable Type have provided a fix this problem - with the release of MT 3.14. Which means I really need to get moving on the migration.

* This is a pretty silly place to keep a log file, but what the heck.

Posted by: Pixy Misa at 05:05 AM | Comments (47) | Add Comment | Trackbacks (Suck)

1 "This is necessary because even when MT Blacklist blocks a comment, it still uses a lot of resources on the server, and when you're getting flooded with crap, the server basically grinds to a halt."

Actually, when MT-Blacklist blocks a post, it uses very little resources (although more than an Apache-level solution) and happens very early on in the process (before mt-comments.cgi even GETS the comment). The problem in 3.121 and earlier is that comments sent to moderation use more resources than they should. This, as you said, is remedied in v3.14.

Of course, regardless of whether your comment form is hooked to Movable Type or a simple formmail script, if you're getting 500,000 requests to your site in a short amount of time, you're going to buckle under the load. Low-cost shared servers weren't intended for that sort of load and there is precious little that Movable Type can do about that. That sort of thing is best handled by the Apache config[1].

I would also like to point out that neither Movable Type (a software program) nor Six Apart (the company who makes it) were sleeping. In fact, we were hard at work on making your software better. I only WISH that I could get more sleep...

[1]: http://it.slashdot.org/comments.pl?sid=133226&cid=11127930

Posted by: Jay Allen @ at December 22, 2004 08:30 AM

2 I would also like to point out that neither Movable Type (a software program) nor Six Apart (the company who makes it) were sleeping. In fact, we were hard at work on making your software better. I only WISH that I could get more sleep...

I apologise for that. I didn't mean it that way, and as a programmer myself I understand and sympathise with you on that semi-mythical stuff called sleep.

Posted by: Pixy Misa @ at December 22, 2004 08:58 AM

3 Would the installation of something like James Seng's SCode be of any use? I've gotten rather handy with that plugin of late, and it seems to work nicely with MTBlacklist...please let me know if I can be of any help in anti-spam warfare. It's a grudge thing with me.

Posted by: Queenie @ at December 23, 2004 07:49 PM

4 Napoli, Ian Kinsler Louis Vuitton and Nelson Cruz all Cheap Louis Vuitton homered for Texas, Replica Handbags which played its season Louis Vuitton Wallets opener exactly five months Louis Vuitton Cruise SS 2011 after a Game 5 loss to Louis Vuitton Spring Summer 2011 San Francisco at home ended Louis Vuitton Fall Winter 2010 its first World Series.Darren Oliver,Louis Vuitton Prefall 2010 the third Texas reliever,Louis Vuitton Spring Summer 2010 got the victory even after Louis Vuitton Monogram Empreinte allowing a homer to David Louis Vuitton Antheia Ortiz in the top of the Louis Vuitton New Collection eighth that tied it at Louis Vuitton Monogram Canvas 5.Murphy pinch-hit for No. 9 Louis Vuitton Monogram Idylle batter Julio Borbon, Louis Vuitton Monogram Macassar Canvas the center fielder whose Louis Vuitton Monogram Multicolore two-base error on the first Louis Vuitton Monogram Vernis play of the game led to the Red Louis Vuitton Monogram Miroir Sox scoring a pair of Louis Vuitton Monogram Mini Lin unearned runs. Borbon remained Louis Vuitton Monogram Denim a starter even after five errors Louis VuittonLouis Vuitton Cosmic Blossom in 21 spring training games. from:http://www.go2handbags.com
http://www.go2handbags.com

Posted by: caroline @ at April 02, 2011 09:07 AM

Hide Comments | Add Comment