Archive for November, 2014

htaccess master list rebuilt and plugin progress.

Tuesday, November 4th, 2014

I started the bad neighborhood concept a few months ago. Basically it’s one strike and you’re out. If I receive spam, a threat, WGET, or an over-active spider at any of my sites, I block the IP immediately.

I have a new plugin tentatively titled WP Protection that does this. This is my replacement for the Stop Spammer plugin. I’ve ripped out the various tests from Stop Spammers and put them into a more Object-Oriented framework. It can either block the spam, or update the htaccess file with a deny, or both.

I have a semi-automated step that does an inquiry at lacnic.net for each blocked IP and returns the IP range for the offending IP. This is called a CIDR. If a server company tolerates a spammer, I can block the whole server company CIDR. This is extremely aggressive and I wound up blocking most of China, Russia, and all of Vietnam. It is necessary to block the CIDR as most residential ISP addresses are dynamically allocated and a user gets a new IP each time they log in. Blocking individual IP addresses is like a game of Whack-A-Mole.

I’ve made some mistakes doing this and the master list wound up a little scrambled. I deleted it yesterday, and overnight I rebuilt it based on the StopForumSpam.com 7 day list of spammers. It lists more than 4,000 bad neighborhoods. I rewrote the compression program that combines contiguous spaces into one CIDR. It was the source of scrambling. It seems to work well now.

A byproduct of the list is White-List of ranges. These are almost all North American and Western European ISPs. It also white-lists search engines and Amazon AWS. Amazon is a major source of spammers who sign up for a free trial and send out spam for a few hours until Amazon blocks them. However, it is fruitless to permanently block Amazon because they shut down the spammers and blocking an Amazon CIDR winds up blocking some services that I count on, like RSS sharing services and other beneficial robots. I white-list CloudFlare, PayPal and a bunch of other services. I do block Digital Ocean, though. I get too much spam from them, because it is so easy to do a test drive on their system for free, and spammers know this.

I am concerned that the white list is American and European biased where the black list is biased against Russia, China and third world nations. I ran a program that automatically notified service providers whenever I received a spam. With the exception up a few Spam hosts in Canada and the US, Western hemisphere providers promptly addressed the problem whereas Easter hemisphere providers ignored me. If an ISP can convince me that it has cleaned up its act, I will gladly white-list them. I have been searching for lists of residential ISP IP ranges. Hosting companies, however, should never hit my site. I don’t want to white-list hosting companies unless they can convince me that they keep a clean house. I recently received a complaint from someone in Thailand that they were blocked from my sites. Unfortunately, Thailand is one of the worst of the spammer nations, and I have received thousands of spam attempts from IPs in the same CIDR as the person complaining. I feel justified in blocking the whole range.

The WP Protection plugin will not be shared via the WordPress Repository. I have learned my lesson. I cannot make money on WordPress. Neither can I find time to support all the problems users have from installing an aggressive anti-spam program. I have opened an account on ClickBank.com and I will try to sell the plugin there. It will be better for me if I support 50 paid users rather than 150,000 unpaid users.

WP Protection is coming along. I don’t have the settings page yet, so I can’t release it for beta. I will let it go free to a few users when I think it is ready for testing.