Accept Headers

Accept headers are sent by the browser to the server to tell the server what kind of stuff the browser can handle. Servers don’t need an accept header. The accept header is described as optional in HTTP documentation.

If you don’t send my software an accept header you are blocked, banned, and reported.

How can this be? The reason is simple. All major browsers; IE, Chrome, FF, etc., send accept headers. If my software does not receive an accept header then the request is not coming from a browser. Humans use browsers to surf the web. If you are not sending me an accept header you are not human, but a robot.

It’s that simple. I am amazed at how many requests are blocked because there is no accept header. I get nervous and check them from time to time, and every time the request comes from a server farm or China or Russia and I can automatically assume that the lack of accept header is because a robot is hitting my site.

I also reject requests because of a missing http_host header, and when the method is “POST”, a missing http_referer header, or a http_referer header that does not match the website.

So if you are writing any software that needs to hit my site, including rss readers or automatic gets of my spam lists, then you must provide a set of real http headers.

2 Responses to “Accept Headers”

  1. Keith says:

    I have worried about this, but browsers ALWAYS send the accept header. The only assumption is that without the accept header, it is a spammer hitting your site. I checked my panel just now and out of 400 or so spam events, 70 were missing accept headers. I even changed the plugin so it does other checks first because I was getting 90% of he hits as missing accept header, and I wanted to check other tests.

    Keith

  2. Dan says:

    I have been using Abyssguard to protect my websites. It has been blocking hundreds of requests and its reason: “No header Accept provided”. I just wanted to make sure this was legitimately blocking the bad bots. Thanks! BTW, if you want to try out Abyssguard, here is the referral code: 916499246564

Leave a Reply