I've decided that Google is simply not an "organic" search engine, but an ad company ... solely, unequivocally — and a very greedy one at that. For this reason, I am blocking their spiders with scripting of my own design.
They have a propensity for bypassing the usual identifications with their crawlers and I have both anticipated and coded up front for this as well.
In an attempt to be fair, I am not blocking search engine results solely because they come from google. If there is a hit the fact that it comes from google will not cause that hit to be forbidden.
I decline to be the victim of a monopoly any longer. I don't shop at Amazon for the same reason. The rest of you can do whatever you like. They can index the business card. They can index the hobby site. The have to stay off the blog however and I'm not going to mess with them anymore about it.
If they get too pushy I still have a set of IP firewall rules already written to keep them off my node entirely. When I think of how many stupid seo "professionals" want to push sites to google it just makes me want them around me even less.