So much for the Asimov Seldonian approach to bot behavior algorithms on the web. We already have a number of bots that come onto the network of another and act as though they own it.
Lets see, I don't want to mention any names ... googlebot.
A "Seldonian algorithm" is a conceptual term coined by one Hari Seldon, the Isaac Asimov science fiction Foundation series character who once proclaimed three laws of robotics:
•
A robot may not injure a human being or,
•
through inaction, allow a human being to come to harm
•
A robot must obey the orders given it by human beings except where such orders would conflict with the First Law
Now a badly behaved bot will pretty much ignore the directives contained in robots.txt ... and this is googlebot up one side and down the other. They seem to observe disallow / pretty much up to a point.
But the fact of the matter is that they are not the primadonnas they believe themselves to be and it is well within my ability to deliver something particularly nasty retrograde were I who they think they are.
I never ceased to be amazed by the little sycophant seo so-called "professionals" touting how 'google knows the best way to walk you site' in that effort to appear to be something they're not: technically proficient.
Then, googlebot will add new subnets into the mix in an effort to curtail firewall usage. Herein is my biggest problem with the thugs. When I firewall one of your subnets that means go away. Don't use another to get in. I'll just research and determine those, put them in the mix, and firewall you even longer than I might have otherwise.
I'm not so opposed to the walking of my sites by their spiders as I am the biased search results produced and generally misleading they engage in as a "matter of business". In other words, behave or begone.
I don't value the leftist libtard google, googlebot, youtube, alphabet, or any other property associated with them and I dare say that generic search engine which worked beautifully is something to be mourned since it has been replaced with an algorithm designed for bias and control of the feeble network denizens who think google is something "good" ... when they are purest pushy technoevil.