Disallow: /iraq/*
The crafty democrats have been looking at the robots.txt file of whitehouse.gov, and found that it denies search engines like google from indexing anything related to Iraq.
Disallow: /911/progress/iraq
Disallow: /911/progress/text
Disallow: /911/remembrance/iraq
Disallow: /911/remembrance/text
Disallow: /911/response/iraq
Disallow: /911/response/text
You get the idea. It is big, and almost everything disallowed is related to Iraq.
The Democrats have an explanation:
Why would the White House do this? Those pages are still public, and the White House search engine itself does index those pages, so users can still get to them.
It's easy enough to understand the reasoning if you look at past White House actions. Earlier this year, the White House revised pages on its website claiming that "combat" was over in Iraq, changing them to say "major combat."
One of the reasons some alert readers noticed the change — and were able to prove it — was that Google had archived the pages before the change occurred. Now that all of the White House pages about Iraq are no longer archived by Google, such historical revisionism will be harder to catch.
Maybe a bit far fetched, but it is interesting that the robots.txt file excludes Iraq-related information specificially.
PS: Here is an explanation of how the robots.txt file works.
2:58:42 PM 
|
