[BlackGriffen]

From Kicking Ass (DNC official blog):

Sometime between April 2003 and October 2003, someone at the White House added virtually all of the directories with "Iraq" in them to its robots.txt file, meaning that search engines would no longer list those pages in results or archive them.

Why would the White House do this? Those pages are still public, and the White House search engine itself does index those pages, so users can still get to them.

It's easy enough to understand the reasoning if you look at past White House actions. Earlier this year, the White House revised pages on its website claiming that "combat" was over in Iraq, changing them to say "major combat."

One of the reasons some alert readers noticed the change — and were able to prove it — was that Google had archived the pages before the change occurred. Now that all of the White House pages about Iraq are no longer archived by Google, such historical revisionism will be harder to catch.

Yes, I know, I'd be hard pressed to find a more biased source.

Is there a defense given that the actions where taken as accused?

Does the Wayback Machine ignore robots.txt files, or not?

BlackGriffen

[BlackGriffen]

Addendum: no the Wayback Machine does not ignore the robots.txt file.

Perhaps the GIA would be willing to provide such an archive?

BlackGriffen

[spacefreak]

Originally posted by BlackGriffen:
Is there a defense given that the actions where taken as accused?

I'm sure that coming up with a defense for modifying the robots.txt file is high atop Bush's priorities. He likely spent the better part of a week figuring out what a robots.txt file does in the first place. How much more time editing his website much Bush commit? Just the graphics production alone must take up the bulk of his time.

Maybe Bush can also explain why patriotism, kids, first lady, early childhood, europe, holidays, tee-ball, and others are also included in the robots.txt file.

This blog's accusations are ridiculous. Their supporting argument is the fact that a title of a page was changed from "combat operations have ended" to "major combat operations have ended", as if there is an attempt to revise history. Nowhere do they mention, however, that the original text of Bush's speech (which that page contains) has always used the phrase "major combat operations". Nor do they even offer the possibility that the page title was done in haste, and subsequently corrected. But it's no surprise that they refrain from making such an offering, for making a correction is much different than revising history, and it would score no political points.

The information is public, and if someone wants to research official White House policy and information, they can go to the website. They even have their own search engine there (which probably took Bush and Cheney a month to develop and program).

Regardless, Bush is a pretty good web builder. His site looks professional, the design and navigation is solid, and there is a ton of information on it. If he doesn't win re-election, he can probably find a job at a leading web company.

[Spliffdaddy]

[petehammer]

Originally posted by spacefreak:
I'm sure that coming up with a defense for modifying the robots.txt file is high atop Bush's priorities. He likely spent the better part of a week figuring out what a robots.txt file does in the first place. How much more time editing his website much Bush commit? Just the graphics production alone must take up the bulk of his time.

Yup, that's what we're all claiming. That Bush himself built the site. Yup, all us crazy liberals.

Or could it be we have a problem with the administration? That someone in the administration didn't like certain past ideas and wants to "clean them up."

Strawman:
"a weak or imaginary opposition (as an argument or adversary) set up only to be easily confuted"

(Edited to add: it also made slashdot:
Slashdot

[Spliffdaddy]

The accusation falls flat on its face unless similarities can be shown between the word "Iraq" and the words "patriotism", "kids", "first lady", "early childhood", "europe", "holidays", and "tee-ball".

good luck.

[petehammer]

Originally posted by Spliffdaddy:
The accusation falls flat on its face unless similarities can be shown between the word "Iraq" and the words "patriotism", "kids", "first lady", "early childhood", "europe", "holidays", and "tee-ball".

good luck.

Huh?

Do you know what's in the robots.txt file? Here's a sample:

Disallow: /911/911day/iraq
Disallow: /911/911day/text
Disallow: /911/heroes/iraq
Disallow: /911/heroes/text
Disallow: /911/iraq
Disallow: /911/patriotism/iraq
Disallow: /911/patriotism/text
Disallow: /911/patriotism2/iraq
Disallow: /911/patriotism2/text
Disallow: /911/progress/iraq
Disallow: /911/progress/text
Disallow: /911/remembrance/iraq
Disallow: /911/remembrance/text
Disallow: /911/response/iraq
Disallow: /911/response/text
Disallow: /911/sept112002/iraq
Disallow: /911/sept112002/text

http://www.whitehouse.gov/robots.txt

And there's a whole state of the union section.

[chris v]

Originally posted by petehammer:
Huh?

Do you know what's in the robots.txt file? Here's a sample:


http://www.whitehouse.gov/robots.txt

Thank you, saved to disk.

CV

[nonhuman]

What's this about I wonder.

Disallow: /easter/iraq
Disallow: /easter/states/iraq
Disallow: /easter/states/text
Disallow: /easter/text

[Mithras]

Originally posted by nonhuman:
What's this about I wonder.

Disallow: /easter/iraq
Disallow: /easter/states/iraq
Disallow: /easter/states/text
Disallow: /easter/text

They took a list of subdirectories:
/easter
/kids/barney
/firstlady
etc.

then added
/text
and
/iraq
to each of them (whether or not /easter/iraq exists),
to prevent indexing of
(a) the text-only versions (so people get pointed to the normal image-full part of the site, not the spare text part)
(b) anything having to do with Iraq

Objective (a) is a standard webmaster objective, no harm done. Objective (b) is what has eyebrows raised.

[spacefreak]

Originally posted by petehammer:
That someone in the administration didn't like certain past ideas and wants to "clean them up."

Strawman:
"a weak or imaginary opposition (as an argument or adversary) set up only to be easily confuted"

This blockbuster big allegation focuses on such a ridiculous premise. It's not like the content of the page (Bush's speech) changed. We're talking about a web page title.

Example 1:

President Bush Announces Combat Operations in Iraq Have Ended
Remarks by the President from the USS Abraham Lincoln
At Sea Off the Coast of San Diego, California

THE PRESIDENT: Thank you all very much. Admiral Kelly, Captain Card, officers and sailors of the USS Abraham Lincoln, my fellow Americans: Major combat operations in Iraq have ended. In the battle of Iraq, the United States and our allies have prevailed. (Applause.) And now our coalition is engaged in securing and reconstructing that country...

Example 2:

President Bush Announces Major Combat Operations in Iraq Have Ended
Remarks by the President from the USS Abraham Lincoln
At Sea Off the Coast of San Diego, California

THE PRESIDENT: Thank you all very much. Admiral Kelly, Captain Card, officers and sailors of the USS Abraham Lincoln, my fellow Americans: Major combat operations in Iraq have ended. In the battle of Iraq, the United States and our allies have prevailed. (Applause.) And now our coalition is engaged in securing and reconstructing that country...

This is a joke of an allegation (that the President is revising history). The ONLY item that was corrected was the addition of the word "major" to the title of the web page. The speech remained the same, and even states "Major combat operations in Iraq have ended." on both the original web page and the one with the corrected title.

[Secret__Police]

Originally posted by BlackGriffen:

you will report to your local Police station for interrogation.
The new America will not tolerate your 'Paranoia'
Are you feeling well??? We can help you..........

[spacefreak]

Originally posted by Mithras:
Objective (b) is what has eyebrows raised.

Perhaps they don't want Google caching pages that are updated frequently, so when someone performs a search on Iraq progress, they don't see months-old cached pages instead of up-to-date information.

If someone wants to claim that the White House doesn't want pages that change often being cached by search engines, that's fine. To say that this is proof of Bush revising history, however, is pure crap.

[Lerkfish]

No, THIS is the important thing to note:

Disallow: /stateoftheunion/2002/behindthescenes/iraq
Disallow: /stateoftheunion/2002/behindthescenes/print/iraq
Disallow: /stateoftheunion/2002/behindthescenes/print/text
Disallow: /stateoftheunion/2002/behindthescenes/text
Disallow: /stateoftheunion/2002/iraq
Disallow: /stateoftheunion/2002/photos/iraq
Disallow: /stateoftheunion/2002/photos/print/iraq
Disallow: /stateoftheunion/2002/photos/print/text
Disallow: /stateoftheunion/2002/photos/text
Disallow: /stateoftheunion/2002/print/iraq
Disallow: /stateoftheunion/2002/print/text
Disallow: /stateoftheunion/2002/text
Disallow: /stateoftheunion/behindthescenes/iraq
Disallow: /stateoftheunion/behindthescenes/print/iraq
Disallow: /stateoftheunion/behindthescenes/print/text
Disallow: /stateoftheunion/behindthescenes/text

mainly because on those pages it specifically told how Bush goes over every single word himself...and had photos of him circling and crossing out words of the speech personally.

This is no longer up (I wondered how long that would take) once the "16 words" debacle was laid at the feet of Tenet.

[petehammer]

Originally posted by spacefreak:
To say that this is proof of Bush revising history, however, is pure crap.

Assert:
"stating confidently without need for proof or regard for evidence "

[BlackGriffen]

Spacefreak, last I checked, you were the only person talking about Bush.

You do realize that the administration does a whole lot of stuff without the Pres's knowledge or consent.

BG

[spacefreak]

Originally posted by BlackGriffen:
Spacefreak, last I checked, you were the only person talking about Bush.

You do realize that the administration does a whole lot of stuff without the Pres's knowledge or consent.

Yeah...I know. My posts were more directed at the link given on your referenced blog...

It's easy enough to understand the reasoning if you look at past White House actions. Earlier this year , the White House revised pages on its website claiming that "combat" was over in Iraq, changing them to say "major combat."

I followed that link to differentstrings.info site to see this:

More Bush Revisionism

The Washington Post's Dana Milbank notes that Bush is once again trying to revise history .

Stated so matter of factly, I was surprised a Post writer would allege Bush is revising history. I then continued to read the Post's article, which did not claim that Bush was revising history - it just mentioned the web change and some clarification by the administration on the differences between combat and majot combat.

So while I may be the only one one mentioning Bush in this thread, it's because I am addressing the accusations made in the link you provided and their supporting information. It is portrayed that a website alteration is instead a Bush revision, as if he personally performed it.

Do any of the Democratic candidates know Flash? I hear that Bush has been training w/ Flash for the last year and will be unleashing some highly animated and interactive web features as soon as his schedule opens up.

I'm still upset Rummy has been dismissed from his heading of the ecommerce section of the website. Rice should do OK, but she's just not up to speed on transaction processing requirements, and I fear the build of the White House E-Giftshop will be delayed even longer.

Kudos to Ashcroft for proposing Bush integrate streaming audio and video on the site. Now, if only we could get Powell to promote the website more effectively, though I'm sure the Google/robots.txt issue has put a dent in marketing plans.

[eklipse]

Originally posted by spacefreak:
Regardless, Bush is a pretty good web builder. His site looks professional, the design and navigation is solid, and there is a ton of information on it. If he doesn't win re-election, he can probably find a job at a leading web company.

Yeah, but those popup ads on his site sure suck ass:

[petehammer]

Hahahahaha, yes, get it, Bush is doing the website changes himself. What a kookie idea!

Odds are, some low-level IT guy "just decided" to change the robots.txt file on his own. No directive, just did it for the hell of it. That's my understanding of how things work.

[mr. natural]

I don't get all the fuss.

My White House link seems to work just fine!