isn't quite ashamed enough to present

jr conlin's ink stained banana

2007-08-08

::Openly Evil

Just noticed something interesting.

(content as of 8:42PM 08/08/2007)
http://local.google.com/robots.txt

User-agent: *
Allow: /searchhistory/
Disallow: /news?output=xhtml&
Allow: /news?output=xhtml
Disallow: /search
Disallow: /groups
Disallow: /images
Disallow: /catalogs
Disallow: /catalogues
Disallow: /news
Disallow: /nwshp
Disallow: /?
Disallow: /addurl/image?
Disallow: /pagead/
Disallow: /relpage/
Disallow: /relcontent
Disallow: /sorry/
Disallow: /imgres
Disallow: /keyword/
Disallow: /u/
Disallow: /univ/
Disallow: /cobrand
Disallow: /custom
Disallow: /advanced_group_search
Disallow: /advanced_search
Disallow: /googlesite
Disallow: /preferences
Disallow: /setprefs
Disallow: /swr
Disallow: /url
Disallow: /default
Disallow: /m?
Disallow: /m/search?
Disallow: /wml?
Disallow: /wml/search?
Disallow: /xhtml?
Disallow: /xhtml/search?
Disallow: /xml?
Disallow: /imode?
Disallow: /imode/search?
Disallow: /jsky?
Disallow: /jsky/search?
Disallow: /pda?
Disallow: /pda/search?
Disallow: /sprint_xhtml
Disallow: /sprint_wml
Disallow: /pqa
Disallow: /palm
Disallow: /gwt/
Disallow: /purchases
Disallow: /hws
Disallow: /bsd?
Disallow: /linux?
Disallow: /mac?
Disallow: /microsoft?
Disallow: /unclesam?
Disallow: /answers/search?q=
Disallow: /local?
Disallow: /local_url
Disallow: /froogle?
Disallow: /products?
Disallow: /froogle_
Disallow: /product_
Disallow: /products_
Disallow: /print
Disallow: /books
Disallow: /patents?
Disallow: /scholar?
Disallow: /complete
Disallow: /sponsoredlinks
Disallow: /videosearch?
Disallow: /videopreview?
Disallow: /videoprograminfo?
Disallow: /maps?
Disallow: /translate?
Disallow: /ie?
Disallow: /sms/demo?
Disallow: /katrina?
Disallow: /blogsearch?
Disallow: /blogsearch/
Disallow: /blogsearch_feeds
Disallow: /advanced_blog_search
Disallow: /reader/
Disallow: /uds/
Disallow: /chart?
Disallow: /transit?
Disallow: /mbd?
Disallow: /extern_js/
Disallow: /calendar/feeds/
Disallow: /calendar/ical/
Disallow: /cl2/feeds/
Disallow: /cl2/ical/
Disallow: /coop/directory
Disallow: /coop/manage
Disallow: /trends?
Disallow: /trends/music?
Disallow: /notebook/search?
Disallow: /music
Disallow: /browsersync
Disallow: /call
Disallow: /archivesearch?
Disallow: /archivesearch/url
Disallow: /archivesearch/advanced_search
Disallow: /base/search?
Disallow: /base/reportbadoffer
Disallow: /base/s2
Disallow: /urchin_test/
Disallow: /movies?
Disallow: /codesearch?
Disallow: /codesearch/feeds/search?
Disallow: /wapsearch?
Disallow: /safebrowsing
Disallow: /reviews/search?
Disallow: /orkut/albums
Disallow: /jsapi
Disallow: /views?
Disallow: /c/
Disallow: /cbk
Disallow: /recharge/dashboard/car
Disallow: /recharge/dashboard/static/
Disallow: /translate_s?
Disallow: /s2
Disallow: /transconsole/portal/

http://local.yahoo.com/robots.txt

User-agent: *
Disallow:

Who's more open?

(For the non-geek, a quick example to show what i'm talking about)

ech
2007-08-08 - 21:21:25

Yes, Yahoo! is definitely more open… with giving political dissidents' information to dictatorial governments, etc… You seem to have a bone to pick with Google's motto; I'm not sure comparing it to Yahoo! is the best way to make that point.


PatrickQG
2007-08-08 - 23:02:32

In that case pretty much all of the disallow's are search forms. Most of which are available on other hostnames, so I guess they're preventing http://local.google.com/cobrand (and so on) from showing up in search results.


jrconlin
2007-08-09 - 06:51:48

Patrick: There's a fair number of base URLs in there that aren't search forms. (news, groups, univ), as well as some sites that are nothing but search results (local, maps, etc.)

[begin personal diatribe]
ech: While I'm obviously not going to comment directly on the current investigation into some lawyer's sloppy work (since I don't have any specifics), and while I too wish that China had more faith in their government system, I will note that the more talked about case started when a guy in a country known as a highly oppressive one to it's citizens, received a document that said "Don't leak this" and then turns right around and posts it from a website also well known to be under said regime's monitoring, it's really hard to see that there can't be repercussions for said individual.


Hey, delicious user, Save This Page
Blogs of note
personal that's my blog
(The Official Blog of the Internet)
memoirs of hydrogen guy matthew shepherd (quebec) rhapsodic.org j$ (right) Henriette's Herbal Blog fanatical apathy lynne ydw i iconophobia slumbering lungfish
geek Y!Cool Thing michael j radwin jeremy z
(The Official Website of the Internet)
dave's picks ultramookie Josh Woodward derek balling j$ (left) simon willison Yahoo! Search Blog
news ars technica search engine watch webmaster world.com
forums uh.net man-man killroy & tina

experimental

Firefox search plugins for Yahoo!

My Living Room media box config

The Official "Official" Registry of the Internet

Powered by WordPress
Hosted on Dreamhost.
And Steveo's page is Totally Fucking Awsome.