isn't quite ashamed enough to present

jr conlin's ink stained banana

2005-10-23

:: Search Meme

So, Jeremy points out that looking at his blog logs ( or {b,}logs for those of you who use bash) that he gets an unholy amount of traffic from Google. Fair enough, there's a reason that they're number one. Jeremy doesn't really elaborate on his findings though.

So i spent some time going over my blog results for the past month too, just to see if there are any interesting trends. Surprisingly, there are.

For what it's worth, here's the baseline i have:
SELECT sum(if(referer like '%google%q=%',1,0)) as 'Google',
sum(if(referer like '%yahoo%p=%',1,0)) as 'Yahoo',
sum(if(referer like '%msn%q=%',1,0)) as 'MSN'
FROM `blog_accesslog`
where time between '2005-10-01' and now();

+--------+--------+--------+
| Google | Yahoo  | MSN    |
+--------+--------+--------+
|    714 |     84 |     64 |
+--------+--------+--------+


Hmm… kinda high for google, but more importantly, i noticed something else. Sorting by who is hitting me and where they're going turns up a LOT of duplication.
If i group by the remote host and path i get:

+--------+--------+--------+
| Google | Yahoo  | MSN    |
+--------+--------+--------+
|    332 |     25 |     18 |
+--------+--------+--------+

i'm not quite sure i know why i get the duplication. They're (technically) different hits because the associated time is different and there's no set pattern about the number of repeat queries per host, yet all of the search engines do this. Granted, there's probably something wrong in how i'm doing my logs, so i'm willing to ignore that for now. Suffice to say that i get several multiples of traffic from Google than i do from Yahoo.

It's also no great surprise that the bulk of queries i'm getting are for the curl/php write up, followed by the perrenial "Annoying Questions" query.

The other thing that was surprising was that google wasn't where i got the bulk of my traffic. That honor went to Yahoo!.

"But, how is that possible", i hear you cry? Well, because of a goof i noticed when i first built out my query. If i just looked for "yahoo.com" i got well into the thousands of links. This included links from My.Yahoo, MyWeb2.0, bookmarks, 360, and other harder links.

So, who's the search engine that folks use most? Google.
What do most folks use to find me? Yahoo.

Odd.

  1. 2005-10-23 19:49:06
    For all their vying to be, uh... whatever they're trying to be, I still view Google as a one-trick pony with a nice "Maps" feature. While I do basic searches on Google more, Yahoo's still by far the best All-in-One reference/portal. In comic book terms, Google would be like the Incredible Hulk, and Yahoo would be an army of X-men.
  2. 2005-10-23 22:07:40
    Oct: referer URL: google: 622, MSNSearch: 42, YahooSearch: 38. Client domain: google: 885, inktomisearch: 766 ... msn doesn't show a domain, so no stats in http-analyze, but they're at least up there with inktomi. Note, that's the same amount of hits, more or less, from all three search engine spiders, and 20 times the amount of hits from google search. Bleh for inefficient spiders.
  3. 2005-10-23 22:17:23
    ... as for search engines, web search would be better if search engines were to filter out all wikipedia content sites except for wikipedia itself. That'd catch most (if not all) of the pagerank farms. I still don't use yahoo search cos they greet me with a "hello, _yoursigninid-here_." I value my privacy a bit more than that. Ditch the cookies, yahoo search.
  4. 2005-10-24 06:39:11
    jr, i think those numbers are skewed since they only take into account "google.com". a lot of the google referrers that i get come from google.ca and elsewhere (and google seems to have a lot of ip addresses referring also).
  5. 2005-10-24 06:55:01
    actually they're not skewed. I goofed with the cut and paste and forgot to remove the ".com".
  6. [...] UPDATE: JR Conlin points out that while Google wins hands down on search referrals, Yahoo kills Google on total referrals, thanks to it having more than two highly-used services. Posted: October 23, 2005 by Nathan Weinberg in: [...]
Wanna join in?

Blogs of note
personal that's my blog
(The Official Blog of the Internet)
memoirs of hydrogen guy matthew shepherd (quebec) rhapsodic.org Henriette's Herbal Blog lynne ydw i slumbering lungfish
geek Y!Cool Thing jeremy z
(The Official Website of the Internet)
dave's picks ultramookie Josh Woodward derek balling simon willison
news ars technica search engine watch

Powered by WordPress
Hosted on Dreamhost.