Better Search-Engine Indexing of My Blog

Every so often I look at my server logs to see by what means people find their way to pages on my blog. One thing that's long bothered me is that people's Yahoo! and Google! searches often bring them to my blog's home page, rather than to the post-specific page containing the text that was matched by the their search.

For example, my previous post contains the text “There's a camera collector in Massachusetts....,” and because my blog home page contains my five most-recent posts, that post and its text will stay on the home page until I've written a few more posts. If the search engines index my blog home page, someone searching for, say, “camera collector” will be directed to my blog home page. That's all fine and well.... until a few days from now and that post has been bumped off the home page, at which point users following a link from a search-engine search-result page end up at the then-current home page and wonder where their content is.

The solution for this is fairly simple: tell the search engines to index only the permalink pages (the individual per-post pages), and not the blog home page, the index continuation pages, category pages, etc.

I finally got off my rear and implemented this on my blog, so starting today, the search engines won't index my non-post pages. What got me going was Fazal Majid's excellent writeup on blogging etiquette, How to Show Respect for Your Readers, which I found today (having only discovered his generally excellent writing yesterday). I'd already known the general approach on how to implement the “don't index me!” directives (one soaks up these things working for Yahoo! for eight years), but I'd been too lazy to look up the actual incantation needed. Fazal kindly included it in his post, and so I'd lost my last excuse for not going ahead and getting it done.

Having done this now, I'm sure I'll see my blog's Google PageRank go to zero (from it's currently dizzying value of 5), but as the title of Fazal's post indicates, it's all about respecting the reader.

There's real satisfaction in seeing that someone did a search on such-and-such, and it brought them to a page I've written exactly on that subject. My photo pages are particularly common, with people arriving via Yahoo! and Google! searches for “japan temples”, “presidential limousine”, and “snowfall” (to list just a few actual examples from the hundreds that arrive every day).

Even more so than with the photos, it's really gratifying to see searches like:
    “nikon d200 compact flash tested”
    “transcend compact flash for Nikon d200”
    “test transcend flash”
    “transcend 4GB 120x”
    “transcend 4Gb CF card”
    “transcend compact flash review”
    “Transcend 120x Compact Flash Card review”

land on my “Timing a Transcend 80x 4GB Compact Flash card with a Nikon D200” and “Update on Transcend 4GB 120x compact-flash card” posts, as happened today. Those having done the search certainly found exactly what they were looking for.

Most of my posts are just for keeping overseas family up to date on our lives here in Kyoto, but some of my posts have a broad appeal, and I'm happy to have them read and (hopefully) be helpful.


One comment so far...

Not related to this topic, but I want people to know a Jeff’s gem:
http://forum.java.sun.com/thread.jspa?forumID=256&threadID=502089

As for the photographs, real beauty of Japan could be found in its rural area.
Not Kyoto.
Rural, rural, deep into ruuuuural!

— comment by Hiroshi Iwatani on September 14th, 2006 at 9:56am JST (11 years, 3 months ago) comment permalink
Leave a comment...


All comments are invisible to others until Jeffrey approves them.

Please mention what part of the world you're writing from, if you don't mind. It's always interesting to see where people are visiting from.


You can use basic HTML; be sure to close tags properly.

Subscribe without commenting