Crawling Blogs

October 9th, 2008

Through a period when my blog was updated only once, this is how Feedburner viewed bots.

blogcrawler

Note that crawling blogs is an interesting problem:

  • Recency is critical
  • Ping servers are available, albeit with incomplete coverage

Crawling blogs is also highly resource intensive:

  • Network latency
  • Disk access/write latency

How do your numbers look?

One Response to “Crawling Blogs”

  1. trackbacks - links for 2008-10-10 Says:

    […] Crawling Blogs (tags: myposts) […]

Leave a Reply