Yesterday afternoon I noticed a few hits on this blog from “diggstage01.digg.com”. The visitor characteristics make me think it could be a content scraper bot.
There were 4 consecutive hits that happened over 1 minute period. The visitor also had javascript disabled. Both are common charecteristics of a bot. It could very well just be someone browsing with js disabled and refers off, but let’s have some fun and just speculate a little bit.
Why would digg be sending out a bot to my little blog?
My theory is that they are finally going to make their own version of duggmirror.com (now called duggtrends.com), the popular site that maintains backup copies of pages that get dugg and can’t handle the onslaught of traffic. Often when a site crashes, duggmirror will have already made a backup copy of the content, which can be viewed on the duggmirror site.
This would be a great move by digg, since sites succumbing to the digg effect is a big annoyance of the site. duggmirror does a good job right now, but digg is losing eyeballs to duggmirror, when they could be keeping them on their own site.
One downside of digg providing a cached version of crashed sites is that they might anger the owners of the original content, who make money by serving ads on their own site.
Maybe digg could create a service that attempts to ping a heavily dugg site to determine if it is still alive. If it is, link to the original content. If it is down, link to the cached version hosted by digg.
Again, this is pure speculation, but some decent food for thought none the less.
- BROWSE / IN TIMELINE
- « Regular Expression for validating US Currency
- » Twitter is the new Friendster
- BROWSE / IN techBits
- « Regular Expression for validating US Currency
- » Twitter is the new Friendster
SPEAK / ADD YOUR COMMENT
Comments are moderated.

