Wednesday, 27 June 2007

Web crawler architectures

A crawler must not only have a good crawling strategy, as noted in the previous sections, but it should also have a highly optimized architecture.

Source: http://en.wikipedia.org/wiki/Web_crawler

No comments: