Wednesday, September 26, 2007

Article: How we funneled searchers from Google to our collections by catering to Web crawlers

In 2006, Marshall Breeding wrote an article entitled "How we funneled searchers from Google to our collections by catering to Web crawlers." As we know, not all software/databases can easily be crawled by Internet search engines. Some databases require extra ($) components, while you must kluge a solution for others. In Marshall's article, he talks about the solution they implemented for the Vanderbilt Television News Archive.

Question -- If your repository is not automatically being crawled by the Internet search engines, what solutions have you put in place to expose your content so that it is crawled? Please let us know. This is a topic that projects are talking about...and an area where we could all benefit from what others have done.


Technorati tags: ,

1 comment:

anne beaumont said...

We did not have our database(s) crawled, but by using an OAI server, and having OAIster harvest the relevant data, as well as our National Library of Australia for PictureAustralia, our resources are listed in Yahoo, Google and a number of other sites. We get far more traffic from outside our own domain than from within it.