jamesglewisf
07-13-03, 05:46 PM
My forum site is getting pummeled by the inktomi search engine spider.
Is that most of your guests too?
I have a hard time deciding whether or not it is worth it to get the coverage in the inktomi search engine. Not too many sites use it, and sometimes I have about 50 guests that are all inktomi. It seems like a huge bandwidth waste. I would rather have Google spider the whole site, but it can't handle the long URLs.
Is that most of your guests too?
I have a hard time deciding whether or not it is worth it to get the coverage in the inktomi search engine. Not too many sites use it, and sometimes I have about 50 guests that are all inktomi. It seems like a huge bandwidth waste. I would rather have Google spider the whole site, but it can't handle the long URLs.