Reply to Topic
My forum site is getting pummeled by the inktomi search engine spider.
Is that most of your guests too?
I have a hard time deciding whether or not it is worth it to get the coverage in the inktomi search engine. Not too many sites use it, and sometimes I have about 50 guests that are all inktomi. It seems like a huge bandwidth waste. I would rather have Google spider the whole site, but it can't handle the long URLs.
Is that most of your guests too?
I have a hard time deciding whether or not it is worth it to get the coverage in the inktomi search engine. Not too many sites use it, and sometimes I have about 50 guests that are all inktomi. It seems like a huge bandwidth waste. I would rather have Google spider the whole site, but it can't handle the long URLs.
__________________
Jim Lewis
To BE or Not to BE, or: How I Learned to Stop Worrying and Love the Barium Enema
Crouching Tiger, Paint Your Wagon - Forums
Jim Lewis
To BE or Not to BE, or: How I Learned to Stop Worrying and Love the Barium Enema
Crouching Tiger, Paint Your Wagon - Forums