[Sigia-l] low-end (ish) search engine/spider combo

[Avi Rappoport]

At 5:08 PM +0100 7/16/02, Ben Whalley wrote:
>Hi,
>
>We're working with a client at the moment who is in the process of 
>rebuilding a number of sites and subsites which are owned by 
>numerous individuals across the company (and around the world). 
>They're all over the place at the moment and are working hard to 
>sort it out, but in the meantime want to implement a search engine 
>which will cover all of their content, even as it changes and 
>develops, and allow the group site to be used as a kind of portal to 
>the rest of the business.
>
>My first though is buy a neat little google-box but it's a little 
>out of our price range here. Does anyone know other sensible options 
>for spidering and returning results on multiple sites? Our budget 
>will be about $1000-$2000.

On Windows for your scale, I generally recommend dtSearch, Phantom 
(recently moved to <http://www.phantomsearch.com>), MondoSearch or 
ASPSeek.  They all have decent admin interfaces, index multiple web 
sites, handle a variety of languages, and work pretty well for 
retrieval.  If you like command lines, SWISH-E is free and 
open-source.

On Unix, I've heard good things about Alkaline and Webinator, and 
there are a bunch of free open-source engines including ht://Dig, 
SWISH-E, mnoGoSearch and ASPSeek.

Another alternative is remote search services, which crawl your site 
to produce an index that is stored on their servers.  Your search 
forms call their search engines and produce a results list with pages 
from your site.  This can save you a lot of server resources and 
works very well for many people.  There are a lot of good services, 
though they can get expensive if you have a lot of pages: Atomz, 
FusionBot, PicoSearch, FreeFind, Spiderline...

My site has a ton of information and you're welcome to contact me 
directly for more help.

Avi
-- 
Search Server Industry Analysis from Search Tools Consulting
    (510) 845-2551  -- <mailto: analyst at searchtools.com>
Complete Guide to Search Engines for Web Sites and Intranets
    <http://www.searchtools.com>