[Sigia-l] Findability

[Simon Wistow]

On Mon, Jul 21, 2003 at 09:27:25AM -0400, Peter Morville said:
> I agree with your overall perspective here, but could you clarify your
> theory on the evolution of algorithms?  Are you suggesting artificial
> intelligence (AI) will drive the rapid development of information
> retrieval (IR) algorithms?  If so, do you have any evidence?  If not,
> why will human-created algorithms evolve faster than humans?  Thanks!

Algorithms can breed faster than people.

Give each algorithm some trait. We'll call these genes. These genes 
dictate some behaviour about retrieving information. 

You make hundreds or thousands of these algorithms - you can make them 
randomly if you want. Then you set them on information and ask them to 
make an inference. Then you rate that inference. Then you 'breed' the 
strongest ones. Lather, Rinse, Repeat.

Basically the problem is that human beings don't scale well - a classic 
example of this is search engine's crawler created indexes vs something 
like Yahoo!'s directory.

Most of the time it's better to get an algorithm to do 99% of the heavy 
lifting for 80% of the result than get a human to do 80-100% of the 
lifting for 90% of the result.

Of course you'll always need a human in the loop somewhere. Probably.

-- 
stay up late ... if we want to