04-09-2004, 09:45 AM
The math is trivial. In fact, despite all the noise about Baysian filtering lately, the whole concept is pretty simple. The interesting thing is that it can be done completely automatically, and yields better results than having a person try to do the same thing by hand.
In plain english, one feeds the analyzer two sets of inputs--a 'Good' set and a 'Bad' set, each containing many objects. Each object has properties that can be analyzed statistically. One typically feeds in several thousand objects of each class.
For instance, the objects might be email messages, with 'Good' ones being real mails and 'Bad' ones being advertizements. In this example, the words that appear in the messages would be the properties that the analyzer uses.
Continuing this example, the analyzer might find that the word 'Enlargement' appears in 10% of the advertizements, but in only 0.1% of the real mails. Thus, when presented with a new email the presence of the word 'Enlargement' is an indication that the mail is an advertizement. In fact, using just that one word as an indicator, the odds are 10% to 0.1%, or 100-to-1 after moving the decimal places.
Now imagine that the unknown email contains several words that are known to be indicators. The Baysian step is to multiply together the odds for each of the words to get an aggregate estimate for the odds of the unknown email being good or bad.
Hope this helps. Try googling for 'Baysian email filtering' if you need to know more. There's also lots of free source-code out there for this.
In plain english, one feeds the analyzer two sets of inputs--a 'Good' set and a 'Bad' set, each containing many objects. Each object has properties that can be analyzed statistically. One typically feeds in several thousand objects of each class.
For instance, the objects might be email messages, with 'Good' ones being real mails and 'Bad' ones being advertizements. In this example, the words that appear in the messages would be the properties that the analyzer uses.
Continuing this example, the analyzer might find that the word 'Enlargement' appears in 10% of the advertizements, but in only 0.1% of the real mails. Thus, when presented with a new email the presence of the word 'Enlargement' is an indication that the mail is an advertizement. In fact, using just that one word as an indicator, the odds are 10% to 0.1%, or 100-to-1 after moving the decimal places.
Now imagine that the unknown email contains several words that are known to be indicators. The Baysian step is to multiply together the odds for each of the words to get an aggregate estimate for the odds of the unknown email being good or bad.
Hope this helps. Try googling for 'Baysian email filtering' if you need to know more. There's also lots of free source-code out there for this.