Word frequency ****yser - Perl

"DVH" <dvh@dvhdvhdvhdvdh.dvh> wrote:

> Hi,
>
> Does anyone happen to know if there's a convenient module which will
> ****yse at least two XML files and list the most frequently-used
> words?
>
> (It would have to be able to reject tags

XML::Parser, which a Char handler.

> and certain words such as
> "the" and "is")

split in the handler on non-words, use a hash for counting. Delete
afterwards all occurences of the, is, etc.

Note that this is a very simplistic approach, since it words are hypenated,
it counts them as two different ones.

--
John Small Perl scripts: http://johnbokma.com/perl/
Perl programmer available: http://castleamber.com/
I ploink googlegroups.com :-)

John Bokma <john@castleamber.com> writes:

> "DVH" <dvh@dvhdvhdvhdvdh.dvh> wrote:
>
>> Hi,
>>
>> Does anyone happen to know if there's a convenient module which will
>> ****yse at least two XML files and list the most frequently-used
>> words?
>>
>> (It would have to be able to reject tags
>
> XML::Parser, which a Char handler.
>
>> and certain words such as
>> "the" and "is")
>
> split in the handler on non-words, use a hash for counting. Delete
> afterwards all occurences of the, is, etc.
>
> Note that this is a very simplistic approach, since it words are hypenated,
> it counts them as two different ones.

Seaching for "word frequency" on search.cpan.org turns up some modules
that are designed for this sort of thing, and may take some of the
trickier issues into account.

----Scott.

On Tue, 25 Oct 2005 01:30:58 +1000, Scott W Gifford wrote:

Hi Folks

A list of stop word, courtesy of MySQL, can be downloaded from:

http://savage.net.au/Ron/mysql-stop-words.txt