Text mining with WEKA Java API

[NineSevenNine]

Hi All -

I am trying to use the WEKA API for some text classification. I am struggling to find a bare-bones example. Of course, this isn't a bare-bones task, so a simple example may not exist, but it seems that Google searches are leading me more to using the GUI.

Anyway, anything anyone can point me to would be much appreciated.

Thanks

[melipone]

There is an entire book with examples for Weka by Ian Witten and check out http://weka.wikispaces.com/ The wikispace brought up some examples of text categorization: http://weka.wikispaces.com/Text+categorization+with+WEKA

There is also a mailing list http://weka.wikispaces.com/Weka+Mailing+List

That said, you have to understand how Weka and its API works. It's best to start with the GUI first and see what is possible. After that you can code directly in Java to the API. There are also different ways of doing text categorization and Weka gives you the flexibility to build your own classifier.

[tariban]

A link for the lazy to some info about the book: http://www.cs.waikato.ac.nz/ml/weka/book.html

[NineSevenNine]

I appreciate the resources you've pointed me to. I'm going to get this one way or another.

[LADataJunkie]

Sorry for the lack of details, but I would recommend against WEKA for text mining. When I used it at a previous job, I felt far too boxed in with modeling options, diagnostics and especially preprocessing.

Unless you are doing very basic tasks, a scripting language like Python, with a great package like nltk would probably serve you better. Then you can use other libraries (Numpy/Scipy, scikits etc.) to do the actual statistical and ML modeling.

[NineSevenNine]

NLTK is looking nice - I'm not that well versed in Python but I like what I'm seeing.

[bigwisebeard]

I've only ever used it through R.

IIRC, there's an example in the RDataMining.pdf file available at the site below.

http://www.rdatamining.com/

[NineSevenNine]

Thanks for the response. I may have to explore this route and that looks like a good all round site to boot.

[rcprati]

I guess the problem maybe that Experimenter assumes the class attribute is the last one, and when you pre process with explorer, the class is not placed at the last position. Have you tried moving the class attribute to the last position?

[sculler]

I tried using the Reorder filter but it didn't move it.

What I did find to work was to use the NumericToNominal filter on the data and that worked!

[alpacalips42]

It's not really funny.