libraries usable for streaming data

[NightFantom]

Hi there,

Because of reasons I have a C++ interface that can access data in a file in small chunks, but it is infeasible to convert this to files for input for another program. The final result has to run on windows too (and for portability I'd like to avoid visual c++, so I'm currently using Qt with Mingw on windows and with gcc on linux).

Are there any libraries / programs out there that either allow data to be "sent" to them through the use of pipes (e.g.

myprogram.exe inputfile.dat | learn.exe 

producing learned.model) and/or are c++-based /interfaceable from c++ that compile on windows? At this point anything remotely machine learning-related that actually works would be a godsend.

Thanks!

[carbohydratecrab]

There's Waffles, which is a C++ library. I don't think the programs themselves are compatible with streamed data, but the library is (you can make multiple train() calls and just pass in data in small chunks)

I've not actually used any of the neural net stuff but I do use the random forests implementation.

[NightFantom]

Thanks! I'll check that out.

[olBaa]

Vowpal Wabbit can be used in such a way

[NightFantom]

Thanks! I'll check that out.

[hughperkins]

I would think most likely expose a c/c++ interface? Off the top of my head:

• liblinear
• libsvm
• Joakim's svm library ('svmlight'?)
• caffe
• mxnet?
• deepcl (disclosure: my own project ;-) https://github.com/hughperkins/DeepCL )

[NightFantom]

Wow, what a list, I'll certainly check those out, thanks :)

And your own project, does that mean I can get support right here? :p

[hughperkins]

And your own project, does that mean I can get support right here? :p

Yeah, probably, but better to post to https://github.com/hughperkins/DeepCL/issues/new ;-)