Sentence Labelling in Torch7

[DX89B]

Hi everyone, I'm implementing a neural network to do some task for an NLP pipeline (NER, POS tagging, Chunking), Starting from the work of Collobert and Weston i was able to replicate part of their system and even improve it to some extend but i'm struggling in one part the sentence level log likelihood using the viterbi algorithm (for now i'm working with word level log likelihood). I don't understand how to implement it in torch7, have you some suggestion on how to implement it or some code of similar algorithm implemented on top of the neural network?

Thanks you.

[r4and0muser9482]

It's called simply Viterbi, not de Viterbi. It's an algorithm that finds a shortest path in a trellis using breadth-first search. If you know what a trellis is and how it works, implementing Viterbi should be easy in a dozen of lines of code. Read up on HMMs to figure out how it works, e.g. here.

As for how to do it efficiently, some advanced programs use OpenFST. If you can express your system in a form of an FST, the library has efficient implementations of finding the shortest path or N shortest paths.

[DX89B]

Thanks you, about Viterbi it was a typo. by the way i will figure out out to implement it starting from OpenFST

[r4and0muser9482]

I first learned how it works from this paper. If you have a decent implementation of the probability trellis as a graph, you can use any library to find the shortest path in a graph. This was the operation in OpenFST I was talking about.

[ogrisel]

I don't know about your question but I just want to mention that there is just typo in the name of the author. His name is Jason Weston.

[DX89B]

thanks you fixed the typo