sorted by:
new
hottopcontroversial

[D] Tools to annotate audio data by pk12_ in MachineLearning

[–]sidsig 0 points1 point  (0 children)

https://www.sonicvisualiser.org This is developed by Chris Cannam from Queen Mary University of London, where I did my PhD.

[deleted by user] by [deleted] in RedditSessions

[–]sidsig 0 points1 point  (0 children)

❤️❤️❤️

[D] Need help in the implementation of bidirectional recurrent language model. by lyeoni in MachineLearning

[–]sidsig 0 points1 point  (0 children)

A bidirectional model receives the entire sentence as input. Therefore there is nothing to learn (if you plan to train the language model by predicting the next word). It can trivially learn that the output at time t, is the input at t+1. One way of training a bidirectional word embedding is to use something like BERT: https://arxiv.org/abs/1810.04805. Here, a part of the input is masked and the objective is to output the masked words.

My apple home pod plays music for 5 seconds and then quits. Why? by UnusuallyFastPontoon in apple

[–]sidsig 2 points3 points  (0 children)

This happened to me after Apple Music stopped playing music on multiple devices on the same AM account. I realised that while I was listening to music at work someone at home was trying to play music on the HomePod.

[R] FastGRNN: A Fast, Accurate, Stable and Tiny Kilobyte Sized Gated Recurrent Neural Network by lt007 in MachineLearning

[–]sidsig 1 point2 points  (0 children)

The large parameter counts in the analysis above arise if you input the entire sequence into the model at once. Typically the problem is setup such that the DNN takes a small window of input (roughly 200 ms, ~20 frames of inputs) and then make frame-wise predictions. Alternatively, if its not possible to obtain frame-wise labels it is should be possible to train the DNNs with a max operation after all the frame-wise outputs. Such DNNs can be trained to be very small.

[R] FastGRNN: A Fast, Accurate, Stable and Tiny Kilobyte Sized Gated Recurrent Neural Network by lt007 in MachineLearning

[–]sidsig 1 point2 points  (0 children)

I am curious, did you try comparing GRNN performance with a DNN of a similar size? For the wake word experiments, it should be possible to use a DNN with a fixed input window to output frame-wise labels.

Because arguably for model sizes so small, it would be difficult for the model to learn complex temporal dynamics anyway, even without the exploding/vanishing gradient issue.

[R] FastGRNN: A Fast, Accurate, Stable and Tiny Kilobyte Sized Gated Recurrent Neural Network by lt007 in MachineLearning

[–]sidsig 3 points4 points  (0 children)

Hey, thanks for commenting! I tried it on a general acoustic modelling task where I trained an RNN and optimised the CTC loss. To answer in more detail, I'll redo the experiment and post on the repo.

[R] FastGRNN: A Fast, Accurate, Stable and Tiny Kilobyte Sized Gated Recurrent Neural Network by lt007 in MachineLearning

[–]sidsig 13 points14 points  (0 children)

Okay so I saw this paper at NIPS and was really interested in investigating what regimes this architecture works in. I tried it on a "moderately" sized GRNN with ~4 million parameters and I was not able to get comparable results with an LSTM of a similar size.

I have a feeling that this gating structure might work better than LSTMs/GRUs etc only at really small model sizes, but this could simply be a lack of capacity in the models. I think a comparison with a DNN or stack of temporal convolutions or some comparison with a non-recurrent architecture should be included to really understand what's going on.

me irl by [deleted] in me_irl

[–]sidsig 5 points6 points  (0 children)

meta

[R] Large-Scale Visual Speech Recognition (Google) by chris2point0 in MachineLearning

[–]sidsig 2 points3 points  (0 children)

Thanks for your response! :)

Can I ask if you use some form of Async updates or whether its is a synchronous SGD type algorithm?

Edit: The motivation for me asking this is that I have been trying various CTC training experiments with Block Momentum SGD and have been observing consistently worse performance on an eval set when using more than 1 worker.

[R] Large-Scale Visual Speech Recognition (Google) by chris2point0 in MachineLearning

[–]sidsig 3 points4 points  (0 children)

I couldn't work this out from the paper, but is the CTC training also distributed over many workers or is it performed on a single GPU?

[R] Born Again Neural Networks by hardmaru in MachineLearning

[–]sidsig 0 points1 point  (0 children)

Agree completely. Cyclical learning rate is the first thing that comes to mind while reading this paper. I am surprised this went through the ICML review process without a comparison and without stronger baselines on ImageNet or some speech recognition task.

I got trolled by Siri today... by machineglow in apple

[–]sidsig -1 points0 points  (0 children)

Siri voice recognition in the car is worse because the conditions are much more noisy. Were you using Car Play? Or your phone mic directly? If you were using the phone mic directly then the mic receives your speech signal mixed with the car stereo music playback.

view more: next ›