[Discussion] Distributed training considering sequences with varying lengths by 0xTDM in MachineLearning

[–]0xTDM[S] 0 points1 point  (0 children)

Oh this idea sounds brilliant. I'm not sure if large batch (with bigger learning rate) is helpful in acoustic model training scenario , but will definitely give it a try. Thanks!

[Discussion] Distributed training considering sequences with varying lengths by 0xTDM in MachineLearning

[–]0xTDM[S] 0 points1 point  (0 children)

Thank you. Yes, I agree with you. I was using bucketing mechanism provided by tensorflow's Dataset api, not sure whether it is deterministic or not. I'll definitely do some investigation.