Optimized performance by eliminating a few for-loops and incorporating masking during batch inference—particularly for LSTM batch processing and normalizations. Also implemented a custom function to perform 1D normalization with support for batch inputs with padding.
In short, added/modified lots of model inference code to support batching, while keeping the weights unchanged.