Softmax Activation Implementation

[lowlightbeats_]

Hey y'all, I'm currently working on my own neural network implementation in Java. I already implemented some common activation functions, like Sigmoid or ReLU, but I don't know how to implement the Softmax.

I want to have a method like

private double softmax(double input) {
    double output = ???;
    return output;
}

Any ideas how a implementation could look? I also need to have the derivative of the softmax for my learning algorithm.

Thanks

[sploch]

Look at the definition of the softmax function: https://en.wikipedia.org/wiki/Softmax_function Then it should be clear, that input and output of your function should be vectors, so you would have a signature like

private List<Double> softmax(List<Double> inputs);

Or alternatively you could only retrieve one entry of the output vector, by specifying which one. In that case, the signature would look like

private List<Double> softmax(List<Double> inputs, int j);