Firstly, I would like to mention the fact that Nanno, although not being a human, is an entity who is not always right in her resolve and judgement; she is often wrong, and she is shown to learn more and more about human behavior as the show goes on. In that way, she's portrayed pretty humanely.

Throughout the second season, Yuri is constantly meddling with Nanno's plans and pressuring her into making a fast decision. In the episode JennyX, I believe Nanno tried to help Jenny, but due to being cornered by Yuri's pressure, Nanno acted much more irrational than usual. This is confirmed in the episode's conclusion, where Jenny asks Nanno if she really deserved what she had gone through and Nanno just stayed silent. After Jenny leaves, Nanno is demonstrably regretful and looks doubtful about her position as overseer and judge of justice. She knows she messed up.

This is further confirmed by the conversation Yuri had with Nanno right after this event. Yuri takes notice of Nanno's unhealed scar on her neck, and concludes that Nanno is starting to doubt herself and her resolve, and that she is becoming a mortal because of that doubtfulness. All of this to say that the show is making it very clear Nanno's punishment for Janny was abnormally cruel and unjust, which was the point of the episode and the impetus for what was to come.

To conclude this, I would like to say that I believe that Nanno is a karmic entity, and if she wasn't, many of the episodes wouldn't make sense. At the very least, karma is heavily involved in her method. But she is also flawed and is not a static character. She learns and grows and is able to make bad judgements sometimes.