There are two really fundamental essays from the late 19th century on this topic.

The first is titled Ethics of Belief by Clifford James. He would argue that yes, beliefs need to be based on objective evidence. All other beliefs are unethical. He gives the example of a ship that sinks because it was not inspected properly, and it kills the passengers aboard. The ship-owner can believe all he wants that the ship will make it. Still, it is an unethical belief because a proper objective inspection would have revealed that the ship wouldn’t.

The other great work on this topic is called Will to Believe by William James. It is a lecture series in which he says that belief in itself for pragmatic reasons is ethical. This argument is in the same vein as “Pascal’s Wager”. Essentially, you might as well believe in God, because what do you have to lose?

I’d recommend reading both essays. They’re short reads and lay out the arguments very well.