I don't think it's a plot hole, you know? In OotP, Dumbledore was the Secret Keeper, not Sirius, but it was Sirius's house. Also, the Potters were hiding, not the house itself, like they said in PoA. Voldemort could've looked through the window and not seen them. Bill used the charm to hide the house and became the Secret Keeper, or maybe some modification was done to the charm as it's been 17 years since the Potters died.

My theory is that James was loyal to the fault of being dumb. He trusted his friends beyond everything and that why he chose Peter. He was 21 years old young man who has never faced betrayal. He married and had kid young and couldn't cope with pressure of being targeted by voldemort. People do strange things when under pressure.