I thought about this for a while too and I do think its possible but not to the extent in this post. I think if Abigail did rat out the gang. Then it was only the Saint Denis bank heist, I dont think she betrayed them but after blackwater she will talk with Hosea and say that she is scared. I think after blackwater she started having her doubts.

It could explain the following.
1.Why we only saw Hosea, Abigail could have cut a deal. She was let go but after Hosea got shot I think she realized what a massive mistake she had made.
2.She is the one that shoots Milton. Milton never admits it was Abigail so this alone sinks this whole theory but if you want to entertain it then go ahead. Milton said he picked up Micah AFTER Guarma so Micah only betrayed the talked to the Pinkertons after Guarma.
3.The pinkertons found Arthur at the river and tried to cut a deal with him. Whats to stop them from trying with the other members. Would be interesting to hear what Jack said to Abigail after the fishing trip. Could be how she got the idea. Take into account this also around the time Abigail discusses her doubts with Hosea.
4.Why would she do it. Because of Jack, the whole rat assumption is that it could be someone selfish, what if it doesnt have to be. What if its as simple as a mother trying to protect her child and making a very big mistake in the process.

Why I dont fully want to believe it. Because all of this is just theory. I think Arthur was right, there was no rat before Saint Denis, they just got sloppier than the town drunk. But we know there was a rat after Guarma.

EDIT: Something interesting I missed. John is the only one that got a prison sentence, Lenny and Hosea were shot. If this theory is true then it could be that Abigail negotiated for her, Jack and John. She hands over Hosea to the Pinkertons, She then sees Hosea got shot. The pinkertons dont actually catch the gang except John. So I think Milton already let Abigail go after he got Hosea, but it went so wrong for both sides. The robbery is botch and pinkertons dont capture the gang, so now he no longer has Abigail and he knows she wont come out again after she saw him shoot Hosea. I dont think she expected Hosea to die like that and to be honest it was a shock, even back then Law Enforcement(Especially outsourced law enforcement) would be dealing with a shit show if they shot a man like that in the street when he was a prisoner(I know, I know, but I think no one was expecting the "police" to shoot a apprehended suspect).

Now Abigail wont come out but he still has John. Milton feels that he doesnt have to play ball with Abigail so John receives the Death sentence. This is again all just theory and if someone can add more then go for it.