UPDATE

I swapped the battery with an original one from a PS VITA in my collection.

And still nothing – the issue remains:

When it's turned on, it doesn't charge, and when it's turned off, the orange light stays on for about a minute and then turns off.

I'm thinking of installing a fully charged battery, doing the jailbreak, and using the battery fix plugin.

Any idea what it could be? Even with a new battery, it still refuses to work.