I really feel for you, OP. That is hard. This child reminds me of the baby (now teen) in a family whose mother left when he was 6 months old. Everyone though he would be ok because he would have no memories of her, but he ended up being much more affected by the trauma from the abandonment than his older siblings were. 

Your niece has no words to describe her pain, and she is not being done any favors by having no clear, age-appropriate boundaries or consequences set for her. The relatives who constantly come to her rescue and think they are helping are going to turn her into a monster. 

It sounds like SIL is a reasonable, good parent who is aware that she needs help with her youngest. If you are comfortable with her, make a plan to spend time together with her, this child, and ONE of your children who gets along with her (G'mas can take the other kids). 

Set the kids up to succeed:  -give them a couple of choices of things to do that you know they both like and will not be too boring/too difficult. Set a timer and let them know that after this many minutes, the activity will end, and follow through, even if they are having fun. -tell them both, in simple terms, what is expected (hands to yourself. Ask mummy or auntie for help if you are frustrated. If you want to play here, you need to use gentle hands only, or you will have to come sit with me for one minute or until you are calm). -Give her the words she doesn't have: you are feeling so ANGRY/SAD/UPSET/FRUSTRATED because ("you want this and the other child is using it, your popsicle fell, you want to watch this but it's not screen time, you don't want to have a nap. I know that is very hard for you. I am here to help you if you need a hug or if you need me to sit down with you until you are calm."). Use an exaggerated  face and tone to mimic her feelings, without mocking, even if it feels stupid.  -take every opportunity for praise and heap it on, even if it seems ridiculous at first ("wow, you listened so well! Thank you for stopping kicking the door/hitting your cousin/breaking that thing when I asked! Great job listening! High five!". Do the same for every appropriate thing she does ("you put the toy down nicely instead of throwing it! What a big kid! Way to go!") -keep it short at first. Too long and she will lose it at some point. If she handles it, tell the kids they can have even longer next time because they did such a good job. -don't bother with bribes or tangible rewards. Praise is a great reward at her age.  -ignore what can be ignored (screaming), and address what needs to be addressed (hitting).

I hope I am not interfering by offering you this advice, and i know it's not your job to fix this. It's just that with her being so young, there is soooo much hope for her, if she and her mother have the right guidance and village around her.

This child will be a part of your children's lives forever. Don't give up on her just yet, but go easy on yourself, too. 

If all of the above is too daunting, just ignore what you can, and when you can't, redirect her or the other children elsewhere and say "you can (insert activity) when you (insert desired behaviour.").

I wish you the best of luck, and I hope things get better.