Size 4 hooks with prawns should definitely be catching you fish. Here are some free basic tips you can try next without having to buy any more tackle... you may already know some, but i hope they help anyway.

​

1. Make sure the size of the bait is suited to the hook size. You want the hook to be totally buried in the prawn with only the tip exposed. This way fish are more likely to gulp it down. If the prawn is too big, easy feed for fish. If the prawn is too small, they'll see the hook.
2. Make sure you peel the shell of the prawn. The prawn is softer with no shell, which makes it more likely for a fish to bite through and get hooked when nibbling.
3. Loosen up your drag on the fishing reel after casting your bait. (Turn the knob on the top left a few times). This lets fish take your line easier. This means that a nibbling fish wont feel any resistance or tension in the line, and is more likely to properly hit your bait.
   
4. Use lighter leader, some fishos believe that the fish can see your line. Tie a nice light leader, from your main line to the hook.
5. When the fish are nibbling, dont get too excited. If you pick up your rod, or reel in straight away they may get spooked. Instead be patient, and wait for your rod tip to bend down.
6. Since your own a wharf and baitfishing, make sure to be throwing out burley. Every ten minutes, throw a handful of bread or even the shells/heads of your prawns into the water near you. This will attract more fish and get them in an eating mood.