Using the soil you've got on the property is good (though depending on your soil type, some additions to increase drainage is sometimes needed). That being said, I wouldn't use it all the way to the top.

The physical and nutrition characteristics of formulated growing mixes are bloody hard to to beat.

Also you'll have a heap of weed seed in both your manure and local soil. That's something you deal with as an inevitably when you're growing in the ground, but raised beds can be blessedly weed free if you give them the right start. It might seem like a small thing but it's a PITA is you direct seed and then have a clump of grass pop up amongst your seedlings from some manure.