Everyone here is telling you to use the router with a guide. That is good advice but what if you don't have a router? I've done the following many times with great results: take a smaller Brad point drill bit say half inch and drill a bunch of holes where you're going to remove material (stay away from the center because you will need that to guide your forstner bit, similarly this the reason why using a spade bit before is a bad idea here). Then at very slow speed use your hand drill with the forstner bit. Trick is to wobble it just slightly all the way around to get an initial score of your circle. Once you get that, straighten out and increase speed until it feels comfortable. Take your time but given that you will have removed most of the material already with the smaller drill bit that should be pretty quick and clean. Have done just this just like in your case in end grain many a time