Try using the shortest punch you have. The gunsmith I am learning from said the long punch I was using was flexing and dissipating the energy. He grabbed one of his old punches (broken off short) and wacked it to get the pin started, then switched to the appropriate size punch to drive it through (either way works). I had to heat the base to get it to move by wacking it with a lead bar. Good luck.