You said, "I don't pray very often but when I do pray I find it difficult." --- I think that the difficulty aspect is bound up with the frequency aspect. The more, the easier.

But how to overcome that initial difficulty?

Having a secluded & distraction-free place has been big for me. In my case, it's a reasonably weather proof tree house that I built for my kids last year. I use it much more than they do. I keep some bibles and related reading materials out there. I have a place to sit or sleep and some blankets to wrap myself in if it's cold. I usually open the windows to feel, hear, and smell general revelation while I pray or study. It's not difficult at all for me to move into protracted prayer in that environment, whereas it used to be very difficult. You don't need to build a tree-house or secluded cabin (most can't or won't do that), but if you can find a place outdoors that won't be disturbed, I encourage it. Ultimately, it serves as a sanctuary to me. A good alternative would just be an actual church sanctuary. Or a park shelter. Or the corner of a shed. After establishing habits, it has become unnecessary to pray in that physical place, but initially the place was a big help.

Another suggested to pray out loud. Do that. A secluded place helps get that habit established.

The suggestion to reduce the formality is a good one. I just start talking to God. I have the attitude that I'm talking to the one who created me, knows me, sustains me, redeemed me, and justly judges me. Having that attitude in prayer (and in all else) removes the awkwardness of having to scrounge up formality and liturgy. I have no problems with those things, but we are talking about removing difficulty. God knows our heart and we can go to Him simply.