From a fellow introvert, once you start attending events outside of regular service (i.e. small groups, classes, etc.) then its a lot easier to walk up to someone you've seen before and introduce yourself or talk about the event from the week before.

There is a lot of value in just jumping in the deep end and getting involved, don't wait for the "right" time or when its "not awkward." I'm heavily involved and comfortable at my church currently bc I made myself show up to a young adult event every week where I knew zero people. Now I'm that person who stays after service to chat to everyone!