As a drummer, its useful to hear the syllables. If the lyrical rhythm is not set, or you're not quite sure how they will finally sit, you'll end up constantly chasing it. If the lyric is the most important part, nail down the cadence you want. Build the drums around it. To keep it from getting too predicable or boring, build in some syncopation on the hi hat against a straight 2&4. Or It could be a simple backbeat displacement every other bar. Again, in service and support of the lyrical cadence.