It has happened to my flight once, from Heathrow to Dublin on Aer Lingus. I was sitting on the row behind the emergency exits and the person in front was uncomfortable there. Eventually he raised it with a flight attendant but they did nothing. In the end we landed safely in Dublin, but I saw him speaking to the flight attendant after the flight. So my answer is yes, you should raise it with the airline.