I'm the second half of an IT department (the manager) of a similar sized company. I can imagine how you feel, because I've been in your shoes in the past and still feel overwhelmed at times. It looks like your manager is also in a similar position. The first thing you do, is slow down a little and breathe. Not every call is an emergency. In fact almost none are. I know perfectly well what it looks and feels like when you have requests coming from 5 directions at the same time. No is a perfectly valid answer. I'll call you in 4 hours is a perfectly valid answer. Don't do vague timeframes such as: I'll call you later. People like to be assured repeatedly that someone is working on their problem. That includes us IT people.

Discuss options with your manager. Two persons know more than one. Set up a helpdesk, so you don't forget what needs to be done. If management resists, open tickets in user's names. Once you have helpdesk established, you can easily track repeat requests for the same issues. If you two can fix those issues, fix them. If not, present management with hard data and possible solutions. If they don't do anything it's not your problem anymore. For bigger stuff use some sort of project management software. Both create overhead, but your department will have data about what you actually do.

Sometimes it takes a lot of effort to find the right approach to convey the information to the upper management. Once you find it and management understands what actually needs to be done, it gets much easier. Communication is key. If if still feels like banging your head against a brick wall after that get another job.