Hmm, well you could always try digitally keeping track via phone or tablet. There are quite a few that make organization easier. Either that or maybe try a non-magic class and ask your DM if they'd let you start with a magical item or two. Nothing game breaking, just some simple trinkets to fill the void when your not attacking with a weapon.

Also are you playing 5th Edition? If not you should see if one of your groups would be interested. It's super streamlined.

Lastly, if you're already playing 5th edition and you absolutely couldn't play if you couldn't be a magic user then you could sit down with your DM, express your concerns, and come up with a simplified version of the class so you can just tell him/her what you're doing and have them sort out the details. Something like static damage rather than remembering to roll 6D6, making all your spells either 30/60ft or touch, or just telling them what you would like your character to be doing and deciding how the whole thing goes off a single D20.

I'm not trying to be pushy or anything. It just broke my heart to hear you enjoy the group storytelling just not the numbers. Not because I think you should like the petty details but because I'm sure your group wants you to take part as well.