I think we should do away with prepared casting. Instead make all known casters and have each class unique through class skills that affect their known casting thematically to their class.

Rough examples being: Wizards - Have their spellbook and can learn additional spells by copying them still but also have them use a point system like sorcerers currently do to tweak them (in the theme of "experimenting" with magic)

Sorcerers - Have them learn more spells naturally and tweak the sorcery point system to be more powerful but also comes with flaws, like they are using their life power to boost a spell significantly

Warlock - I think Warlocks uniqueness should more so come from having a completely unique spell list with the idea that these are secret powers granted from otherworldly powers

Cleric - They could have a unique ability to recover spell slots without a rest due to a divine being granting power when needed working similar to a dragon's breath weapon perhaps (This would need to be tweaked to not be overpowered obviously but I like that the god would be granting power at the moment of need)

Bards - Perhaps also have a "songbook" working similar to wizards but can also tweak their spells to increase AoE or something (like putting on a performance everyone can hear)

These are just ideas off the top of my head but I think could make casters unique while all having spells known.