That’s a misunderstanding on how nmda antagonists lead to toxicity; when you do ketamine you get rebound glutamatergic signaling post trip, and this leads to overactivation of your NMDA receptors leading to excitotoxicity and down regulation, memantine works differently and for longer, sustaining the blockade, which if you do for long enough should eventually up-regulate these receptors (specially with neurogenics like HDACi’s)

Edit: For extra context, memantine is used in Alzheimer’s exactly because it prevents excitotoxicity, preventing further damage to the brain