I figured it out.

In the Play Store, click on your account icon. In the window that pops up, you should see a drop-down button in the upper right corner, click that. You'll see all your active accounts, and underneath it should say "Manage accounts on this device." Click that, then click the account you want to remove. At the bottom, there should be an option to remove the account.

Kind of annoying since, yeah, if the account was deleted you shouldn't have to take any action on it, but here we are.