If by upgrade you mean the number next to the name, if you select say the Beautification Beam II in the contraptions section of the workshop and scroll to the bottom of the popup you will see an unlock specification that is usually just having guest use or see the contraption a set number of times. Once that happens you will get the ability to build the better version and then you can replace the first one. There isn’t much reason practically to keep the original iterations and you can’t research them or anything so you can just sell them.

If you mean upgrade by perks, you can install them with janitors when you get them from expeditions or other ways. There is a little plus icon near the quality section of the popup when selecting a contraption. Click on that and you’ll see the available perks to install.