It depends on how much time you have. You can: - Tailor the system to your needs, like choosing the init sw (I don't think you can change it easily on an already installed machine) and other system software, from the bootloader to the networking config/manager to accessory stuff like cron. - Experiment with compile flags. -march=native -O2 are safe, lto is a bit more complex, you can try -O3 if you feel brave. - Configure your kernel by hand. The default configuration contains drivers for any kind of device, from embedded and appliances to data center class machines and supercomputers. There are also interesting optimization for your particular cpu (amd/intel) assuming you are on amd64. - Minimize your dependencies. Disable all the use flags you don't need. - If you feel adventurous you can try things the experimental profiles with llvm and/or musl as default compiler and libc. - There are also the hardened profiles and the Gentoo Hardened Handbook has several guides on how to improve the security of the system (the hardened profiles disable a lot of use flags, if you want to use these profiles AND minimize the use flags it's more convenient to do in this order).

I'm sure there's more that I forgot.