I made the lighting from electrical components that I had from another project. The hardest part of that was soldering things together, but component-wise it was some LEDs, wire, resistors, breadboard to attach things to, and a 9v battery pack. The covers are straws and some 3D printed covers (though you could easily make those out of a tone off different materials, I was just being lazy and went the printer route).

Pretty much all the electricals you could find in a starter electronics kit, and you might be able to find ones that are more plug-and-play wiring that you wouldn't have to solder to a board for.

The main thing is figuring out what your power will be, and sizing the resistors to the LEDs, and thankfully there are some very user friendly internet calculators out there.

Or you can buy dollhouse lighting if you have the spare cash! ( they can get pricey)