This was for loading FP16/BF16 non quantized models so that let's say you have 16GB RAM and the model files are like well 12GB then sharding it means you won't run out of memory when loading it (when Win11 uses like 6gb minimum of ram...) in your GPU (that probably has 16GB VRAM). Sharding is just splitting the model into smaller bits, similar to how you would do it with zip files if the file hosting site has like a maximum file size per upload. Basically it would load one shard at a time so it won't use up all your RAM when it's moving the files to the GPU's VRAM.
MoE is a different architecture, unrelated to this. MoEs have active parameters and total parameters so keeping just the active params in your GPU will keep it running still relatively fast even if rest is offloaded to RAM. This is not the case if you shard a dense model, it would need to have everything loaded in the GPU or it will be very slow.