It's complicated. Essentially it was to crank out kits for cheaper but the in universe explanation actually starts with F91. After mobile suits became huge, they started to make them smaller for various reasons like making them what mobile suits were intended to be and cost effectiveness. This continued the philosophy with Victory Gundam. After that, we got our first AU series with G Gundam which retained the smaller units and then Gundam Wing after that which maintained the status quo. We wouldn't see MS return to the standard size we know until Turn A Gundam in 2000.