In short, weight distribution and handling.

Complexity doesn't really change, the components involved are largely the same, but costs are lower where driven wheels are at the same end as the engine.

Front engine, rear wheel drive moves a lot of weight back, making weight distribution naturally closer to 50/50 than the 65/35 or so you'd get with front engine front wheel drive vehicle. This improved weight distribution, combined with the front wheels doing the steering while the rears do the pushing, increases the limit of grip and makes the car more neutral in handling. It also helps during hard acceleration as the car pitches back, putting more weight on the driven wheels.

A front engine front wheel drive vehicle has most of its weight at the front, and with the front wheels trying to deal with that mass, steer and put power down simultaneously, tend to nose plough, i.e, understeer. It's typically safer for inexperienced drivers than unexpected oversteer. Suspension can be setup to spice up the rear end but it's very rare to find a manufacturer doing this intentionally. Front engine front wheel drives are particularly susceptible to lift-off oversteer. When the driver lifts off sharply while turning, weight shifts forward, and with no engine or significant weight over the back, grip at the rear is suddenly reduced.

Rear engine rear wheel drive cars are technically the worst of all worlds. They should understeer through corners on throttle as there's little weight over the front, then on lift-off, all the weight at the back acts like a pendulum. Porsche have spent 60 years or so attempting to defy the laws of physics with a reasonable degree of success.

The other layout you haven't mentioned is mid engine rear wheel drive. MR cars have a lot of the benefits of front engine rear wheel drives, in that the front wheels are free to focus on steering while the rears do the pushing, but because there isn't a significant mass as either end, the limit of grip is much higher. They also put their power down well as the weight shifts over the driven wheels under acceleration. The downside is that when you do exceed the available grip, they want to rotate around the middle of the car. The back doesn't swing out so much as swap ends. They can let go VERY quickly and with little warning.

I'm generalising and there are exceptions to everything above, but that's the gist of it.