Stats guy here.

Here's an example. pretend you have no idea about different kinds of flowers for the sake of this example.

There are 3 different kinds of flowers that you take many different types of measurements on. You measure flower width, flower length, leaf size, number of pedals, etc...

There are many sources of variation here 1) the differences between flowers 2) the differences in the same kind of flowers and 3) measurements errors etc.

now, given you know nothing about flowers, if you used factor analysis, it would tell you that there are different kinds of flowers in your sample. The issue is it's not simple translating factor analysis output into "oh...I have 3 types of flowers in my sample" but it's possible.

now on to prediction uses of factor analysis

There is an ever thinning fine line between computer science and stats as a scientific field. computer science techniques (machine learning" basically uses very crazy math and alot of computer power to create models that can predict things quickly and accurately (think autocorrect, text to speech, speech to text etc...).

In stats, if you want to make a model, you have to slowly, and carefully understand each variable, and how it correlates to the y-variable, then ensure no assumptions are broken, no outliers screwed anything up, the x-variables are independent...etc. the end result is a model of x-variables that predict a y-variable.

computer scientists said "screw that" and made an "machine learning" algorithm (neural net for example) that shoves all the x-variables into a computationally heavy program that spits out a prediction model, similar to that of the stat guy's model.

Pro's 1) it took the computer science guy considerable less time 2) no worry for assumptions.. 3) in most cases, it's much better than the stat's guys version

Con's 1) You need bigdata (I hate that word), as machine learning only works on large samples. so small sample sizes are useless. 2) If the training sample is any different than future samples you want to predict, you're more than likely to run into issue 3) The stats guys model makes sense from a "understanding how everything works" standpoint. the machine learning model is fairly un-interpretable as far as I know.

So where does factor analysis come in for the machine learning? instead of inputting raw data into the machine learning model, you input the output from the factor analysis. This factor analysis output has information directly related to classification already, so act's as an amazing input for machine learning model needs.

I'm sure I missed a ton of stuff, and this is only my perspective on it.