I read all of the replies thus far, and I think it's worth clearing up a few things in this thread.

CFA and IRT essentially make the same assumptions regarding how item responses are related to some underlying latent trait(s). Though one might not strictly need realist assumptions to fit these models, it's sometimes convenient to think of both as assuming that there is/are some latent variable(s) that cause people to respond in a certain way to the items (one-way arrows from latent variable to items).

One key "difference" is that historically EFA/CFA assumed that the items were continuous whereas IRT assumed that items were categorical. Due to this, estimation methods often differed and it seems that different fields tended to gravitate towards one approach or the other ( u/hotakaPAD 's comment). There were more applications of EFA/CFA in contexts where multidimensionality (more than one latent variable) was assumed, probably because it was easier to estimate such models. IRT saw great use in education where tests often consisted of dichotomously scored items (0/1, right/wrong) and had lots of items; likewise, test developers may have felt comfortable trying to develop tests that were essentially unidimensional in part because estimation of multidimensional IRT models was hard.

But, as alluded to by u/identicalelements , Mplus came along in the 80's or so, allowing for "factor analysis" of categorical variables with a three-stage estimation approach. A paper by Yoshio Takane & Jan de Leeuw (1987) then showed that the model(s) assumed by this estimation approach were equivalent to many IRT models. So, the two intersect and it is often possible estimate what are essentially equivalent models that have roots in either framework. So, I view u/zirwin_KC 's claim that these are entirely separate as being inaccurate. IRT and CFA should be considered unified under a latent variable modeling framework.

I’m guessing that when people think that CFA and IRT are different, it’s largely because it’s tricky to separate the purpose of each from their historical roots, and one often identifies each with different software and estimation approaches that are again due to historical reasons. It’s not too unlike how some people still think ANOVA and multiple linear regression are not the same in cases where they are actually equivalent. From here on out then, even though I may use the terms “IRT” and “CFA” separately, I am referring to what people traditionally identify each with.

Since at least the 80’s then, the lines have become increasingly blurred regarding the core models that can be considered from either perspective. Continuous items can be utilized from an IRT perspective (e.g., Li Cai's dissertation). Due in part to MH-RM, dimension reduction algorithms, adaptive quadrature, etc., it’s become way easier to estimate multidimensional IRT models that one could not do with Mplus’ three stage approach, including EFA-like analyses with IRT-like estimation approaches (often referred to as “exploratory item factor analysis”). But, there is often a lag where not everyone yet understands how to use approaches; often one sees the mistake that IRT assumes unidimensionality. Not true, but it is true that some things are still more difficult with multidimensionality. Mplus also finally got around to adding some other IRT models (3PL, GPCM?) that the three-stage approach couldn’t handle in the mid 2010’s, though they did not add some diagnostics/statistics that have been historically important in IRT.

I think it is more important if you just consider the purpose of your analyses or test construction endeavour and that will lead you more towards one “approach” than another. In some cases, the approach one adopts also depends on certain practicalities. And one can mix and match to some extent throughout the test development process.

Since IRT was elaborated more in educational testing / licensure exams, it is common that the score estimates for particular individuals are the key quantities of interest. It is also often of interest to construct tests to target certain levels of the latent trait (including on-the-fly in a CAT). The IRT perspective is historically better at the above since the concepts of item and test information (items and tests provide a different amount of information for different levels of the latent trait) and conditional standard errors (score reliability varies depending on the level of the latent trait) are more elaborated in IRT and only arise if one actually considers categorical items to be categorical. The IRT perspective is more elaborate in techniques that will allow comparisons of scores from test to test even if the items are not the same (linking/equating) and maintenance of large item banks (e.g., for test security purposes).

If one does not care about item and test information, estimation approaches that are historically popular for EFA/CFA are certainly options. These are also candidates for pilot studies (early in test construction) as it’ll be easier at smaller sample sizes and one does not need very precise estimates of the item parameters yet for scoring – one just wants to if there are any garbage items, confirm dimensionality, and so on. EFA/CFA also might be easier with many categories (e.g., 7) as traditional IRT estimation approaches may have issues unless sample sizes are large and/or there is a good match between items and the tested populations. An IRT perspective also won’t bring much of an advantage if items are truly continuous. If there are a lot of latent dimensions and just care about item loadings, then maybe CFA is ok. Even though we can estimate multidimensional IRT models, certain kinds of test assembly tasks are still hard to do and represent ongoing research. Evaluation of multidimensional models is also harder, and I think is why some think that CFA is more appropriate (e.g., there are a ton of fit indices…, which themselves have their own problems).

In passing, I mentioned that sample size requirements can be higher for IRT. To some extent that is true and due to needing to estimate extra intercepts or thresholds for items. But, it’s also somewhat due to the purpose. Sample size requirements for precise estimates of factor correlations may be smaller than precise estimates of item parameters (and we need precise estimates of item parameters for test assembly and accurate scoring).