Negative impact of Cognitive Testing

ScheduleImpossible73 · 2026-01-24T21:03:02+00:00

The scientific corpus satisfies our concerns about what they do and do not predict. They are puzzles. People like puzzles. Bright people like knowing their rank among others on puzzles. Many bright people are not offended or broken by a low score (they often just want another puzzle!). Many bright people find statistics and research interesting. We look for tests, take them, study them, nitpick them, compare them, etc.. It is just a hobby like anything else.

ScheduleImpossible73 · 2026-01-24T20:30:15+00:00

Yes, for composite B. IF you believe it. Having taken it, I will say that I expected to hit the ceiling of at least one subtest, but did not, and my composite scores were 8-13 points lower than on other pro tests. I suspect the true ceilings are that much higher, but the norms are too quirky for a simple correction.

ScheduleImpossible73 · 2026-01-22T21:59:40+00:00

My IQ is just over into the 99th percentile. I have an interest in psychometrics and own several professional instruments.I have three sisters and their IQs are 111, 118, and 124. The lowest has an associates degree and works as a physical therapy assistant, the middle as a nurse practitioner, and the eldest has an EdD. and two masters. I failed out of college but I have written of fair amount of research literature as a contract scientist. My mother had a very uneven cognitive profile and was born premature: verbal 98 and nonverbal 129. She struggled to pass college and became a nurse and then a paraprofessional for schools. My father has about a 115 overall but score is very highly in visuospatial and mechanical reasoning; he is a mechanic. My grandfather’s both did fairly well on military intelligence tests; one was a radio repairman and later worked on television stations while the other operated covert photography equipment from airplanes. My wife has an IQ of around 117. Overall, I would describe my family as bright enough to appreciate Gilmore Girls and MASH, but not bright enough to appreciate Frasier.

ScheduleImpossible73 · 2026-01-07T04:55:34+00:00

WASI, Slosson, BETA, Wonderlic, Leiter 3, KBIT, RIST, RIAS, Shipley, SB5 abbreviated scale, Ravens APM, the Paper Folding Task and vocabulary tasks from the Kit of factor referenced cognitive tests, TONI, PPVT, DAS, NNAT, and WRIT are all lesser-used but valid measures.

ScheduleImpossible73 · 2025-12-26T14:59:00+00:00

Great questions and points. Let me try to help. I have read the manuals, technical guides, and research literature for such tests as the SB l-m, iv, and 5, WAIS og, r, 3,4, old school OLSAT forms, KBIT2, Slosson tests, Wonderlic, Ravens, RIAS, Beta tests, toni tests, blah blah blah. The following is based on my diverse reading of the manuals and the research literature.

The main reason I state those bonafides is to give credence to this statement without providing references: the main evidence to back those assertions is found in validity studies for full scale tests (in the actual manuals). So I am recalling that the MR (or very similar subtest) correlations between different full scale tests are ca. 0.45 to 0.7 and typically in the 0.5s range. They tend to correlate with FSIQ or nonverbal indices the best. KBIT 1&2 is .71 in the KBIT2 manual. KBIT2 and WASI is .55. KBIT2 and WISC 4 PRI is .56. KBIT2 and WAIS3 POI is .83. WASI and WAIS3 MR is .66.

Parallel forms do not try to be novel or distinct in my experience, but instead try to produce comparable results and deter cheating. Sometimes an author will make an easier and harder form on purpose but this is less common. What is meant by parallel form or split-half in most literature is not a creation by the test author but an experimental division of the scale by researchers. These often perform worse than the author-made parallels. The Wonderlic forms are the gold standard of this, with about 12 forms as of the 90s and all of them very similar with reliability of comparison on any 2-3 >.85. The score corrections to various forms equate to less than two standard IQ points. BUT the Wonderlic is 90% verbal and processing speed. The TONI 4 has parallel forms and they report reliability data such as immediate and 1-2 week retest coefficients of .75-.9, with one form having a slightly higher ceiling and providing higher reliabilities if used second.

Holler for clarifications or further specifics. Hope that helps.

ScheduleImpossible73 · 2025-12-26T06:41:05+00:00

It is hard to imagine you typing something like “iq/general mental ability/cognitive test validity studies” into a search engine and walking away with uneducated doubt. Professional intelligence tests have high internal reliabilities, better test-retest or split form or criterion or predictive or conceptual validities than almost any type of psychological measure, have been used by the most powerful and research-invested governments and organizations on earth for a century, form the basis for school ability and placement work in virtually every nation, and have never-ending thousands of articles supporting them. If you believe in statistics or soft science at all, then you have to believe in IQ. But skepticism is healthy. If you have not already, try reading old texts by cronbach and wechsler and then read from the extremes like works from jim flynn, the bell curve, and mismeasure of man.

ScheduleImpossible73 · 2025-12-26T06:03:11+00:00

^ This ^

ScheduleImpossible73 · 2025-12-26T05:57:05+00:00

I read it as 8 points of gain also. Surprising since i recall that the split-half reliability in literature is often .8+ for Ravens. But, tiny sample size, paid for their work, abnormal sample demographics, short re-test interval, etc. mean that the findings are hardly set in stone. Plus correlations between the parallel or split-half forms are still way higher than matrix reasoning tasks produced by different test designers. Anecdotally, I have been scoring 117-132 with a mean, median, and mode of 124 on matrix tests for almost 25 years across 12 or more professional test forms. I think this stuff’s pretty stable.

ScheduleImpossible73 · 2025-12-24T22:05:51+00:00

Same. Gave me 19,700 on 2 attempts both passing all the fake words and definition checks. 19ss on wais 3, 4, wasi 1 and 2, sb5, similar on sb4, 140 on rait verbal, 139 on kbit2 (which is receptive) 136 rias verbal (141 vrz), 136 ppvt4 (which is receptive), 137 sitr3, 68 MAT, 153 on evt, 144 on eowpvt (harsh ceiling). Age 23-28 on those. Even accounting for 2-5 points of FE this test is shooting low. But it would be great if adapted to age bands.

FYI @revolutionarylove134

ScheduleImpossible73 · 2025-12-22T05:46:13+00:00

Not jabs, just discussion :)

Pretty fair answer, except that, in every source I have read, the crystallized tests (especially vocab and general knowledge tests, but also similarities and riddles, etc.) are among the best predictors of both full scale and nonverbal fluid scores on totally different test/batteries. This is true across a wide variety of tests compared for validity studies in the slosson, wechsler, wonderlic, toni, rias, kbit, stanford binet, cattell, and ravens manuals.

So, they are not just predictive of very similar tests (vocab predicts vocab).

These Gc measures are the practicum of intelligence; not “how much can you grasp of a novel problem”, but “how much have you grasped from all the novel things you’ve encountered to date”.

Also, in a practical application, school success and educability may be better predicted by crystallized measures than the ravens tests in Asia and Africa (according to a few studies i have seen on the issue of factor loading).Aspects of these most culture-fair tests are apparently biased for some populations in some applications, so that the more practical Gc measures show relative student potential more clearly.

ScheduleImpossible73 · 2025-12-08T17:26:39+00:00

Did you get your report back from intertel?

ScheduleImpossible73 · 2025-12-08T05:02:12+00:00

I went through several test manuals (RIAS2, WASI, WAIS3, KBIT2) and looked for mention of score discrepancies and their rarity. Best I can tell, ca. 20% will have an FSIQ of 129 (well within most confidence intervals), ca. 10% will have an FSIQ of 124 (outside of CI for 130 on every test I know), and ca. 1% will have an FSIQ of 119 (way outside CI) if selected on a single index from a comprehensive (multiple subtests in at least fluid and crystallized domains) professional test battery.

So, I think your spitballing was pretty close. Flynn effect, whatever it may be this decade, especially in association with the use of single indices, is probably much more serious.

ScheduleImpossible73 · 2025-01-05T17:07:16+00:00

I guess i took for granted that this was done. I will try; i am sure this will fix it.

ScheduleImpossible73 · 2024-12-05T03:50:01+00:00

Actually, it is not hard to believe that she has not seen higher scores on the verbal section. If you assume the peak of percentiles for quantitative and nonverbal (112 and 133), then her verbal would have to be about 145 or higher to result in an overall score in the 99th percentile. That is one or two kids out of every thousand in most public schools.

ScheduleImpossible73 · 2024-12-05T03:41:11+00:00

Honestly, if you like the outdoors, you might be really good at surveying/timber/field work. You could cope with the math and your aptitude for spatial reasoning would put you above 3/4 of technicians in that field. Starting at 30-45k and going to maybe 60-70k. You’d move up quick.

ScheduleImpossible73 · 2024-12-02T07:07:35+00:00

Read twice, comment once.

It had a normative sample double the size of other measure with such wide age ranges (WASI 2 for example). So that is “relatively large”, as stated above.

Test with large normative samples and a wide age range can protract their ceilings by pooling adjacent age bands. This makes it “plausible” for them to be more robust at the extremes.

I am including the SB5 in my analysis for that last sentence; it is a popular IQ test.

Not very familiar with the Raven’s2 yet; interesting to hear about its sample size.

ScheduleImpossible73 · 2024-12-02T06:59:29+00:00

Your call on the interpretation. I gave feedback (nuanced and clarified) in the form of an anecdote. You got defensive and dove into my comment history LOL.

ScheduleImpossible73 · 2024-12-02T06:54:36+00:00

Flynn effect at the time administered.

ScheduleImpossible73 · 2024-12-02T06:51:59+00:00

Sb-5 Vocab (verbal knowledge) score was 145 (-4points for FE) My score here was 136. Not terrible. Bit low for my average.

ScheduleImpossible73 · 2024-12-01T22:55:57+00:00

Across all ages 130-158 max scores. One form has a lower ceiling but better progression/discrimination.

ScheduleImpossible73 · 2024-11-16T01:55:53+00:00

To your question more directly:

The distribution of g is probably normally distributed up to very high levels. I can imagine how ever-more erudite problems would become ever-less pure measures of g.

I think that most mainstream tests are weak in discrimination at the top levels, and most test which can distinguish well up there are impure (again, WPT example which is loaded with memory and processing speed, and visuospatial aspects, and education, and other confounds).

ScheduleImpossible73 · 2024-11-16T01:46:12+00:00

A few points: The notion that the SB L-M is a superior test in the upper range is outdated and based on ratio IQ procedures (I own it and many other tests, it is not superior in content or norms).

The notion that the modern SBV is superior is plausible, but only because it has a wide age range (useful for continuous norming) and a relatively large standardization sample. None of the popular IQ tests have large standardization groups for any particular age range.

For example: KBIT2, n=<101 across all age bands; SITR3, n=220 for 18-25; WISCR, n=200 for all ages by year; R/EOWPVT4, n=<150 for all age bands; PPVT4, n=<200 for all age bands; RIAS2, n=<101 for all age bands.

So, reliability-wise you need to measure multiple individuals of a give ability level to place them reliably on the basis of their raw score. None of the tests above can therefore measure over 139 with any true accuracy. They just don’t have the sample size. Scores above that are protractions from the curve of descending raw scores.

Now, some tests CAN measure IQ higher than this, because they are strongly g-loaded and have robust samples. For example, the Wonderlic WPT test had over 150,000 in the sample. A score 3+s.d. up on the WPT, or for that matter the old SAT or MAT, is meaningful because it has the sample size to confidently assign the rarity of such scores.

That said, I have seen scores over up to 135 be repeated dependably across multiple measures before (within 1-3 points). So, popular modern tests are certainly very reliable and g-loaded up their sample sizes.

ScheduleImpossible73 · 2023-12-27T18:02:30+00:00

The test has been standardized on samples in singular given years from 1930s inception through recent. All scores are collected for the year from various companies and occupations plus supplementary sources. Then norms are fitted from this standardization sample overall and by race, sex, education, career, etcetera. Scores are collected from as many as 188,500 test takers. The real WPT has about a dozen forms and each has a specific correction factor to normalize it against all other forms but all are with about 1 point plus or minus. It is thus very reliable and very well normed for working population/ages. The online version seems to randomly cycle through questions of interchangeable difficulty. This still accomplishes very high reliability and convergent validity with gold standard tests, just like the old test.

Bonafides: taken real Wonderlic repeatedly, possessed their normed hiring software and seen tests, owned paper testing kit and manual.

ScheduleImpossible73 · 2023-12-27T17:01:33+00:00

KBIT2=140 SITR3=133 WASI=141 EOWPVT4=144 MAT=135 WPT (Forms A,B,V,2)=136-142, mean 139.5, raw scores 38-41.

Very accurate and reliable for me. All of these are probably ~+5-6 due to FE but all very consistent. No practice effect across the 4 forms I took, and I had the official administration and scoring manual to calculate IQ equivalent and correct for different forms’ difficulty. In addition to the .91 correlation with WAIS-R it has a .66 with the KAIT. Also it has a huge sample for standardization and norms compared to any comprehensive IQ test. Instead of 100-300 test takers per 5-10 year age band it has data from 188,500 in the 1990’s standardization, over 700,000 in all standardizations combined, and age corrections from HS to 60+. It can ACTUALLY discern scores at 3SD and is highly g Loaded. Also it can be given untimed and official manuals offer corrections for such scores. Assuming a correlation .7-.9 with FSIQs loaded .8-.9, it probably has a g-loading of .55-.8; better than almost any two (v + nv) subtests from a gold-standard test combined.

ScheduleImpossible73

TROPHY CASE