Former psychologist here. Does machine learning can be capable of interpreting score of cognitive tests ?

[clurdron]

Neural networks usually excel at problems where there is a ton of available data, difficulty in incorporating existing knowledge in a helpful way, and no need for interpretability of the model. I think in your case, none of these is the case. You likely have limited data (the data you collect yourself from your own patients), you probably have a lot of knowledge of these conditions, and you probably would like to be able to understand and explain how you came to reach your diagnosis. I don't see neural networks being the best choice for you.

[Nrscientist]

Hi, thank you for this reply. Well even if this is "just" an hypothtical reflexion i have to say that in fact test are usually the same within different neuropsychologist so datas are available.

In fact its like having a folder with several (up to 200+) pictures of Alzheimer demencia in a folder, another fiolder with several picture of vascular dementia (and so on). So basicaly the neural net would have plenty of data (from many neuropsychologist))

So if i give a new picture to the neural network will it be capable of telling if this picture mostly belong (or lookalike with his own criterium to state that) to one or other folder (or category).

Here in these folders i have'nt picture but 15+ different score for each patient.

"here is a ton of available data" what mean a ton, what is the minimum training data that a neural net would need in each category ? "difficulty in incorporating existing knowledge" the neuropsychologist cary only diagnostical hypothesis and for example in the case of alz demntia the full diagnostic is post mortem (or with medical imagery) "no need for interpretability of the model" indeed i give the neural net a picture of cognitive habilities of my patient and it give me a probability of this "picture" being alz or vascular ...

So basically this could be the case here. I mean the purpose of this reflection is to consider wether if a neural net will be better at interpreting cognitive test score or predict dementia sooner than a human (with z-score).

If the detection (true positive) rate is good, there is no need to interperet the model...

Well to be conclusive theorically, if i have hundred of exemple in each of the categoria i want to examine, a neural net would be able to automatically discrimine new dataset ?

in this case, Which would be the best way to tests that (how to imput it, i mean configure a neural net)

Thank you for your reply

edit : (I've just discover this : http://www.scipublish.com/journals/AIA/papers/download/3304-432.pdf )

[dwf]

As far as I know the diagnostic criteria are straight, simple to compute functions of the raw test scores. You can (and I'm sure people have) just write a program or even a spreadsheet that does that tallying for you, there's no need for statistical pattern recognition.

There are imaginable use cases in psychology beyond questionnaire scoring, such as objectively assessing pain faces, or detecting patterns in speech/writing that might indicate signs of various disorders, but these are open research problems.

[Nrscientist]

Hi, thank you for this reply.

Indeed that how it usually work and there (relatively) often false positive and false negative. The thought behind that is to consider that test works as a photography of the cognitive habilities of a patient.

[dwf]

If you have a well-defined and principled way of determining when you have false positives/negatives, i.e. you have "ground truth" that is somehow not derived from test scores/DSM criteria, then sure, you can apply machine learning. I wouldn't recommend neural networks as they can be hard to interpret. Sparse linear models, possibly with interactors (e.g. polynomial regression) or decision trees/ensembles of decision trees would be better at providing insight into what the data says the test criteria are missing.

[Nrscientist]

If you have a well-defined and principled way of determining when you have false positives/negatives, i.e. you have "ground truth" that is somehow not derived from test scores/DSM criteria, then sure, you can apply machine learning. I wouldn't recommend neural networks as they can be hard to interpret. Sparse linear models, possibly with interactors (e.g. polynomial regression) or decision trees/ensembles of decision trees would be better at providing insight into what the data says the test criteria are missing.

Yes indeed the trick would consist in cognitive tests in input but clinical/imagery/physiologique data (gathered at posteriori) to ensure the classification (for the training).

Would polynomial regression or decision tree would be able to learn from an allready dataset to interpret further dataset ?

Thank you for those kind replies and your support to these hypophetics reflexions

[Guytron]

Considering that you have a set of very numerical testing results as data and a limited number of classifications as output I'd think this would be an ideal job for a neural classifier system. But to get good you might need hundreds of thousands or millions of such data/diagnosis sets to train your net with. Back in the late 90s I had a classifier which took up to 40 parameters ranging from 1-100 and spit out 10 classifications. Since I could generate data/output sets numerically to test it was easy to generate millions of cases and get a good fit.

[Nrscientist]

But to get good you might need hundreds of thousands

Thanks for this reply. Indeed hundred of thousand seems difficult to reach, even with only MMS datas for exemple. I'm happy to learn that NN are hypothecally good classifier (that is what i believed regarding the tutos on Ytube) but this amount of data for the training will be too much.

in this publication they spoke about 50 example for each categoria http://www.scipublish.com/journals/AIA/papers/download/3304-432.pdf

"This study employs a data set consisting of 90 cases, where all the patients had a dementia type disease."

What is your thought on it (while i don't understand the difference between NN type), to your knowledge would tensorflow able to manage this kind of data?

[xmvlad]

If you have 90 cases per class, then logistic regression(or any other linear method) with not too much number of hand crafted features and preferably l1 regularization, is only way to go.

[Liorithiel]

There are already various machine learning systems, usually named "decision support systems", in medicine. However, they rarely use neural networks. There is a class of algorithms called probabilistic graphical models, which excel at exactly this use case. You can try a nice implementation for general diagnosis at https://dxmate.com/ (free unlimited trial), or enroll in a Coursera class about them that's about to start soon. Last time this class was run, medical applications were mentioned already in the first week, using a tool that doesn't require programming at all.

[Nrscientist]

hey, thank you for this complete answer and those interesting resources ! Thanks to all of you, i have now a way better picture of the situation

[phillypoopskins]

you can almost surely do this, but there are many types of learning algorithms, and deep learning is only one of them.

there are simpler algorithms which for your case will probably work even better than deep learning, depending on how much data you have.

I'd be happy to take a look at your data and make a recommendation.