DTU algorithm analyzes patterns in eye movements in patients with neurological and psychiatric disorders and helps in diagnosis.
The ability to make eye contact with other people is an important part of our social interaction with each other. Among other things, we read joy, trust, and interest through eye contact, but we also look at the nose, mouth, and the other elements of the face in a dynamic process.
Researchers have found that people with autism and other neurological or psychiatric disorders tend to look at faces in a slightly different way and for these people, it can be difficult to keep eye contact. It is possible to measure these differences using sensors, the so-called eye-trackers. Researchers from DTU have found a new and - in their words - more general way to examine data from eye-trackers.
"Our data-driven method is based on machine learning and can contribute to the diagnosis of patients and to determining whether a treatment or - further down track - newly developed medicine has the desired effect," says Paolo Masulli, a former postdoc at DTU Compute and now employed by iMotions, who has been involved in developing software for data analysis and modelling of the biometric data.
The research result is part of a recently completed project supported by the Innovation Fund Denmark, and it was published a few weeks ago in the international neuroscience journal Cortex.
Model analyzes heatmap
When patients are examined for neurological or psychiatric disorders using eye-tracking, researchers often present the patients with a series of photos or videos with faces on a screen. The eye-tracking sensor then tracks the patient's eye movements and keeps track of specific areas in the images. The results can be seen on a so-called heatmap; The amount of time patients spend looking at certain areas of the image will affect the colour or the heatmap.
"... There is no exact science that defines the size and location of these fields in the images. There is a lot of subjectivity in it. Our method is different. We do not define specific areas in advance, but let the data speak and our method thereby provides us with a more objective assessment of the patient's eye movements."
Paolo Masulli, former postdoc at DTU Compute
“Researchers typically define and frame which areas are of interest. However, there is no exact science that defines the size and location of these fields in the images. There is a lot of subjectivity in it. Our method is different. We do not define specific areas in advance, but let the data speak and our method thereby provides us with a more objective assessment of the patient's eye movements,” says Paolo Masulli.
DTU has had access to eye-tracking data from 111 outpatient psychiatric patients, which the Swedish university partner, Gillberg Neuropsychiatry Center, has recorded. Patients aged 18 to 25 showed symptoms of autism, depression, or ADHD and wanted to participate in the research project and make their anonymized data available for research.
In the trial, patients answered some standard clinical tests, which place them on numerical scales according to the severity of their symptoms. They were then presented with a series of black-and-white images in which the person in the image expresses joy, anger, or looks neutral, while the eye-tracker collected data from the entire image. For example person 1 spent more time looking at the left eye, person 2 looked spent time looking all around it. This has resulted in the generation of a heatmap from each patient.
Subsequently, all heatmaps were analyzed using machine learning, where the most important components (points on the face) were statistically identified from the entire data set and thus without selecting data from specific areas, as the conventional method does.
The conventional method; hand-drawn areas of interest. Credit: Attribution 4.0 International (CC BY 4.0) / Cortex: ’Data-driven analysis of gaze patterns in face perception: Methodological and clinical contributions’.
The researchers then used the components and numbers from the clinical tests for the symptoms to set up a mathematical model that links the components to the degree of autism, depression, and ADHD. Based on the model, new data-driven heatmaps have been created that correspond to a specific level of the clinical tests.
Research has shown, among other things, that a person who does not have autism or only a few symptoms will typically look a little more at the left eye, nose, and mouth. A person with autism, on the other hand, will typically look more at the right eye, part at the forehead, nose, mouth, and elsewhere.
"Our method confirms the trends that recent studies have found and shows that machine learning can be used in neuroscience research," says Paolo's supervisor, Associate Professor at DTU Compute Tobias Andersen.
Helps in diagnosis and treatment
There are several benefits to the data-driven method. It will be able to be used in clinics, where doctors often meet patients who have not already been given a diagnosis. Implementing the algorithm in eye-tracking software will enable it to show how the patient ranks on the scales of autism, depression, and ADHD.
Often there is also a question of comorbidity, where a patient has more than one psychiatric or neurological disorder. Based on this method, doctors can see how patients are ranked on the scales and treat with medication and training based on that particular combination.
Thus, the project has provided a better understanding of what characterizes patients with neurological and psychiatric diagnoses and helps therapists in making the diagnosis, says Tobias Andersen:
“Some people with autism want to get better at social interactions, and here the method can be used to detect eye movements and see how the patient is affected by his or her autism and give him or her automatic feedback. And, if you train patients to have an easier time looking at faces, you will be able to use the method to see if the training works. ”
Swedish partner: An important advance
The research result contributes significant new knowledge, according to the Swedish partner, Gillberg Neuropsychiatry Center at the University of Gothenburg:
“Most of the research done using eye-tracking has used arbitrary, experiment-defined areas of interest. Our approach constitutes an important advance, because it is objective, data-driven, and it enables the application of a dimensional (multilateral) approach to the study of neuropsychiatric conditions and their correlates in terms of gaze behavior,” says Nouchine Hadjikhani, Professor at the University of Gothenburg and Harvard Medical School.
Gillberg Neuropsychiatry Centre points out this new approach also can generate innovative hypotheses:
“For example, we observed that a reduction of a left visual field bias is associated with depression. Whether this is this a state, or a trait, and whether this could be used as a biomarker for the follow up of treatment effects related to depression is one of the questions that could, for instance, be explored in future studies,” says Nouchine Hadjikhani,