DTU will make it easier to analyze unique 3D images

Researchers and the industry have high expectations for the research facilities MAX IV and ESS in Lund in Sweden, where materials and systems can be examined down to the atomic level in 3D in the search for new discoveries in materials, medicine, and the environment.

With a grant of DKK 11.5 million from the Novo Nordisk Foundation, DTU is now starting to develop a standard procedure for how images taken on e.g. MAX IV can be analyzed using artificial intelligence. The work will take place through a new research infrastructure called QUAITOM.

"Today, the data analysis of 3D images often becomes a research project in itself, and it takes 10-100 times as long as everything else in the experiment. This puts a limit on how much benefit to receive from a research facility like MAX IV because data is not utilized as much as it could with better analysis techniques. We want to solve that problem,” says professor and section leader for Visual Computing at DTU Compute Anders Bjorholm Dahl.

He refers to the potential in the field of life science. In recent years, a strong research environment has been built up around 3D imaging, especially in life science with biomedical applications. Examination of tissue samples with 3D imaging makes it possible to see and understand structures very close to the structure that the tissue has in the living organism from which the samples originate. In addition, the tissue can be subsequently examined with histological techniques using light microscopy or electron microscopy.

The timing is perfect, as MAX IV will start research at two beamlines in 2022. The Danish DanMAX handles everything from life science to building materials, as well as ForMAX, adapted biomass experiments. In addition, the sister facility ESS, European Spallation Source, will also be ready for use soon.

Complicated 3D imaging
Artificial intelligence based on machine learning and especially deep learning has become the dominant approach to computer vision.

The basis for doing deep learning based on computer vision algorithms is that you have large datasets with thousands of images.

The researchers then develop the algorithm by showing it all the examples that it must learn to recognize, and when it has seen enough images, you can make something similar and make the algorithm recognize what the image contains. If there is a lot of variation, maybe millions of images are needed to make a usable algorithm. Therefore, areas where it has been easy to obtain data or where research groups have made large data sets have been studied in particular.

Data from 3D imaging is differently complicated because data lives in three dimensions, and you can extract many different types of information about the size and shape of the structures from the same image data. In addition, these are unique images where the researchers want to study the microstructure of materials, which they have not normally studied before, and they do not have access to images on which they can train the algorithm in advance. Typically, it will be a case where you look for something new and specific. But the variation within a data set will be relatively small.

“On the one hand, we have the problem that within 3D imaging there is a lack of methods for making effective analyzes. On the other hand, we have the research field of machine learning and deep learning, which works with relatively general analyzes that can provide very accurate information about the content of the images. We believe that there is a need to connect the two worlds and develop image analysis algorithms that can be trained with much less data,” says Anders Bjorholm Dahl.

Great potential in life science
The advantage of using imaging with X-rays or neutrons to examine the properties and structure of materials is that the samples remain intact. It has a wide range of applications in technical science and natural science as it makes it possible to record an image, change the sample, and record again. It is also possible to subsequently examine samples with other techniques.

Among other things, DTU has been involved in 3D-life science projects on understanding the brain's microstructural organization, how Covid-19 affects blood vessels in the heart, how peripheral nerves are affected by diabetes, and how muscle cells change in paralyzed patients.

“With MAX IV and ESS, we have two of the most advanced microscopes in the world, which makes it possible to make completely extraordinary experiments. It is still very complicated to analyze data coming out of such experiments, so it is crucial to be able to standardize the analysis of 3D images. A collaboration between researchers who create new AI-based image analysis methods and imaging researchers is unique, and holds great potential for world-class research results,” says Anders Bjorholm Dahl.

DTU's research becomes more visible
The data analysis platform will be a large computer that researchers can log on to and visualize, analyze and store 3D data. Users will be both people who make analyzes of their own data and people who will test their own algorithms on a data set provided by the platform. Including data for competitions in machine learning, where users must develop algorithms that solve specific problems.

Via the platform, DTU will also educate PhD students through workshops and PhD summer schools and, in the long term, master's students and others interested in 3D image analysis tools.

Finally, Anders' own Visual Computing research section will have a number of machine learning-based algorithms implemented on the QUAITOM platform, which people from outside can use to analyze their own data.

"It's quite amazing. When you as a researcher make an algorithm and get it published in a scientific article, then you can just hope that others will use it. With the QUAITOM platform, we get the muscles to grab the users themselves and take them to the methods we have created. Our research will then be used in a completely different way than before,” says Anders Bjorholm Dahl.

The grant from the Novo Nordisk Foundation extends five years.