Multivariate Statistics in Image Analysis

Food science

Opening example on application of multivariate statistics in food science: The fermentation process of salamis was monitored with the Videometer multispectral imaging system using 19 different spectral bands. The images in the first column are from days 2 and 42 after production shown as false colour composites based on three spectral bands (660 nm, 470 nm, 435 nm).The next column shows the meat and the fat phases at day 14 found by means of canonical discriminant analysis. Column 3 show images from day 2 and day 42 using a statistical meat colour scale designed to enhance fermentation stages. The darker blue is fresh meat, whereas yellow and orange represent darker red, fermented meat. The last column shows the coefficients for computing respectively the discriminant function and the colour scale values.

For more information see Camilla H. Trinderup, Flemming Møller, Anders Bjorholm Dahl, and Knut Conradsen (2018): Investigation of pausing fermentation of salamis with multispectral imaging for optimal sensory evaluations. Meat Science, vol. 146, pp. 9-17.

 

Multivariate statistics

Multivariate statistics deals with development, computer implementation and application of methods related to the analysis of data where more than one attribute or variable is available for each observation taking into account the relations between these variables. Typically, the data are related to physical, chemical, biological, other natural, economical or societal phenomena.

Traditional multivariate statistical methods include test theory, regression analysis, discriminant analysis and classification, principal component analysis and other orthogonal transformations. We also work with iterative extensions of some of these methods and more computer intensive statistical learning based extensions such as kernel methods, binary decision tree based methods, ensemble methods, artificial neural networks, machine learning and deep learning.

In the context of image analysis and image processing more specifically, the analysis of colour and multispectral images almost inevitably leads to the use of multivariate methods. Multivariate statistics (MS) or statistics is the concept of describing what we observe taking into account the randomness of observations and all their inter-dependencies. In the section, multivariate statistical methods have been used in designing tools for enhancing relevant properties in such images. Examples are multivariate alteration detection using (iterated) canonical correlation or mutual information methods, mapping of changes in polSAR images using complex Wishart distributions, and monitoring food fermentation using color scales based on canonical discriminant functions and kernel versions of other orthogonal transformations, here principal components and maximum autocorrelation factors.

A substantial fraction of the applied research in the section addresses problems on designing diagnostic tools where the proof of concept will involve quantification of uncertainties and deviations from state of the art that necessarily will involve statistical modelling.

A list of application areas is nearly endlessly varied and include

  • food science,
  • materials science,
  • medical applications,
  • industrial applications, and
  • remote sensing/earth observation (related to mapping, mineral exploration, geology, agriculture, forestry, environmental monitoring, marine biology, oceanography, geodesy, and security).

Computer vision is one the core research fields of the Image section at DTU Compute. We aim to develop fundamental methods that allow for fast, accurate, and precise detections and measurements of the real world. Our area of focus spans all of object geometry, optical properties, lighting environments, as well as sub-resolution micro-geometry. The porpose is to be able to record the full digital twin of a natural scene by taking into account the interactions between light and material.

Highlights with links to original papers and other material illustrated below include

  • change detection in remote sensing images with fast computation of eigenvalues,
  • sea ice mapping in Greenlandic waters based on Sentinel-1 and AMSR-2 data by means of deep learning,
  • in-vivo dosimetry, and
  • fibre directionality.

Change detection in Earth observation data

Multispectral Optical Data

Here we use Sentinel-2 data to detect change between two time points. The figure below shows the Tubbs Fire north and northeast of Santa Rosa (immediately above, i.e., to the south of the 101 sign), California, October 2017 seen from north looking towards the San Francisco Bay (with a part of a larger fire around Kenwood in the Sonoma Valley - route 12). Change variates are shown as RGB, burned areas are dark green (built-up areas), lighter green (mostly wooded) and bright yellow (mostly non-wooded), other non-fire related change mostly in blue (for example near Calistoga in the Napa Valley - route 29), and pale yellow (south of Santa Rosa).

For more information see original IEEE Transactions on Image Processing paper, 2007 and ESA Big Data from Space paper, 2019.

 

Synthetic Aperture Radar (SAR) Data

Here we use Sentinel-1 data to detect change between two time points. The figure to the left shows the Frankfurt Airport, Germany, as depicted in Google Earth with aircraft, cars and ships coming and going (red means going, green means coming, yellow means we can't say but change is statistically significant).

For more information see original IEEE Transactions on Geoscience and Remote Sensing paper, 2015.

 

 
Eigenvalues of Hermitian 3x3 Matrices

 

The figure to the right shows a small area around Tjele Gods near Foulum, Denmark. The top row shows the three eigenvalues of the complex (Hermitian) covariance matrix of airborne multilook EMISAR data at two time points as RGB, the middle row shows change images (dark is no change), and the bottom row shows a measure of direction of change (red means going, green means coming, yellow means we can't say but change is statistically significant).

 

For more information see original IEEE Geoscience and Remote Sensing Letters paper, 2019.


Automated sea ice products (ASIP) by means of deep learning


Here we use Sentinel-1 and AMSR-2 data to detect sea ice concentrations in Greenlandic waters. The figure to the left shows a probability map for sea ice coverage in a small area near the Greenland coast (blue is low probability going over green into yellow for high probability).

For more information see

Statistical modelling - in-vivo dosimetry

The behaviour of a novel soft tissue marker, that functions both as fiducial marker and as an in-vivo dosimeter, was validated through statistical modelling. In the image we see the x-ray contrast in the CT images and the activation in the PET images, that makes dosimetry possible.

For more information see paper in Nanoscale, 2016.

Statistical modelling - fibre directionality

The focus of the work described in the sequel is on developing statistical image analysis pipelines to characterise fibre geometry in unidirectional (UD) fibre reinforced composites at the micro-scale level and to provide experimental data enabling modelling of composite behaviour at different loads.

In the image below we compare X-ray CT (XCT) at different resolutions to Synchrotron Radiation CT (SRCT), optical microscopy (OM) and scanning electron microscopy (SEM) (left). The marginal distributions and correlations are depicted through histograms and scatter plots (middle). The spatial dependence of the size is modelled by regression analysis (right).

 

As part of investigating the sudden compressive failure of UD fibre reinforced composites at loads well below their tensile strengths we combine fast in-situ X-ray CT with advanced image analysis to capture the changes in fibre orientation in 3D during uninterrupted progressive loading in compression of a glass fibre reinforced polymer. Below we see – prior to load - all fibre trajectories in a region of interest, to the left a 3D side view and in the middle a plan view. Fibres completely aligned with the z-axis will in (b) appear as a point, misaligned fibres as a projected curve. To the right we see histograms for the fibre inclanations for different loads (0, 200, 600, 895 Newton).

 

For further information see

Contact

Allan Aasbjerg Nielsen
Emeritus
DTU Compute
+45 45 25 34 25

Contact

Anders Nymark Christensen
Associate Professor
DTU Compute
+45 45 25 52 58

Contact

Knut Conradsen
Professor emeritus
DTU Compute
+45 45 25 34 16