Congenital Heart Diseases (CHD) are the most common birth defects. Based on numbers from [1], [2], between 64 and 120 potentially life-threatening major CHD diagnoses are missed each year in Denmark. This is largely attributed to the inherent difficulty of the scanning task, which requires a lot of expertise. The challenge comes from the fact that only a single plane through the body can be viewed at any given time during a routine examination, as shown in the picture above.
While there is a possibility to acquire a 4D (3D + time) reconstruction using the industry standard Spatio-Temporal Image Correlation (STIC)[3], it requires an expensive probe and is obtained from one side, making it susceptible to various artefacts.
However, neural rendering and Neural Radiance Fields (NeRF) [4] in particular, have been shown to produce high-fidelity rendering of novel views when trained on a sparse set of images. In this approach, a 3D scene is represented by a continuous function, approximated by a neural network whose input is a query coordinate e.g. (x,y,z) position, and the output is density and colour at the given point. This representation can be sampled, and with volume rendering techniques, it can be converted to an image. The network is optimised by minimising the error between images captured from known poses and rendered ones.
Inspired by the success of this technique for natural images, this project aims to transfer it to ultrasound images.
However, this comes with a multitude of challenges:
• Adaptation of NeRF to a different imaging modality
• Estimating the ultrasound probe position with respect to the scanned anatomy.
• Fetal and cardiac motion has to be accounted for.
• Ensuring reasonable performance. The training of the underlying neural network should finish before the examination is completed.
Developing a unified system that can handle the challenges listed above would lead to a higher quality 3/4D visualisation of anatomies scanned with an ultrasound machine without requiring an expensive probe. This would make it more accessible and, hopefully, lead to a better diagnostic rate of congenital heart diseases.