Across hospitals, medical image analysis is gradually gaining ground as a support tool for clinicians and radiologists in their work with X‑ray images, ultrasound, and MRI and CT scans.
Start‑ups within the field are emerging, and the research area has a strong presence at DTU Compute where almost 40 researchers in the Visual Computing section work within the field. DTU is also part of Technical University Hospital of Greater Copenhagen (TUH), where there is extensive collaboration on image analysis.
Earlier in March, we had the pleasure of hosting Stephen Pizer from the University of North Carolina. He wrote what is considered the world’s first dissertation in medical image computing and has a long‑standing history of collaboration across computer science, mathematics, statistics, and healthcare.
Since then, he has refined a set of shape‑based methods that help clinicians understand how anatomical structures vary and change over time. These tools support better diagnosis, improved treatment planning, and deeper insights into disease processes.
"Diagnosis frequently depends on shape. Clinicians often use very simple measurements – a length, an angle, or a handful of numbers – which are few, imprecise, and extracted with error. The result is poor diagnostic accuracy. The methods I have developed allow far more sensitive diagnosis. There are many applications, but the core reason to study shape representation is to support better diagnosis, better treatment planning, and better biomedical research, including understanding disease processes,” he said.
At DTU Compute Professor Aasa Feragen said, it has been a great pleasure and professionally very interesting to host Stephen Pizer:
“Stephen Pizer has supported me greatly as a young researcher, and it was a pleasure to see him here at DTU for the first time. From a professional perspective, it was also extremely interesting to discuss the applicability of open-source tools and the direction in which medical image analysis is heading.”
Workshop and talks at DTU
Stephen Pizer has known Professor Aasa Feragen from Visual Computing at DTU Compute for years and was already familiar with several other DTU researchers.
That personal connection – combined with DTU Compute’s strong environment for method development – convinced him to reach out and ask whether he could visit.
During his stay, Pizer gave a talk on the intersection of geometry, imaging, and statistics, and he led a hands‑on workshop demonstrating a collection of shape‑analysis tools in practice. The workshop showed how to use the methods in both 2D and 3D (X‑ray, ultrasound, MRI, or CT), moving through object extraction, representation, and analysis.
He even found time to give additional disscussion about his career.
“You asked how I got into this field. I realised early on that what I call ‘medical driving problems’ – real clinical needs – depend fundamentally on the shapes of anatomical structures. My entry into this particular area began with a very bright doctoral student who told me that someone in engineering had proposed a promising idea for representing shapes, as I now understand, including its interior. The student wrote a dissertation on it. I saw its potential and also discovered its limitations. Over the next 45 years, I refined the method into the powerful approach that exists today,” he said.
From nuclear medicine to 3D
“My first exposure to medical imaging came even earlier, during a summer job between my last two undergraduate years. I was an applied mathematics student, but I worked in a physics group at Massachusetts General Hospital. The group used nuclear medicine both as an imaging technique and as a way to measure biological processes.”
In nuclear medicine, a substance involved in metabolism is injected, labelled with a radioisotope, and measured over time as it accumulates in different areas. From this, metabolic behaviour can be inferred – for example, how the kidney extracts fluid from blood and produces urine.
“My task was to analyse these time‑dependent measurements and compute the rates of metabolic exchange – values that were diagnostically meaningful.”
Nuclear medicine was appealing at the time because computers had extremely limited memory. Nuclear medicine images were unclear – requiring only the small number of 32×32 or 64×64 pixels – and therefore manageable for the small computers of the time. Other modalities, such as radiography, required thousands of pixels in each direction and could not be processed. Early work in medical image computing thus focused on nuclear medicine simply because the images were small enough to handle.
“In 1973, during a sabbatical, I shifted from improving image quality (restoration) to working on display – how images are shown to humans. Just before that time, Geoffrey Hounsfield invented CT, which introduced higher‑resolution imaging and, for the first time, made 3D imaging possible,” Stephen Pizer said.
Shape and Appearance
Pizer has seen imaging technologies evolve dramatically. Each modality measures different physical properties, he noted, and provides different challenges to extract shape properties.
“Later, I worked on contrast enhancement for CT, X‑ray, and radiation‑therapy images – all of which often have low contrast. Already with CT, 3D images became feasible, and with 3D images came the ability to study 3D shape.”
Researchers realised that segmentation – determining where an object is – depends on understanding two things:
- Shape (the anatomy)
- Appearance (the intensity patterns distinguishing the object from its surroundings)
The principle of image analysis is rooted in the idea that understanding the body requires understanding both shape and appearance: how organs look, how they vary, and how disease alters them over time.
Effective segmentation therefore requires knowledge of both the typical shape and its variability across people, and the typical appearance and its variability.
Pizer illustrated this with examples ranging from prostate‑cancer treatment to lung biopsies. In both cases, anatomy changes subtly from day to day – and even breath to breath – making precise targeting essential.
“This insight led me into probabilistic modelling of shape and appearance – work that eventually evolved into the method you saw today.”
A cautious look at AI
Naturally, the conversation turned to AI at the visit at DTU Compute.
Stephen Pizer first studied AI in the 1960s, and while some techniques remain familiar, today’s progress is driven by computing power and the ability to store the vast amounts of training data modern methods require. But he cautioned against assuming AI will replace careful method development.
“We can see this becoming increasingly widespread. There is collaboration between hospitals, universities, and now also startups. Things are developing. You asked when these technologies might take over analysis, or whether they will function as support tools.”
“It is a continuing process. Many methods are adopted, but medicine – as practised by physicians – is very conservative. Many technologies are never adopted, or adopted extremely slowly, simply because physicians are cautious. They do not accept new ideas quickly.”
In imaging, the inputs are known – the images – and the goal is to map them to medically relevant properties. That transformation is what deep‑learning systems do, he said.
“They perform extremely well on some fractions of objectives, but poorly on others. And they offer little ability to tell you when or why they fail. You cannot understand the process inside them. One of the main areas I see people at DTU working on is how to extract from the behaviour of these systems what they do well and what they fail at. Without that understanding, the adoption rate among competent physicians will be slow.”
“I cannot predict how long it will take before understanding increases to the point where AI becomes a dependable part of clinical practice. It will vary by problem. In some, AI now does far better than anyone could without it. But many other problems are far more complex, and AI does not behave nearly as effectively there. Scientists argue intensely about how to structure AI to make it more accurate.”
A more promising approach
Some believe one can simply feed the original image into a machine and let it produce the answer.
“In my experience, that is often unsuccessful, though in some problems it works well. A more promising approach is to apply fundamental analysis first – such as analysing shape and appearance – and then transform the original image data into more fundamental measurements, which Aasa Feragen and I see as a more hopeful direction. These measurements can then be used as input to AI. That transformation can itself be performed by AI. But using these fundamental measurements gives a better chance of both success and understanding.”
Despite being 84 and at what he modestly calls “the end of a long career”, Stephen Pizer remains fully engaged and curious. He has been at the University of North Carolina for 59 years and expects the biggest changes to arrive after his time.
“Still, it is exciting to see how things are developing and how my foundational research feeds into new collaborations. That is why I am here,” he concluded.
