Camus Challenge

Tin Hadži Veljković, Andrea Lombardo, Tadija Radusinović, Matteo Tafuro, Thomas Wiggers

AI for Medical Imaging, MSc. Artificial Intelligence, University of Amsterdam

Medical imaging technology has significantly improved the quality of healthcare, with methods such as MRI, CT and Ultrasound imaging. While MRI's provide valuable information for medial diagnosis, its cost of operation is substantial and limits the wide applicability of the method. Other imaging methods, such as ultrasound imaging, are far cheaper to operate. This is not without drawbacks however, as the produced images are not three dimensional and of significantly lower resolution than an MRI. Echocardiograms produced by ultrasound scanners are often noisy, and it is difficult to distinguish the appropriate anatomical parts. However, this does not render them useless. Coronary heart disease is a major cause of death worldwide. Cheap diagnosis can help medical professionals in their work, and echocardiographic imaging can be used for this. However, manually annotation or processing these images using experts is time consuming, and could benefit from automatic tools.

The CAMUS (Cardiac Acquisitions for Multi-structure Ultrasound Segmentation) challenge has been set out to improve segmentation of 2D echocardiographic images. The authors released a publicly available, 500 patient dataset to allow for comparable results across heart segmentation research. The nature of ultrasound imaging produces noisy images that are difficult to understand and accurately segment the cardiac regions. These segmentations can, for example, be used to diagnose coronary heart disease.

The dataset

The CAMUS challenge uses the largest publicly-available and fully-annotated dataset for 2D echocardiographic assessment. It contains 2D apical four-chamber and two-chamber view sequences acquired from 500 patients and is made available for download here.

Data preparation

Please follow the steps below:

• Register your account on the download website and download both training and testing data.
• Extract the compressed files (training.zip and testing.zip), and move the resulting folders to the /data directory of the repository.

Model Trainings

U-Net implementation

Firstly, we have used a forked version of [ADD GINO BASELINE GITHUB REPO] for the training of the UNet. However, for the implementation of the self-supervised we needed to heavily modify the low-level structure of UNet (modification of the forward pass). For this reason we have implemented our own version of UNet, which can be found in the unet.py script. We also added Albumenations transforms in the training pipeline. The training is done in the Training Notebook.

VICReg - Self-Supervised learning implementation

The goal is to leverage the unlabeled in-between frames of the recordings to help build meaningful representations. We apply a VICReg objectiv to the downsampled vector representation of a UNet, training until convergence. It includes an covariance loss, which acts as an attractive force between the representations, and the variance loss, which prevents collapse by enforcing feature variance. Both these terms are scaled with a coefficient of 25. The covariance loss, which aims to decorrelate individual representation feature, is not scaled, in line with the suggestions from the VICReg paper. Our implementation, which is used for pre-training UNet is done in this VICReg Training Notebook, and the VICReg module itself is imeplemented in the vicreg.py script.

LadderNet implementation

For our LadderNet implementation, we used an implementation from this repository.

nnUNet implementation

For nnUnet, please take a look at create_camus_dataset.py to convert to hdf5, create_CAMUS_task_from_hdf5.py to convert to nnU-Net format, perpare_submission.py to convert back to the CAMUS dataset format. For instructions on how to run the task, please see the detailed instructions in the medium article.

Results

In the figure below, we can see comparison of different results for different models.

U-Net results

U-Net performs remarkably well to be the simplest model of our study, and marks a competitive baseline performance. We can also see that SSL training didn't much improve on the results, and had acutally induced some distortions at some masks.

LadderNet results

LadderNet, despite the architectural improvements, does not show significant deviations from the results of U-Net in any metric, with a notable exception being the EF correla- tion coefficient. Along the same lines, the U-Net model aug- mented with SSL pre-training does not reveal remarkable im- provements over U-Net. Contrarily, it results in a slight degra- dation of the EF correlation coefficient.

nnU-Net results

The nnU-Net exhibits a superior performance, achieving incredibly high values for the DICE index and all of the clini- cal metrics (EF, LVEDV and LVESV). As the method makes use of parameters optimization on top of a U-Net, these improve- ments were expected. Nonetheless, the distance-based metric seem off and the Hausdorff distance (dh) is an order of magni- tude larger than the other approaches. We have not been able to investigate the causes of the problem, since the evaluation on the test set is performed by a third party on a closed-source platform. We suspect a problem in the processing of our seg- mentation masks, because the evaluation platform computed the metrics for only 33 of the 50 test patients. It is notewor- thy that the same submission format was used for all models,