In various scenarios, accurate 3D tracking of ultrasound probes (e.g. by optical or electro-magnetic tracking) is not feasible, but recovering the 3D trajectory of the ultrasound scan would be helpful for diagnosis, measurements, or image registration.
Using only Deep Learning, potentially an inertial measurement unit, and the powerful 3D image registration capabilities of the ImFusion Suite, we have been able to accurately predict probe motion in extremity scans, to reconstruct the 3D shape of the thyroid, and to register liver scans even when there are breathing artifacts.
This work did not only lead to successful productization by piur imaging but also high-profile publications:
Reconstruction of a sweep following the great saphenous vein (more than 60cm)
Related Publications
2020
Three-Dimensional Thyroid Assessment from Untracked 2D Ultrasound Clips
Wolfgang Wein, Mattia Lupetti, Oliver Zettinig, Simon Jagoda, Mehrdad Salehi, Viktoria Markova, Dornoosh Zonoobi, and Raphael Prevost
In Medical Image Computing and Computer Assisted Intervention – MICCAI 2020, Sep 2020
This work aims at creating 3D freehand ultrasound reconstructions from 2D probes with image-based tracking, therefore not requiring expensive or cumbersome external tracking hardware. Existing model-based approaches such as speckle decorrelation only partially capture the underlying complexity of ultrasound image formation, thus producing reconstruction accuracies incompatible with current clinical requirements. Here, we introduce an alternative approach that relies on a statistical analysis rather than physical models, and use a convolutional neural network (CNN) to directly estimate the motion of successive ultrasound frames in an end-to-end fashion. We demonstrate how this technique is related to prior approaches, and derive how to further improve its predictive capabilities by incorporating additional information such as data from inertial measurement units (IMU). This novel method is thoroughly evaluated and analyzed on a dataset of 800 in vivo ultrasound sweeps, yielding unprecedentedly accurate reconstructions with a median normalized drift of 5.2%. Even on long sweeps exceeding 20 cm with complex trajectories, this allows to obtain length measurements with median errors of 3.4%, hence paving the way toward translation into clinical routine.
