In various scenarios, accurate 3D tracking of ultrasound probes (e.g. by optical or electro-magnetic tracking) is not feasible, but recovering the 3D trajectory of the ultrasound scan would be helpful for diagnosis, measurements, or image registration.
Using only Deep Learning, potentially an inertial measurement unit, and the powerful 3D image registration capabilities of the ImFusion Suite, we have been able to accurately predict probe motion in extremity scans, to reconstruct the 3D shape of the thyroid, and to register liver scans even when there are breathing artifacts.
This work did not only lead to successful productization by piur imaging but also high-profile publications:
Reconstruction of a sweep following the great saphenous vein (more than 60cm)
Related Publications
2025
DualTrack: Sensorless 3D Ultrasound needs Local and Global Context
Paul Wilson, Matteo Ronchettin, Rüdiger Göbl, Viktoria Markova, Sebastian Rosenzweig, Raphael Prevost, Parvin Mousavi, and Oliver Zettinig
In MICCAI 2025 - International Workshop of Advances in Simplifying Medical UltraSound (ASMUS), Sep 2025
Three-dimensional ultrasound (US) offers many clinical advantages over conventional 2D imaging, yet its widespread adoption is limited by the cost and complexity of traditional 3D systems. Sensorless 3D US, using deep learning to estimate a 3D probe trajectory from a sequence of 2D US images, is a promising alternative. Local features such as speckle patterns can help predict frame-to-frame motions, while global features, such as coarse shapes and anatomical structures, can situate the scan relative to anatomy and help predict its general shape. In prior approaches, global features are either ignored or tightly coupled with local feature extraction, restricting the ability to robustly model these two complementary aspects. We propose DualTrack, a novel dual encoder architecture leveraging decoupled local and global encoders specializing in their respective scale of feature extraction. The local encoder uses dense spatiotemporal convolutions to capture fine-grained features, while the global encoder utilizes an image backbone such as a 2D CNN or foundation model and temporal attention layers to embed high-level anatomical features and long-range dependencies. A lightweight fusion module then combines these features to estimate trajectory. Experimental results on a large public benchmark show that DualTrack achieves state-of-the-art accuracy and globally consistent 3D reconstructions, outperforming previous methods and yielding an average reconstruction error below 5 mm.
2020
Three-Dimensional Thyroid Assessment from Untracked 2D Ultrasound Clips
Wolfgang Wein, Mattia Lupetti, Oliver Zettinig, Simon Jagoda, Mehrdad Salehi, Viktoria Markova, Dornoosh Zonoobi, and Raphael Prevost
In Medical Image Computing and Computer Assisted Intervention – MICCAI 2020, Sep 2020
This work aims at creating 3D freehand ultrasound reconstructions from 2D probes with image-based tracking, therefore not requiring expensive or cumbersome external tracking hardware. Existing model-based approaches such as speckle decorrelation only partially capture the underlying complexity of ultrasound image formation, thus producing reconstruction accuracies incompatible with current clinical requirements. Here, we introduce an alternative approach that relies on a statistical analysis rather than physical models, and use a convolutional neural network (CNN) to directly estimate the motion of successive ultrasound frames in an end-to-end fashion. We demonstrate how this technique is related to prior approaches, and derive how to further improve its predictive capabilities by incorporating additional information such as data from inertial measurement units (IMU). This novel method is thoroughly evaluated and analyzed on a dataset of 800 in vivo ultrasound sweeps, yielding unprecedentedly accurate reconstructions with a median normalized drift of 5.2%. Even on long sweeps exceeding 20 cm with complex trajectories, this allows to obtain length measurements with median errors of 3.4%, hence paving the way toward translation into clinical routine.
