Multispectral image analysis for patient tracking in spinal surgery


During spinal surgery patient’s tracking is needed for optimal motion compensation, in order to guide the physician in a minimally invasive way by a computer-assisted navigation system. Despite the widely use of references for motion tracking, several are the drawbacks of this approach and there are some limitations in their usage. Using natural landmarks detected on the skin, we aim at improving the unobtrusiveness of the procedure and track the patient position without any attached markers. Multispectral imaging allows subcutaneous vein detection and localization in human subjects [1]. It has to be noted that multispectral imaging has been applied in several fields, from the cannulation to the biometric purpose for human identification, in order to enhance the detection of veins for intravenous cannulation [2] as well as to extract subcutaneous information (i.g. concentration of hemoglobin in tissue). Veins or skin properties can be used as detected feature for motion tracking.

[1] V. P. Zharov, S. Ferguson, J. F. Eidt, P. C. Howard, L. M. Fink, and M. Waner, “Infrared imaging of subcutaneous veins,” Lasers Surg. Med., vol. 34, no. 1, pp. 56–61, 2004.
[2] M. Asrar, A. Al-Habaibeh, and M. Houda, “Innovative algorithm to evaluate the capabilities of visual, near infrared, and infrared technologies for the detection of veins for intravenous cannulation,” Appl. Opt., vol. 55, no. 34, pp. D67–D75, 2016.


In our research we have already analyzed multispectral images from 30 subjects. The investigated wavelengths are 430, 550, 680, 740, 850 and 970 nm. Two anatomical areas are investigated: the cervical and lumbar area in the back of the human body. To simulate skin patient motion, an image translation has been performed. Our approach consists of the following steps: image preprocessing to enhance the desired features, feature detection and tracking for the one-to-one match. The blob detection algorithm was found in literature as a good method for detection and tracking, and it should be applied easily to the landmarks in cutaneous and subcutaneous tissues, such as moles and veins. Maximally stable extremal regions (MSER) and Speeded Up Robust Feature (SURF) are used for blob detection and matching.
The proposed tracking approach showed high accuracy. However new experiments have to be performed on healthy subjects, using a hyperspectral camera, and simulating the real surgical scenario (patient breathing and skin movement). The research activities involved in the future steps mainly consist of improvements and adaptation of the existing algorithm to the new data set, as well as design and automatic selection of new natural features for motion tracking in spinal surgery.

This project will be mainly executed internally at the TU/e, but also partly at Philips Healthcare.
hyperspectral imaging, motion tracking, computer-aided diagnosis, spinal surgery
Contact person: