Multispectral image analysis for patient tracking in spinal surgery


During spinal surgery patient’s tracking is needed for optimal motion compensation, in order to guide the physician in a minimally invasive way by a computer-assisted navigation system. In order to position implants such as pedicle screws, there has been a definite shift to the use of minimally-invasive techniques, performed by manipulating surgical tools through small incisions in the patient’s skin, in order to minimize blood loss and soft tissue damage, and consequently reduce postoperative pain. Due to inherently reduced visibility of the spine during these procedures, surgeons have to rely on real-time imaging and navigation solutions to guide their surgical tools and implants.
The current approach, developed by Philips Healthcare, consists of using optical markers, attached to the patient skin with a defined protocol, for motion tracking and compensation. Four cameras are integrated and rigidly fixed in the X-ray detector to track the marker positions and correct for their movement for guiding the instruments. Despite the widely use of optical markers for motion tracking, several are the drawbacks of this approach and there are some limitations in their usage. The main limitations of the actual optical markers are given by the fact that this procedure can lead discomfort due to the presence of liquids, that can alter markers placement, and/or due to markers movement caused by the surgeon during the surgical operations. Moreover, placing the markers in a correct way requires effort and additional time.
Using natural landmarks detected on the skin, we aim at improving the unobtrusiveness of the procedure and track the patient position without any attached markers.
Multispectral imaging is allows subcutaneous vein detection and localization in human subjects [1]. It has to be noted that multispectral imaging has been applied in several fields, from the cannulation to the biometric purpose for human identification, in order to enhance the detection of veins for intravenous cannulation [2] as well as to extract subcutaneous information (i.g. concentration of hemoglobin in tissue). Veins or skin properties can be used as detected feature for motion tracking.

[1] V. P. Zharov, S. Ferguson, J. F. Eidt, P. C. Howard, L. M. Fink, and M. Waner, “Infrared imaging of subcutaneous veins,” Lasers Surg. Med., vol. 34, no. 1, pp. 56–61, 2004.
[2] M. Asrar, A. Al-Habaibeh, and M. Houda, “Innovative algorithm to evaluate the capabilities of visual, near infrared, and infrared technologies for the detection of veins for intravenous cannulation,” Appl. Opt., vol. 55, no. 34, pp. D67–D75, 2016.


In our research we have already analyzed multispectral images from 30 subjects. The investigated wavelengths are 430, 550, 680, 740, 850 and 970 nm. Two anatomical areas are investigated: the cervical and lumbar area in the back of the human body. To simulate skin patient motion, an image translation has been performed. Our approach consists of the following steps: image preprocessing to enhance the desired features, feature detection and tracking for the one-to-one match. The blob detection algorithm was found in literature as a good method for detection and tracking, and it should be applied easily to the landmarks in cutaneous and subcutaneous tissues, such as moles and veins. Maximally stable extremal regions (MSER) and Speeded Up Robust Feature (SURF) are used for blob detection and matching.
The proposed tracking approach showed high accuracy. However new experiments have to be performed on healthy subjects, using a hyperspectral camera, and simulating the real surgical scenario (patient breathing and skin movement). The research activities involved in the future steps mainly consist of improvements and adaptation of the existing algorithm to the new data set, as well as design and automatic selection of new natural features for motion tracking in spinal surgery.

This project will be mainly executed internally at the TU/e, but also partly at Philips Healthcare.
hyperspectral imaging, motion tracking, computer-aided diagnosis, spinal surgery
Contact person: