Hyperspectral image analysis for surgical application


Hyperspectral imaging (HSI) consists of an acquisition of a series of two-dimensional images across many adjacent narrow spectral bands and reconstruction of reflectance spectrum for every pixel of the image [1]. In recent years, developments in HSI, image analysis methods, and computation power make its applications in medical field possible. The camera acquires a complete spectrum for a line of pixels and then spatially scans the scene. Thus, scanning over the tissue the HSI camera collects 2D images, creating a hypercube with two spatial dimensions and one spectral dimension [2].
Lu et al. [2] demonstrated the potential of HSI as a non-invasive tool for tumor visualization and classification during surgery by proposing a framework of hyperspectral image processing and quantification, and testing it on images from mice with head and neck cancer.
The success of a surgery is highly depended on a surgeon’s ability to make judgments to identify the lesions and their margins. Intraoperative tumor imaging and visualization is an important step in aiding surgeons to evaluate and resect tumor tissue in real time, thus enabling a more complete resection of diseased tissue and better conservation of healthy tissue. HSI has been explored in various surgeries.
With spatial and spectral information obtained, image classification methods for hyperspectral datasets are required to classify relevant spectral information. It aims to discriminate between different tissues (such as between normal tissues and tumors). The basic steps for hyperspectral image analysis generally involve preprocessing, feature extraction and feature selection, and classification [2].
Feature extraction and representation is a crucial step for classification tasks. Efficient feature extraction can lead to optimal classification performance. However, HSI has just been applied to medical application field recently, and it is still not well understood what features are the most effective and efficient to differentiate cancer from normal tissue in medical hyperspectral images [2]. With only the reflectance features, classification performance is still far from optimal. Therefore, there is still space for adding new features to improve the distinction of cancer from normal tissue.

[1] J. Solomon and B. Rock, “Imaging spectrometry for earth remote sensing,” Science, vol. 228, no. 4704, pp. 1147–1152, 1985.
[2] Lu, Guolan, and Baowei Fei. "Medical hyperspectral imaging: a review." Journal of biomedical optics 19.1 (2014): 010901-010901.


The squamous cell carcinoma of the tongue is a malignant tumor that can spread towards the lymph nodes in the neck and the rest of the body, if left untreated [3]. Treatment mainly consists of surgery in order to remove the tumor. Prior to surgery, diagnosis using MRI, OCT and biopsy is performed to evaluate the anatomy and tumor type. However, for the success of the intervention, it is very important to properly remove tumor residues. Intraoperative tumor imaging and cancer detection, in a non-invasive, rapid and simple way, can help surgeons to resect tumor tissues with a safe margin of normal tissue. Currently the gold standard to assess the tumor margins and to evaluate the malignancy is the histopathology. The sample is subjected to histological examination, 2-5 days after surgery. If tumor positive resection margins are found, a second operation is indicated to clear possible tumor deposits, left behind during the initial surgery. However, this method is time consuming, expensive and subjective, but most importantly, it does not allow intraoperative assessment [3]. A new method to detect tumor margins in real time, during the intraoperative scenario, is needed to prevent re-operation or recurrence of cancer.
The carcinoma of tongue affects mostly patients between 40-50 years old. The number of patients undergoing tongue tumor resection, at NKI, is about 30 per year. This type of tumor is interesting for our research because the resected sample mainly contains only two types of tissue: muscle and cancer (binary classification).
The main goal is to detect the safety margins of the tumor with the hyperspectral imaging, and then integrate the algorithm into the intraoperative workflow to allow intraoperative margin assessment.
The research activities involved are:

  • Creation of data preprocessing algorithms and registration with the pathology (labelled by an expert)
  • Identification of discriminative features and machine learning methods for cancer detection
  • Evaluation of the developed cancer detection methods validation to alternative approaches proposed in literature.

[3] Liu, Zhi, Hongjun Wang, and Qingli Li. "Tongue tumor detection in medical hyperspectral images." Sensors 12.1 (2011): 162-174.

This project will be executed internally at the TU/e.
machine learning, image analysis, hyperspectral imaging, healthcare/oncology, computer-aided diagnosis
Contact person: