Computer-aided esophageal cancer detection in endoscopic video


People suffering from gastric reflux over a longer period of time will develop a so-called Barrett’s Esophagus (BE). This is a condition, in which the lower part of the esophagus is covered with a different type of tissue. Normal esophageal tissue cannot resist gastric acid, which typically escapes from the stomach in people suffering from reflux. As a defense mechanism the body starts replacing the esophageal tissue with a tissue type similar to that of the small intestine, which is acid-resistant. However, this process greatly increases the chance of developing cancer. Patients with a BE are regularly screened and if the developing cancer is detected at an early stage, it can removed by a small surgical procedure. When the cancer is detected at a later stage of the disease, the only treatment is surgical removal of the entire esophagus and the survival chances of the patient are drastically decreased (nearly 100% five-year-survival for early detection vs. 10-15% five-year-survival for late detection). Hence, it is of crucial importance to the patient that these cancers are detected at an early stage. However, these early cancerous lesions have shown to be very hard to identify by endoscopic inspection and are therefore regularly missed. For this reason, a Computer-Aided Detection (CAD) system to aid the gastroenterologist in identifying these early cancerous lesions in BE is highly desirable. Furthermore, such a system offers an objective second opinion, to reduce the considerable intra-observer variability for Barrett cancer detection.
In this project, a CAD system is developed that inspects the esophageal tissue during endoscopic screening. Over the past couple of years, we have explored different machine learning methods, such as Support Vector Machines (SVM), Decision Forests and Neural Networks for this purpose. In our studies, we have analyzed the spectral characteristics of early cancerous tissue and exploited this knowledge to develop tuned wavelet-based filters that are able to capture the distinctive texture patterns associated with early cancer. Furthermore, we have experimented with different features that are often used in computer vision algorithms, such as Local Binary Patterns (LBP), Gray-Level Co-occurrence Matrices (GLCM) and Histogram of Oriented Gradients (HOG). Our latest results show that trained computer algorithms are able to detect early cancer in BE, with an accuracy close to that of internationally recognized experts.


Although promising results are demonstrated for automated Barrett’s cancer detection, a number of steps still need to be taken to prepare such an algorithm for clinical application. The current algorithm operates on still endoscopic images, while a clinical system must process endoscopic video. Although endoscopic video analysis offers additional challenges, such as variable frame quality and strong demands on processing time, it also yields a number of opportunities, like exploiting temporal information. The three involved hospitals are expert centers regarding Barrett’s esophagus and provide a continuous stream of endoscopic videos for our joint research collaboration. In this project, we employ these endoscopic video’s for the development of an system for the automated detection of early Barrett’s cancer. This involves a number of research activities, which are listed below.

  • Efficient implementation of adequate preprocessing to enable (real-time) endoscopic video processing
  • Identification of discriminative features and machine learning methods for robust and accurate cancer detection
  • Evaluation of the developed cancer detection methods on endoscopic videos, annotated by at least three experts on Barrett cancer

A C++ framework is currently available to allow frame-by-frame analysis of the endoscopic videos. However, the detection performance is not optimal and should become more robust and novel methods should be investigated and implemented to achieve this result. Although the current C++ algorithm executes very fast, for investigation the potential of Convolutional Neural Networks (CNN), other programming environments could also be used, such as TensorFlow and Python.

This project will be mainly executed internally at the TU/e, but also partly at the involved hospitals in Eindhoven (Catharina hospital), Amsterdam (AMC) and Leuven (UZ).
Machine learning, Image analysis, Computer-Aided Diagnosis (CAD), Healthcare/oncology
Contact person: