Biophotonics Imaging Laboratory

AI-driven optical imaging

Images showing the classification of human breast cancer by a DNN.

Image providing an overview on the ML-based classification platform for ear diagnostic imaging.

In BIL, we have been generating multi-channel images using our unique optical imaging systems. Considering the rich content our images have, it is challenging for a human to examine every layer of those images and extract all the useful information that is potentially relevant to the question we are investigating. With the assistance of Artificial Intelligence (AI), we are now able to accomplish those tasks by training and implementing different types of Machine Learning (ML) and Deep Learning (DL) models. To fully utilize the power of AI, we are actively investigating the potential of adopting various AI techniques in every aspect of our imaging and data analysis process, including image generation, image processing, object segmentation, feature extraction, and downstream analysis. Current applications include DL-based single-cell profiling, DL models for breast cancer diagnosis, Generative Adversarial Network (GAN)-based style transferring between modalities, Weakly-Supervised Learning models for tissue classification, ML-based classification platform for ear diagnostic imaging, AI-driven automated parameter tuning for our imaging systems. Continuing efforts are being made to develop customized ML/DL models with better performance, enhanced interpretability, and less dependence on manual annotations. We believe that the advancement in AI will assist us to efficiently generate higher-quality images, to extract more meaningful information from the images, and to guide us through the process of research.

RECENT PUBLICATIONS:
  • Monroy, G.L., Won, J., Dsouza, R. et al. Automated classification platform for the identification of otitis media using optical coherence tomography. npj Digit. Med. 2, 22 (2019).
  • You, S., Sun, Y., Yang, L. et al. Real-time intraoperative diagnosis by deep neural network driven multiphoton virtual histology. npj Precis. Onc. 3, 33 (2019).
  • Sun, Y. et al. Real-time three-dimensional histology-like imaging by label-free nonlinear optical microscopy. Quantitative imaging in surgery and medicine.10(11), 2177(2020).

AI driven solutions for fluid diagnosis

Using our extensive library of imaging data and years of experience in ear diagnostic imaging, a machine-learning platform was developed that can automatically detect the signs of infection in the middle ear. When an OCT image of the ear is taken using one of our portable systems, the software takes approximately 20 seconds to automatically evaluate the data and identify middle ear fluid or biofilms behind the eardrum, or, if the ear appears normal. With further development, this platform can enable any user to have the diagnostic ability of an expert physician. Armed with precise knowledge of the infection in this patient, physicians can prescribe the ideal treatment for each patient. 

PUBLICATIONS:
  • Monroy GL, Won J, Dsouza R, Pande P, Hill MC, Porter RG, Novak MA, Spillman DR, Boppart SA. Automated classification platform for the identification of otitis media using optical coherence tomography. Nature Digital Medicine, 2:22. 2019.

Automated diagnosis of cancer using deep learning models

Histopathology has been widely adopted in tissue assessment in clinical decision making and in research. However, its complex procedure of tissue preparation and analysis prevent histopathology from intraoperative assessment and cause delays in treatment planning. Our lab developed an Artificial Intelligence (AI)-driven real-time intraoperative diagnosis system based on stain-free slide-free multimodal multiphoton microscopy. This system captures rich molecular and structural information from living or freshly excised tissue and use Deep Neural Networks (DNN) to extract meaningful features and generate diagnoses. Compared to traditional histopathology, which may take several days to process the tissue and generate analysis reports, our highly automated system only requires minimal tissue processing and generates reliable diagnosis within minutes, which makes this technique an attractive alternative or adjunct to histochemistry in clinical settings, especially in time-sensitive scenarios such as the intraoperative assessment of breast tissue during surgical oncology procedures.

Images showing classification results of human breast cancer made by our system. (a) Multiphoton composite image of the four channels (THG, NADH, SHG, FAD) generated by our stain-free slide-free multimodal multiphoton microscopy. (b) Classification results corresponding to the multiphoton images in (a). The cancer probability map was coded with red color and overlaid with the original multiphoton composite image. Lipid-filled adipocytes are coded with blue color. (c) AUC statistics on test set, for different DNN architectures and different tile sizes. (d) Representative example of false positive (normal tiles classified as cancer, highlighted by the yellow oval) and false negative (cancer tiles classified as normal, highlighted by the yellow oval). Scale bar: 200 µm. 

PUBLICATIONS:

Virtual histology

Histology imaging, invented centuries ago, serves an essential clinical imaging method to provide evidence for cancer-related diagnosis. However, the time- and labor- intense tissue preparation processes of histology causes delay in cancer diagnosis and increases the cost of cancer treatment. Our lab uses a novel imaging method, the simultaneous label-free autofluorescence multiharmonic (SLAM) microscopy, to generate histology-like images from label-free tissues, saving the time and labor of tissue processing. By imaging fresh human and rat tissues without any tissue processing or staining, various biological tissue features are effectively visualized by one or multiple imaging modalities of the SLAM microscope and converted to histology-like contrasts. In particular, we realized for the first time the real-time 3D histology-like imaging. 

PUBLICATIONS:
  • Sun, Y. et al. Real-time three-dimensional histology-like imaging by label-free nonlinear optical microscopy. Quantitative imaging in surgery and medicine. 10(11), 2177 (2020).  

/res