Augmented Reality Microscopy Offers a New Perspective for Biomedical Applications

Augmented reality microscopy (ARM) is a new technology that transforms the traditional light microscope into a hybrid device, allowing pathologists to deploy computer-assisted diagnostics and other digital pathology software directly from the microscope.

ARM integrates within the existing microscope, maintaining the optical view of the specimen while facilitating multidirectional communication with a PC. The technology uses a camera to acquire real-time images, which can be processed by a PC, and then projects the computer-generated output — such as annotations, morphometrics, or image analysis — as an overlay directly within the microscope eyepiece.

ARM technology has been the subject of several recent studies, including its use as a tool for real-time telepathology, scoring of Ki-67 NETs (neuroendocrine tumors) with QuPath algorithms, quantification of NAFLD (nonalcoholic fatty liver disease) using deep learning AI algorithms, and tracking movement of the slide on the stage during the routine pathology workflow.

As opposed to traditional digital pathology technologies that operate outside the microscope, ARM has been found to be quicker to use, it provides for the real-time and seamless integration of algorithms without the need to first acquire digital images, it is cheaper, and it requires no special technical skills.

In the future, ARM could enable pathology labs to use AI algorithms, synchronize diagnostic data with the laboratory information system (LIS), and automate mundane tasks.