Skip to Main Content



  • Although computer vision functions in a manner unique from biologic vision, many principles were influenced by biologic systems.

  • Digital images are composed of fundamental units and shapes represented on a coordinate system. Digital images are represented by a matrix of values that contain information (eg, color, transparency) that affects the way we perceive the images.

  • Videos are composed of a series of still images, of which a single image is referred to as a video frame. Many principles of computer vision relating to still images thus apply to video but with the added component of time.

  • Computer vision problems often rely on detection of features, which are attributes in the data that can be used to assist in modeling.


Computer vision refers to machine understanding of visual data (ie, images and videos). Although computer vision is a field in and of itself, it also heavily intersects with machine learning and related fields such as image processing, signal processing, optics, and the cognitive sciences such as psychology and neuroscience.1 While object detection/recognition (eg, identifying items, animals, or people in an image) has become more popular in our daily lives through social media and other platforms, computer vision encompasses a field of work that is quite broad and includes research on describing and analyzing the visual world in numerical or symbolic form to allow for interpretation of images for subsequent action (eg, analyzing visual information for computer-assisted driving). Research and applications in computer vision have embodied the intersectional spirit of the field as techniques have drawn methodology from various schools of machine learning, including deep learning.

In this chapter, we will cover fundamental principles necessary for a foundational understanding of computer vision, particularly as it applies to surgery. As with other chapters in this section, the information in this chapter is not intended to imbue expertise, as entire series of textbooks are needed to cover aspects of computer vision; rather, it is intended to provide knowledge on the principles of computer vision so that the reader may be better informed when reading and assessing surgical literature that may use computer vision in its methodology. Additional resources are available in the appendix for further study and a deeper dive into topics in computer vision.


Although an in-depth review of the human visual system is outside of the scope of this chapter, a brief review of the human visual system can help provide context to understanding the similarities and differences between our own visual systems and that of computers.2

Optical reception within the human visual system occurs in the eye as light passes through the cornea and lens and onto the retina (Figure 6-1). The retina is composed of a number of cell types (eg, rods, cones, bipolar and ganglion ...

Pop-up div Successfully Displayed

This div only appears when the trigger link is hovered over. Otherwise it is hidden from view.