++
HIGHLIGHTS
Applications of deep learning in surgery fit mostly into 3 broad categories: preoperative planning, intraoperative navigation and tracking, and intraoperative control.
Deep learning for surgery has explored applications including surgical skill assessment, preoperative risk stratification, robotic control of devices, and automation of operative tasks.
Deep learning could augment surgeons’ abilities by further empowering their skills via technologies such as facial tracking for automated control of a laparoscope, estimated force feedback during robotic surgery, or even trajectory planning for the placement of sutures.
++
Deep-learning (DL) techniques have witnessed great success and achieved performance levels that are almost as good as or even better than human performance in many fields, including computer vision, natural language processing, and control systems (see Chapter 4).1-9 DL is marked by its computational structure composed of multiple (from tens to thousands) neural layers for feature extraction, probability prediction, and learning optimized representation of data.9 Because of the different receptive fields in this deep structure, it can learn informative features from training data through a hierarchical level of abstraction that retains information coherence at each level of the hierarchy,6 hence capturing the distribution of high-dimensional data (eg, an image) with a large number of model parameters.10
++
The advancement of the following elements makes it possible and promising to solve various traditionally difficult challenges in the field of artificial intelligence (AI): (1) DL methods able to take advantage of large training data information; (2) big data technology for storing, transporting, and accessing data of unprecedent sizes; and (3) computing hardware, especially the extensive utilization of graphic processing units (GPUs) for more powerful yet inexpensive computational power.
++
With reports of various applications of DL in robotic surgery, some hope that DL has the potential to address some of the technological challenges of advancing robot-assisted and computer-integrated surgical systems to a level of targeted autonomy. These hopes are also drawn from advances in the field of medical image analysis, where DL has rapidly become a dominant methodology for applications such as classification, detection, segmentation, registration, and reconstruction.11 DL’s intrinsic capability of automatically learning representative features from data, rather than relying on hand-crafted features based on domain expertise, has proved to be efficient and robust in analyzing medical images of various modalities and diseases.12
++
From the algorithmic perspective, DL is well known for 2 main categories of algorithms: modeling of data and modeling of actions. The algorithms that largely perform modeling of data through hierarchical representation learning (ie, network structure) effectively transform data (eg, images) from their original high-dimensional space (eg, image pixels/voxels) to a lower-dimensional feature and abstract space. The learned features can then be used for further analysis of tasks mainly centered around a logistic regression model (eg, classification). Algorithms that center on modeling of action aim to infer the optimal strategy of interactions between ...