Diagnostic devices, methodological approaches, and traditional constructs of clinical pathology practice, cultivated throughout centuries, have transformed radically in the wake of explosive technological growth and other, e.g., environmental, catalysts of change. Ushered into the fray of modern laboratory medicine are digital imaging devices and machine-learning (ML) software fashioned to mitigate challenges, e.g., practitioner shortage while preparing clinicians for emerging interconnectivity of environments and diagnostic information in the era of big data. As computer vision shapes new constructs for the modern world and intertwines with clinical medicine, cultivating clarity of our new terrain through examining the trajectory and current scope of computational pathology and its pertinence to clinical practice is vital. Through review of numerous studies, we find developmental efforts for ML migrating from research to standardized clinical frameworks while overcoming obstacles that have formerly curtailed adoption of these tools, e.g., generalizability, data availability, and user-friendly accessibility. Groundbreakingvalidatory efforts have facilitated the clinical deployment of ML tools demonstrating the capacity to effectively aid in distinguishing tumor subtype and grade, classify early vs. advanced cancer stages, and assist in quality control and primary diagnosis applications. Case studies have demonstrated the benefits of streamlined, digitized workflows for practitioners alleviated by decreased burdens.
1. Introduction
Nearly 2000 years have passed since Emperor Marcus Aurelius sought reinforcement for a society decimated by the first wave of the deadliest pandemic to impact ancient Rome. The same factors lauded as strengths for the seemingly impenetrable empire, e.g., expansive trade networks and large, crowded populations, were those which ultimately led to its demise. These precarious elements had long lingered as a silent plague within a territorial superpower fully primed to combat the fiercest of invaders, yet one which succumbed to those overlooked behind its volcanic rock fortifications. A hidden tinderbox of similar proportion was ignited to plume within many pathology departments upon inception of the 2019 coronavirus (COVID-19) pandemic [1]. New safety and practice restrictions following the wake of the pathogen’s propagation increased the demand for digital pathology (DP) solutions and remote services. Issues that had lingered throughout many departments were fervently exacerbated, e.g., specialist deficits and demands of shorter turnaround times (TAT) amidst increasing caseloads and complexity of pathology reports for aging patient demographics harboring higher disease incidence. New solutions were necessitated upon the exhumation of long withstanding problems [2,3]. Diagnostic surgical pathology remains the ‘gold standard’ for cancer diagnosis despite substantial inter-observer variability from human error, e.g., bias and fatigue, leading to misdiagnosis of challenging histological patterns and missed identification of small quantities of cancer within biopsy material. Digital (whole slide) imaging, now synonymous with DP, has achieved significant milestones within the last 20 years, with whole slide image (WSI) scanning devices evolving in tandem with challenges pervasive throughout the modern pathology landscape. Batch-scanning and continuous or random-access processing capabilities enabling the concurrent uploading of glass slides during the image capture and digitization processes of others have improved laboratory efficiency [4,5]. Many WSI devices can now handle an array of mediums cast on slides of varying dimensions, with single slide load capacity of some devices reaching up to 1000 [2]. WSI scanning cameras and image sensors deliver superior sensitivity, resolution, field-of-view (FOV), and frame rates for optimal capture and digitization of glass slide specimens [2]. Newer scientific CMOS (sCMOS) sensors are featured in many current WSI scanning devices, often as adjunctive to multiple CCD and CMOS sensors for optimization of image quality.
The Ohio State University (Columbus, OH, USA) was among the first academic institutions to invest in DP devices initially purposed for research and archival, i.e., retrospective scanning of oncology cases [6]. Complete transition to a fully integrated digitized workflow for primary diagnosis followed one year after initial steps toward DP adoption in 2016 (Figure 1).
Figure 1. Digital pathology integration at the Ohio State University James Cancer Hospital and Solove Research Institute (captured by David Kellough of The Ohio State University Comprehensive Cancer Center—Arthur G. James Cancer Hospital and Richard J. Solove Research Institute): (a) Philips UFS scanners; (b) Integrated LCD touchscreen for WSI review; (c) Scan failure indicator; (d) example of scanning error (“Venetian blinding”); (e) Histology laboratory.
Beneficial returns from the preemptive digital transformation were evidenced throughout the first wave of the coronavirus pandemic in 2020, during which the department was well positioned to continue educational and research activities with minimal disruption [6]. Clinical services persisted with relative fluidity with digital workflow emerging as a pillar of stability during an otherwise catastrophic downtime event for many. Temporary remote sign-out authority issued by the Centers for Medicare and Medicaid Services (CMS) emphasized a growing acknowledgment of the utility that digital practice may afford during such times. The substantial percentage of pathologists (71.4%) who were already trained and approved for on-site WSI for primary diagnosis at the department increased to 90.6% during the pandemic (reflecting a conglomerate percentage of pathologists using WSI exclusively for primary diagnosis and those using WSI in conjunction with glass slides). Diagnostic quality assurance (QA) evaluation noted little discrepancy pertaining to the percentage of major and minor diagnostic errors accrued prior to and following the viral catalyzation of digital workflow. Intraoperative consultation services also remained considerably unaffected from digital deployment. Real-time rerouting of slides to available pathologists in different locations increased staffing flexibility. Loosening of work-from-home restrictions including sign-out fostered greater pathologist latitude. Reduction in in-person interactions and the number of individuals handling case materials served to reduce viral transmission while also reducing glass slide contamination potential. An aging population of pathologists at the department, reflective of US specialist demographics, reported greater satisfaction from improved office ergonomics following DP implementation, e.g., forward screen-viewing fostered a more natural reading position in comparison to microscopy techniques requiring bending movements [3]. Lastly, WSI viewing software equipped with tools for WSI annotation, precision measurements, and side-to-side WSI viewing programs with virtual magnification and annotation tools enabled pathologists to effectively collaborate via image sharing and real-time slide examination mimicking laboratory conditions despite working from remote locations. WSI viewing software also facilitated comparison of H&E images to corresponding immunohistochemistry (IHC) or special stained slides, further aiding the ease and efficiency of intradepartmental consultations.
Diagnostic merits of WSI are evidenced in scores of investigations reporting significantly high concordance rates with conventional microscopy throughout numerous disciplines and increasingly for arenas formerly posing hurdles curtailing digital adoption, e.g., cytopathology [7,8,9,10]. Obstacles in modeling business viability for laboratory digitization are surmounted as advanced technology enables a similar roadmap to ubiquitous DP diagnostics already traversed by radiology [10]. The interconnectivity of pathologists, staff, and resources observed following WSI implementation at The Ohio State University reflect a primary endpoint of laboratory digitization. Augmentation of DP tools with artificial intelligence (AI)-based algorithms reflect another. As the university’s primary diagnostic novelty recedes amongst a growing global normalcy of automated workflows, increased efforts for diagnostic quality, and the creation of integrated ecosystems supportive of computational pathology [2,11,12,13], further capitalization from digital integration is now within reach for digitized departments primed to actuate the clinical potential of predictive diagnostic AI-technology for WSI.
2. Development of Computer Vision for Pathology
Computer-aided image processing and pattern recognition, e.g., classification, of histological and cytological structures for pathology has developed from the early 1970s [14,15].
Primordial AI tools for pathology classification tasks typically find genesis at the same vantage point from which modern machine learning (ML) tools began their evolution. Pixel-based analysis, e.g., computer-recognition of a unique series of numerical values that form a shape of interest, is used for classification, e.g., segmentation, tasks that are now among the most essential applications included within integrative workflow image analysis (IA) tools. Traditional morphometric feature evaluation entailed calculation of object size via computational counting of pixels occupied by an object followed by calibration for magnification [16]. Description of object shape resulted from computer determination of a specific shape from a rigid set of preprogrammed rules. Traditional programming directives utilized shape descriptors, e.g., elongation factor and nuclear roundness factor, to identify structures such as peripheral blood erythrocytes. Substantial focus has been directed toward development of computational IA for genitourinary (GU) pathology. Prototypal quantitative light microscopy applications for urological oncology were initially applied to histological sections for rudimentary tumor recurrence and grading predictions [16].
As evidenced from early explorations in computer vision for pathology, traditional programming methods were inherently prone to rapid devolution when image shapes did not adhere to specific pre-programmed rules/definitions, thereby confounding the narrow window of computational interpretability allotted through the ridged training modus. For example, nuclear roundness factor (NRF), defined as the ratio of an area to a perimeter, was observed to decrease when an object shape, e.g., ellipse, deviated from congruence with a perfect circle. The restrictive nature of the programmed code for NRF had predisposed it to conflating “roundness” with “circularity”.
ML techniques have widened the window of interpretability through algorithmic modeling via the use of images rather than preprogrammed rules as input data for algorithm training, allowing computers to correctly visualize shapes regardless of their size, symmetry, or rotation. ML for computational pathology has enabled the interpretive ability of algorithmic tools to extend beyond the limited output yielded from cast-iron programming codes to a system that is able to deduce patterns with increasing accuracy through training. Most current ML approaches utilize methods such as “Random Forest classification”, an algorithmic approach developed in 2001 (the same year as the Leica Aperio T1 gained distinction as the first WSI device released for commercial use) by which a series of decision trees are employed to make an aggregated prediction (Figure 2) [17].
Figure 2. Traditional Programming vs. Machine Learning for Computer Vision (original figures). Squares are representative of pixels comprised of binary graphical indicators for computer recognition, with blue squares comprising a pixelated “input” shape to be recognized by a preset formula that may direct the computer to correctly identifying the shape in its output determination. Green circles are indicative of computer-recognized elements of the pixelated input shape per input programming rules. Red circles represent areas in which programming rules neglected to recognize blue input image elements. Computer programmed rules for defining shapes in figures (a) through (e) are (1) shape is “X” if the center and corner pixels are full and “O” if the center and corner pixels are empty: (a) Pathologist/human interpretation of image: shape is “X”. Computer interpretation of image: shape is “X”, as dictated by rule. Outcome: concordant with pathologist visual interpretation; (b) Pathologist interpretation of image: shape is “O”. Computer interpretation of image: Shape is “O”, as dictated by rule. Outcome: concordant with pathologist visual interpretation; (c,d) Pathologist interpretation of image: shape is “X”. Computer interpretation of images: shape is “O”, as dictated by rules. Outcome: discordant with pathologist visual interpretation; (e) Pathologist interpretation of image: shape is “X”. Computer interpretation of image: image is not recognized, as complete criteria are not fulfilled for either rule. Outcome: discordant with pathologist visual interpretation, i.e., unidentifiable image.
Machine learning allows computers to recognize patterns and make predictive decisions without explicit, step-by-step programming. Trial, error, and extensive “practice” are the core elements of model building for ML, the essence of which follows an iterative approach akin to flashcard memorization. Algorithms are fashioned from a point of zero training data through learning from an output set. A preselected group of image/shape descriptors are chosen by a computer, initially at absolute random, to describe input data fed into the system by a developer. An incorrect label ascribed to an input image by a computer will be amended to display the correct image description from which a machine may demonstrate its learning capacity via correct attribution of the label to a future input image. The ML system takes account of every image pixel and its surrounding pixels with each estimation to ultimately build its own set of rules/algorithms, progressively fashioning an adroit apparatus for predictive accuracy and precision as the cycle continues. Predictive classification models may be tuned and optimized via additional data input providing more opportunities for improvement through trial-and-error for increased accuracy of pattern recognition within new images.
Deep learning (DL) has further expounded upon the cognitive model of ML algorithms, achieving remarkable mimicry to the neural network of the human brain. Artificial neural networks (ANNs) consisting of weighted, interconnected nodes comprise the scaffolding of DL modeling for pathology (Figure 3).