Improving CNNs classification with pathologist-based expertise: the renal cell carcinoma case study

Sci Rep 2023 AI 7 Explanations View Original
Original Paper (PDF)

Unable to display PDF. Download it here or view on PMC.

Plain-English Explanations
Pages 1-2
The Limitation of Standard CNNs for RCC Subtype Classification

Convolutional neural networks applied to histopathology classification typically follow a flat architecture: a single backbone network processes the input image and outputs class probabilities for all subtypes simultaneously. While this approach works well for visually distinct categories, it struggles when the diagnostic logic a pathologist uses is inherently hierarchical.

Expert pathologists do not diagnose RCC subtype by comparing all possibilities simultaneously. Instead, they follow a decision tree based on characteristic features: first assessing nuclear morphology, then cytoplasmic characteristics, then architectural patterns, in a structured sequence that mirrors clinical diagnostic guidelines.

Standard CNNs have no built-in mechanism to replicate this structured reasoning. They must learn to perform all discriminations from a single shared feature space, which may be suboptimal when the features that distinguish two classes are not the same as those that distinguish other class pairs.

This paper proposed ExpertDeepTree (ExpertDT), a novel CNN architecture that embeds the hierarchical decision structure used by expert pathologists directly into the model's topology, creating a tree-shaped network that mirrors the clinical diagnostic workflow.

TL;DR: Standard CNNs cannot replicate pathologists' hierarchical reasoning, motivating a tree-structured CNN that mirrors expert diagnostic logic.
Pages 2-4
ExpertDeepTree: Architecture Inspired by Pathologist Decision Trees

ExpertDT consists of four specialized CNN components arranged in a tree structure: a Root CNN, a Node CNN, and two leaf-level classifiers (Leaf1 and Leaf2). This topology was designed in direct collaboration with expert renal pathologists who articulated their diagnostic decision process.

The Root CNN processes all input images and performs a first-level discrimination, determining whether the tumor is more likely to belong to one major diagnostic group or another based on high-level visual features. Its output routes each sample to the appropriate Node or Leaf network for further refinement.

The Node CNN receives samples passed from the Root and applies a second level of discrimination using features relevant to distinguishing the subtypes within that branch of the tree. This component focuses on the specific visual characteristics that differentiate clear cell from papillary RCC, for example, rather than trying to learn all distinctions simultaneously.

Leaf1 and Leaf2 are the terminal classifiers that output final subtype assignments for their respective branches of the tree. Each leaf is trained only on the subset of cases relevant to its branch, allowing it to develop highly specialized feature representations for the most challenging fine-grained distinctions.

All CNN components in ExpertDT share a common backbone architecture but are trained independently on the subsets of data relevant to their position in the tree. This training strategy ensures each component develops the specialized expertise appropriate for its role in the diagnostic hierarchy.

TL;DR: ExpertDT is a four-component tree-structured CNN (Root, Node, Leaf1, Leaf2) that routes cases through diagnostic branches mirroring pathologist reasoning.
Pages 3-4
Dataset and Study Cohort

The study used histopathology images from 91 RCC patients representing four subtypes: clear cell RCC (ccRCC), papillary RCC (papRCC), chromophobe RCC (chrRCC), and oncocytoma (ONCO). Oncocytoma is a benign tumor that is pathologically distinct from RCC but is included because it frequently enters the differential diagnosis and requires accurate discrimination from chromophobe RCC.

Hematoxylin and eosin (H and E) stained slides were digitized and divided into image patches for model training and evaluation. Patch-based analysis is standard in computational pathology because whole slide images are too large to process as single inputs to a CNN, requiring tiling into smaller sub-regions that are subsequently aggregated.

The relatively small cohort of 91 patients is a limitation acknowledged by the authors. Transfer learning from ImageNet-pretrained weights was used to compensate for limited training data, leveraging the general visual features learned from millions of natural images to initialize the network before fine-tuning on histopathology images.

Cross-validation was used rather than a single train-test split to produce more reliable performance estimates given the small dataset. This approach involves repeatedly partitioning the data into training and test sets, training a model on each partition, and averaging performance across folds.

TL;DR: The cohort of 91 patients with 4 subtypes was augmented by transfer learning and cross-validated to produce robust performance estimates.
Pages 5-7
Performance Improvement Over Standard CNN Baseline

ExpertDT achieved an overall accuracy of 95% on the 4-class RCC subtype classification task. The best-performing standard CNN baseline achieved 81% accuracy on the same task and dataset, representing a 14-percentage-point improvement attributable to the expert knowledge-guided architecture.

Matthews Correlation Coefficient (MCC) is reported alongside accuracy as a more balanced metric for multi-class classification. ExpertDT achieved MCC of 0.80 compared to 0.64 for the baseline, indicating substantially better correlation between predicted and actual subtype labels across all class pairs.

MCC is particularly informative in multi-class settings with class imbalance because it accounts for all cells in the confusion matrix simultaneously, unlike accuracy which can be dominated by performance on the most common class. The larger MCC gap (0.80 vs 0.64) compared to the accuracy gap confirms that ExpertDT's improvement was genuine rather than driven solely by better performance on the majority class.

Per-class analysis showed improvements across all four subtypes, with the largest gains for oncocytoma versus chromophobe RCC discrimination, which is the most clinically challenging distinction and the one most directly benefiting from the specialized leaf-level classifiers in the tree architecture.

TL;DR: ExpertDT achieved 95% accuracy and MCC 0.80, compared to 81% accuracy and MCC 0.64 for the best standard CNN baseline.
Pages 7-8
How Expert Knowledge Improved Model Generalization

The key insight underlying ExpertDT is that structuring a model to solve subproblems that are individually simpler than the full classification task enables better learning from limited data. Each component of the tree only needs to learn the features relevant to one branch of the decision process, reducing the complexity each individual CNN must master.

This partitioned training approach also reduces the risk of cross-contamination between discriminative features for different class pairs. In a standard CNN, gradients from difficult class pairs can interfere with the learning of features for easier distinctions, leading to suboptimal representations. The tree structure prevents this interference by separating the optimization targets.

The collaboration with expert pathologists to define the tree topology was essential: without domain knowledge about which distinctions are hierarchically prior, a tree architecture could impose the wrong structure and perform worse than a flat approach. The success of ExpertDT demonstrates the value of human-AI collaboration in model design, not just in labeling.

TL;DR: The tree structure partitions learning into manageable subproblems, preventing feature interference and leveraging pathologist-defined diagnostic hierarchy.
Pages 8-9
Implications for Pathologist-AI Collaboration

ExpertDT demonstrates that AI models can be made more interpretable and trustworthy by aligning their computational structure with the diagnostic reasoning of clinical experts. A pathologist reviewing ExpertDT's output can understand which branch of the tree was activated and what first-level discrimination was made before the final subtype was assigned.

This transparency is a significant advantage over black-box CNN approaches where the same input image produces a classification without any indication of the reasoning path followed. Interpretable AI is more likely to be accepted by clinicians and regulators who need to understand and audit AI decision-making.

The approach is generalizable beyond RCC: any cancer type with a well-defined hierarchical diagnostic taxonomy could potentially benefit from tree-structured CNN architectures designed in collaboration with domain experts. Breast cancer (luminal A, luminal B, HER2-positive, triple-negative) and lymphoma subtyping are natural candidates.

TL;DR: ExpertDT's interpretable tree structure increases clinician trust and is generalizable to other cancers with hierarchical diagnostic taxonomies.
Pages 9-10
Summary and Future Work

ExpertDeepTree demonstrates that embedding domain expert knowledge directly into CNN architecture design, rather than solely into labeled training data, produces substantial accuracy improvements for RCC subtype classification from histopathology images.

The 14-percentage-point accuracy gain and MCC improvement from 0.64 to 0.80 over the best standard CNN baseline are clinically significant, particularly for the most challenging discriminations such as oncocytoma versus chromophobe RCC that carry direct treatment implications.

Validation on larger multi-center cohorts is needed to confirm that ExpertDT's advantage holds as dataset size increases and variability across staining protocols and scanner types is introduced. The small 91-patient cohort used in this study may not capture the full range of morphologic presentations encountered in clinical practice.

Future work could also explore dynamically routing cases through the tree based on uncertainty estimates, allowing the model to self-select the most appropriate diagnostic branch for each input rather than following a fixed routing determined by the prior node's output.

TL;DR: ExpertDT's pathologist-guided tree architecture sets a new accuracy benchmark for RCC subtype classification and points toward interpretable AI for pathology.
Citation: Open Access, 2023. Available at: PMC10517931.