Early-stage lung cancer often appears as a subsolid nodule on CT scans. Lung adenocarcinoma - the most common type of non-small cell lung cancer - frequently manifests early as pulmonary nodules that appear either as pure ground glass opacities (hazy areas through which vessels remain visible) or as part-solid nodules with both ground glass and solid components. These subsolid nodules carry a higher malignancy risk than fully solid nodules and often represent a spectrum of disease ranging from pre-invasive lesions to frank invasive cancer.
The biological stakes of invasiveness classification are enormous. The spectrum of lung adenocarcinoma includes atypical adenomatous hyperplasia and adenocarcinoma in situ (pre-invasive), minimally invasive adenocarcinoma (where invasion is limited and well-defined), and fully invasive adenocarcinoma. Non-invasive or minimally invasive lesions carry 100% 5-year recurrence-free survival after complete surgical resection - meaning patients who are correctly identified and treated within this curative time window can effectively be cured. Missing this window by incorrectly classifying a tumor as non-invasive can allow disease progression to an incurable stage.
Current radiological classification of nodule invasiveness is challenging and subjective. Radiologists rely on visual assessment of morphological features - shape, density, spiculation, calcification - to characterize nodules on CT. However, this is known to be challenging and highly variable between observers, since the features distinguishing minimally invasive from fully invasive adenocarcinoma are subtle and require histological confirmation. The gold standard - tissue biopsy or surgical resection - carries procedural risks and is not feasible for all patients.
AI offers a path to objective, non-invasive invasiveness classification. Machine learning and deep learning models can extract quantitative information from CT images that goes far beyond what human eyes can perceive, potentially revealing hidden patterns that correlate with histological invasiveness. This study proposes a novel hybrid framework that combines three complementary types of image-derived information through an attention-based fusion system to classify nodule invasiveness with higher accuracy than any single approach alone.
I-VISTA stands for Integration of Visual, Spatial, and Temporal features with Attention. The framework processes each lung nodule's CT images through three parallel paths simultaneously, each capturing a fundamentally different type of information. The outputs of all three paths are then combined through an attention-based fusion module that learns which features matter most for distinguishing invasive from non-invasive nodules. This design reflects a key insight: no single view of a nodule - whether spatial patterns within a slice, changes across slices, or quantitative texture measurements - tells the whole story.
Path 1 captures spatial features within individual CT slices using a Shifted Window Transformer. The SWin Transformer is a sophisticated vision AI architecture that divides each image into small patches and analyzes relationships between patches at multiple spatial scales. Unlike earlier vision transformers that compare every patch to every other patch globally (computationally expensive), the SWin Transformer computes attention within local windows that shift between processing steps - achieving both local detail and broader context efficiently. This path focuses on spatial patterns within individual views of the nodule.
Path 2 captures temporal features across CT slices using a Convolutional Auto-Encoder Transformer. Because a nodule spans multiple consecutive CT slices, each slice provides a different cross-sectional view showing how the nodule changes in shape, size, and position through the imaging volume. The CAE path first compresses each slice into a compact representation using a convolutional encoder-decoder network, then uses a Transformer to analyze how these representations change sequentially across slices - capturing the three-dimensional character of the nodule from its inter-slice variations.
Path 3 extracts hand-crafted radiomic features from the nodule volume. Using established texture analysis software (TexRAD), the framework applies Laplacian of Gaussian filters at six spatial scales to each nodule's 3D volume. These filters selectively emphasize texture patterns at different granularities - from fine (2 mm scale) to coarse (6 mm scale). For each filter scale, six statistical measures are computed: mean intensity, standard deviation, entropy, mean of positive pixels, skewness, and kurtosis - yielding 36 radiomic features per nodule that quantify the tumor's internal textural heterogeneity.
Combining three heterogeneous feature sets requires a fusion approach that can model their interactions. Simply concatenating all features and feeding them to a linear classifier assumes all features are equally important and interact only additively. Instead, I-VISTA uses a Criss-Cross Attention (CCA) module, a mechanism that can dynamically weight features based on their contextual relevance and capture non-linear interactions across feature sources.
The CCA module works through two sequential operations: affinity and aggregation. In the affinity step, the module measures the similarity between features from different paths in both horizontal and vertical directions simultaneously - hence "criss-cross." This directional attention allows each feature to gather contextual information from all other features that share its row or column in the feature map, without the quadratic computational cost of comparing every feature to every other feature globally.
The aggregation step combines the weighted feature maps into a final unified representation. Having computed attention weights reflecting feature relevance and similarity, the module performs a weighted sum to produce an integrated feature vector that preserves the most discriminative patterns across all three input sources. This final representation is passed to a dense classification layer with two outputs (invasive vs. non-invasive) and a softmax activation.
The CCA approach was originally designed for image segmentation but was re-purposed here for feature fusion. By assigning the spatial, temporal, and radiomic feature maps to the Query, Key, and Value roles in the attention mechanism respectively, the researchers adapted the CCA to serve as a smart combinator for heterogeneous medical image features - a novel application that is central to I-VISTA's design.
The study used an in-house dataset of 114 pathologically confirmed subsolid nodules. All nodules were surgically resected and classified by pathology as either Group 1 (G1, low-risk: 58 nodules including atypical adenomatous hyperplasia, adenocarcinoma in situ, and minimally invasive adenocarcinoma) or Group 2 (G2, high-risk: 56 invasive adenocarcinomas). CT scans were acquired without contrast using 1 to 3 mm slice thickness, representing realistic routine clinical practice with natural variation in acquisition parameters.
Expert radiologist annotation anchored the study at both segmentation and interpretation levels. A thoracic radiologist with 19 years of experience identified which CT slices contained the nodule and validated all segmentations. This expert oversight ensures that the deep learning models are trained on correctly identified nodule regions rather than surrounding tissue, and that the study's ground truth is clinically sound.
Ten-fold cross-validation was used to maximize the statistical reliability of evaluation on a small dataset. With only 114 cases, randomly splitting into training and test sets even once would yield unreliable estimates. By rotating through ten different train-test splits and averaging performance across all ten, the evaluation provides stable and reproducible estimates of true generalization performance. All hyperparameter optimization was conducted exclusively within training folds to prevent information leakage.
Pre-training on larger public datasets compensated for the limited in-house data. The SWin Transformer was pre-trained on ImageNet-21k (a massive natural image dataset) before fine-tuning on the nodule data, allowing it to leverage general visual feature detection capabilities. The CAE was pre-trained on the LIDC-IDRI lung nodule dataset before fine-tuning. This transfer learning approach is essential when training sophisticated deep networks on small medical imaging cohorts.
I-VISTA achieved 93.93% accuracy and AUC of 0.93 across ten folds. The full hybrid model combining all three paths achieved mean accuracy of 93.93%, sensitivity of 92.66% (correctly identifying invasive nodules), and specificity of 94.99% (correctly identifying non-invasive nodules). The AUC of 0.93 indicates excellent overall discriminatory ability. These results represent a substantial improvement over the reference radiomics-only model from the same institution, which achieved 81% accuracy and AUC of 0.89 on the same type of data.
The best classical machine learning approach with radiomic features alone achieved only 79.16% accuracy. Random Forest applied to 36 hand-crafted radiomic features - the best-performing traditional classifier - reached accuracy of 79.16% and AUC of 0.88. While this baseline is already clinically meaningful, the 15-percentage-point gap in accuracy between Random Forest and I-VISTA demonstrates the substantial added value of incorporating deep learning features and attention-based fusion.
Neither deep learning path alone matched the hybrid model. The SWin Transformer alone achieved 78.10% accuracy, while the CAE-Transformer alone achieved only 69.46%. This confirms that spatial information alone (within-slice patterns) and temporal information alone (across-slice variations) are each insufficient - both are needed, and both are further enhanced when combined with the quantitative radiomic perspective. The performance hierarchy (CAE alone, SWin alone, combined DL, full hybrid) illustrates that each successive integration step adds genuine value.
Model stability across folds was notably better in the hybrid than in single-path models. Statistical analysis using Kruskal-Wallis and Friedman tests found no significant differences in performance across the ten folds (p greater than 0.05), and the coefficient of variation across folds was only 7.23% - well below the 10% threshold considered acceptable for clinical AI. This consistency reflects the robustness that comes from combining complementary information sources, each of which independently reduces sensitivity to the specific cases that fall in any particular test fold.
Hand-crafted radiomic features capture biologically grounded but limited information. By explicitly measuring texture at multiple spatial scales, radiomic features reflect known biological properties of tumor heterogeneity that correlate with invasiveness. However, they are constrained by predefined calculations that may miss complex patterns not anticipated by the feature designers. Their consistent performance across many studies makes them a valuable foundation, but they cannot adapt to novel visual patterns the way deep learning can.
The SWin Transformer's performance advantage over the CAE-Transformer reflects the primacy of spatial context. Because lung nodules are small relative to the surrounding lung parenchyma, identifying the specific spatial patterns within nodule patches - the task of the SWin Transformer - is more directly diagnostic than analyzing how nodule appearance changes across slices. However, neither dimension alone suffices: small nodules whose appearance varies subtly across slices require both spatial and temporal perspectives for reliable characterization.
The Criss-Cross Attention fusion goes beyond simple feature concatenation. Most prior hybrid models simply concatenate features from different sources and train a linear classifier, which assumes features combine additively. The CCA module instead learns the non-linear interactions and contextual dependencies between spatial, temporal, and radiomic features - dynamically weighting each according to how informative it is in combination with the others for each specific nodule. This flexible, data-driven integration is what produces the dramatic performance leap.
The study also demonstrates the practical efficiency of using a focused small radiomic feature set. Many radiomic studies extract thousands of features and then apply complex selection algorithms. Here, 36 biologically meaningful features extracted at six spatial scales proved sufficient - avoiding the overfitting risk that comes with very high-dimensional feature spaces in small datasets, while providing complementary information that neither deep learning path captures independently.
Accurate non-invasive invasiveness classification could transform the management of subsolid nodules. Currently, the path from CT detection to invasiveness classification requires surgical resection or biopsy. An AI tool that reliably distinguishes invasive from non-invasive subsolid nodules from CT alone could guide clinical decisions: non-invasive lesions might be safely monitored with CT follow-up, while invasive lesions would be prioritized for earlier intervention. This triage capability has significant implications for resource allocation and patient outcomes.
The study's retrospective design and single-institution data limit immediate generalizability. Because only surgically resected nodules were included, the dataset contains selection bias - nodules that were left under surveillance rather than resected are excluded. The 114-nodule cohort from one Canadian institution may not fully represent the variability in CT acquisition protocols, patient demographics, and pathological distribution seen globally. External validation on independent multicenter datasets is the essential next step.
The interpretability of deep learning components remains a challenge for clinical deployment. While the radiomic path provides biologically interpretable features, the SWin Transformer and CAE-Transformer operate as relative black boxes - their internal reasoning is not directly explainable to radiologists. Integrating attention visualization methods or other explainability tools would enhance clinician trust and support regulatory acceptance. This is recognized as an ongoing area of research.
The framework's tolerance for non-thin CT slices is a practical strength. Many AI models for lung nodule assessment require thin-slice (1 mm or less) CT acquisitions that are not universally available in routine clinical practice. I-VISTA was developed and tested on CT scans with 1 to 3 mm slice thickness - the realistic range used in standard clinical workflows - making it more immediately applicable to real clinical settings than models requiring specialized acquisition protocols.
The I-VISTA framework demonstrates that hybrid deep learning and radiomics outperforms either approach alone. By integrating spatial features from the SWin Transformer, temporal inter-slice features from the CAE-Transformer, and quantitative 3D radiomic features through a Criss-Cross Attention fusion module, I-VISTA achieved 93.93% accuracy and AUC of 0.93 for classifying lung nodule invasiveness - substantially outperforming both classical radiomic models and standalone deep learning architectures on the same dataset.
The three-way attention fusion approach represents a novel contribution to hybrid medical imaging AI. Previous hybrid models combined deep and hand-crafted features primarily through concatenation and linear fusion. I-VISTA's use of CCA for three-way non-linear feature integration is, to the authors' knowledge, the first application of this mechanism in lung nodule invasiveness classification, and the results confirm its effectiveness in leveraging complementary information from heterogeneous feature sources.
Future work should focus on external validation and interpretability. The model's clinical translation pathway requires validation on multicenter prospective datasets, development of attention visualization methods to support radiologist understanding, and extension to multi-class classification of the full adenocarcinoma spectrum rather than binary invasive versus non-invasive. The framework's code and dataset are available to qualified researchers, supporting the broader research community in building on these results.