Bone tumors arise in the musculoskeletal system and span a wide biological spectrum, from benign lesions that require only observation to highly aggressive malignancies that demand immediate surgery, chemotherapy, or radiation. Primary bone tumors occur at a rate of 2 to 3 per 100,000 people and account for roughly 6.2% of all tumors. Among them, osteosarcoma is the most prevalent primary malignant bone tumor and is the second leading cause of cancer-related death in adolescents and children. Without timely detection and intervention, malignant bone tumors carry serious risks including amputation, distant metastasis, and death. Early limb salvage surgery, which preserves both function and life, depends almost entirely on catching the tumor at a stage where it has not yet spread.
The conventional diagnostic process for bone tumors combines clinical history, radiological imaging (X-ray, CT, MRI), and histopathological examination of biopsy tissue. While this multi-modal approach is the accepted standard, it is slow, expensive, and susceptible to significant inter-observer variability driven by clinician expertise. Bone tumors are rare enough that even seasoned radiologists and orthopedic oncologists encounter them infrequently, making pattern recognition genuinely difficult. The result is a system where misdiagnosis and delayed diagnosis are common pitfalls, carrying direct consequences for patient outcomes.
Deep learning entered this space as a response to these diagnostic challenges. As a subset of machine learning, deep learning trains multi-layer neural networks, particularly convolutional neural networks (CNNs), directly on imaging data to learn discriminative features without manual feature engineering. Since AlexNet outperformed all traditional methods at the 2012 ImageNet competition, deep learning has progressively penetrated medical imaging for tasks such as fracture detection, cartilage lesion identification, osteoarthritis severity grading, and cancer classification in prostate, breast, and lung tumors. This review, published in Frontiers in Oncology in 2022 by Zhou, Wang, Tu, and colleagues from Central South University (Hunan, China), synthesizes the emerging body of work applying deep learning specifically to bone tumors, covering both primary tumors and skeletal metastases across radiological and pathological imaging modalities.
Scope of the review: The authors cover detection, segmentation, volume calculation, grading, tumor necrosis rate prediction, recurrence prediction, and prognosis estimation. Imaging modalities examined include plain radiography, CT, MRI, bone scintigraphy (99mTc-MDP), PET/CT, spectral CT, and whole-slide digital pathology (H&E-stained tissue images). The review also discusses radiomics as a complementary quantitative imaging approach and critically examines barriers to clinical deployment.
Deep learning is a subset of machine learning in which the model learns hierarchical representations directly from raw data. Unlike traditional machine learning, which requires domain experts to manually engineer features (such as tumor margin sharpness or density gradients), deep learning uses multiple stacked layers to extract progressively abstract representations. Early convolutional layers detect low-level features like color, edge orientation, and texture; deeper layers detect complex semantic structures such as tumor borders, necrosis patterns, and vascular involvement. This automated feature extraction eliminates the bottleneck of manual engineering and scales well when large annotated datasets are available.
CNN architecture: The dominant architecture in medical image analysis is the convolutional neural network (CNN). A typical CNN consists of convolution layers (which apply learned filters to extract spatial features), pooling layers (which downsample feature maps to reduce computational cost and introduce translation invariance), nonlinear activation functions (most commonly ReLU), and fully connected layers (which perform final classification or regression). Established architectures such as VGG, ResNet, DenseNet, EfficientNet, and Inception have all been adapted for bone tumor imaging tasks, often through transfer learning from ImageNet-pretrained weights.
Building a model pipeline: The authors describe a standard three-stage workflow. First, image preprocessing ensures uniform format, resolution, and orientation, often accompanied by data augmentation techniques such as digitally reconstructed radiographs (DRRs) or generative adversarial networks (GANs) to artificially expand small datasets. Dataset splitting typically follows a 70%:20%:10% ratio for training, validation, and testing. Second, the model is trained on labeled data with iterative parameter refinement via backpropagation and gradient descent. Third, model performance is evaluated on the held-out test set using metrics including accuracy, sensitivity, specificity, AUC of the ROC curve, F1-score, and Dice similarity coefficient (DSC) for segmentation tasks.
Labeling as the critical bottleneck: Image annotation, where experienced radiologists or pathologists manually mark tumor regions on thousands of images, is described as the most time-consuming and critical step. These annotations serve as the ground truth ("gold standard") against which the model learns and is later benchmarked. In bone tumors, the annotation burden is compounded by the rarity of cases and the expertise required. Running-time constraints are also addressed: the authors note that when speed and accuracy must be balanced, typical clinical deployment targets inference times under 100 milliseconds per case.
Plain radiographs remain the first-line imaging modality for bone tumors because they are widely available and inexpensive. Deep learning has been applied directly to X-rays for both detection and classification. In 2020, He et al. developed a CNN-based model trained on 2,899 plain radiographs from 1,356 patients at multiple institutions to classify primary bone tumors as benign, intermediate, or malignant. The model achieved AUC values of 0.877 for benign classification and 0.916 for malignant classification. Three-way classification accuracy reached 72.1%, a performance level comparable to subspecialist radiologists and meaningfully above that of junior radiologists. In a separate study, Do et al. applied a Multi-Level Seg-Unet architecture to detect and classify knee bone tumors from 1,576 radiographs (1,195 tumor images and 381 normal images), achieving a remarkable accuracy of 99.05% with a mean intersection-over-union (IoU) of 0.848.
CT imaging: CT provides three-dimensional structural detail that plain radiographs cannot capture, including cortical destruction, periosteal reaction, and soft tissue extension. Yin et al. developed a deep neural network (DNN) using 3D CT images combined with clinical characteristics from 459 patients, generating 1,316 manually cropped radiomics features. This model distinguished benign from malignant sacral tumors with an AUC of 0.83 to 0.84. Masoudi et al. applied both 2D ResNet-50 and 3D ResNet-18 architectures to classify bone lesions as benign or malignant using 2,880 annotated CT scans from 114 prostate cancer patients, achieving 92.2% accuracy and an F1-score of 92.3%.
MRI imaging: MRI is particularly sensitive for detecting bone marrow involvement, soft tissue invasion, and fluid content, features that CT and X-ray cannot reliably resolve. Eweje et al. applied the EfficientNet-B0 architecture combined with a logistic regression model to classify benign and malignant bone lesions from routine MRI images and patient demographics across 1,060 images (split 70/20/10). The model achieved expert-level performance: accuracy 0.76, sensitivity 0.79, specificity 0.75, and AUC 0.82. Liu et al. used a CNN based on Inception_v3 applied to X-ray images and clinical characteristics from 982 images, achieving AUCs of 0.898 for benign, 0.894 for malignant, and 0.865 for intermediate classification, with a macroaverage AUC of 0.872.
Clinical relevance: The consistent theme across these studies is that deep learning models trained on multi-institutional datasets and validated against specialist readers can either match or exceed the diagnostic accuracy of junior clinicians in specific classification tasks. However, the authors note that most models were trained on single-task frameworks and have not been tested prospectively in live clinical workflows.
Segmenting the exact boundaries of a bone tumor on imaging is a prerequisite for accurate volume measurement, surgical planning, and radiation field definition. It is also one of the most time-consuming tasks in musculoskeletal oncology, where tumor geometry is highly variable across cases. Huang et al. addressed this in 2017 by developing a Multiple Supervised Fully Convolutional Networks (MSFCN) approach to automatically segment osteosarcoma on CT. Trained on 1,900 images and tested on 405 images from 23 patients (using VGG-16 as the backbone with multiple supervised side-output layers), the model achieved a Dice similarity coefficient (DSC) of 87.80%, sensitivity of 86.88%, and F1-measure of 0.908. A related Multiple Supervised Residual Network (MSRN) model incorporating ResNet with FCN also performed at high accuracy across 1,900 CT images from 15 osteosarcoma patients.
Pelvic tumors and 3D context: Pelvic bone tumors present a particularly challenging segmentation problem because of their complex anatomical surroundings and large spatial variability. A multiview fusion network designed to extract pseudo-3D spatial information was applied to pelvic bone tumor MRI segmentation and volume calculation. The model reduced average segmentation time by a factor of 100 compared to conventional methods, a clinically significant efficiency gain given the laborious nature of manual contouring in radiation therapy planning.
Tumor grading: Grading bone and soft tissue tumors into low-grade (G1) and high-grade (G2/G3) is critical for treatment selection, as high-grade tumors require neoadjuvant chemotherapy before surgery. Navarro et al. developed a deep learning grading model applied to T1-weighted and T2-weighted MRI sequences in soft-tissue sarcoma patients. The model achieved an F1-score of 0.90 and AUC of 0.76, performance that could potentially guide surgical planning while reducing the number of invasive biopsies. The authors note that analogous models have not yet been developed for primary bone tumors, representing an open research gap.
Radiomics, which extracts quantitative texture, shape, and intensity features from imaging, is frequently used alongside deep learning models in these segmentation and grading tasks. Because radiomic features are mathematically defined, they can capture imaging biomarkers invisible to the human eye, and pairing them with CNN-derived feature maps has been shown to improve performance in several tumor types, though bone tumor-specific validation is still limited.
Neoadjuvant chemotherapy is the standard of care before surgery for high-grade osteosarcoma and Ewing sarcoma. After surgery, pathologists measure the proportion of necrotic versus viable tumor cells in the resected specimen: a necrosis rate of 90% or greater defines a "good responder" and carries a substantially better prognosis than "poor responders" below that threshold. However, this assessment requires analysis of multiple pathological slices across the entire resected tumor, a process that is labor-intensive, highly variable, and only available after surgery, limiting its utility for real-time treatment adaptation.
Predicting necrosis non-invasively: Huang et al. conducted a preliminary study applying traditional machine learning to multiparametric MRI before and after chemotherapy in osteosarcoma patients, demonstrating a potential correlation between contrast-enhanced MRI signal characteristics and postoperative pathological necrosis features. Kim et al. extended this approach using PET/CT: texture features derived from 18F-FDG PET/CT images captured fluorodeoxyglucose uptake heterogeneity in osteosarcoma and predicted treatment response to neoadjuvant chemotherapy. While these studies used supervised ML rather than deep learning due to small sample sizes, the authors argue that expanding datasets would enable CNN-based prediction of histologic response using both MRI and PET/CT, potentially allowing oncologists to adapt chemotherapy regimens before the surgery date.
Local recurrence prediction: He et al. applied a CNN to presurgery MRI from 56 patients with confirmed giant cell bone tumors (GCTB) who underwent curettage to predict postoperative local recurrence. The pure image-based CNN (no clinical variables) achieved an accuracy of 75.5% and sensitivity of 85.7%. When the CNN output was combined with tumor location, patient age, and other clinical parameters using logistic regression, accuracy improved to 78.6% and sensitivity to 87.5%. Both figures significantly outperformed experienced radiologists, who achieved only 64.3% accuracy and 58.3% sensitivity on the same task. This demonstrates that fusion models integrating imaging-derived deep features with structured clinical data can generate actionable recurrence risk estimates that exceed specialist human judgment.
The clinical implication is significant: if a CNN model can reliably flag high-recurrence-risk GCTB cases before curettage, surgeons could elect more aggressive primary resection rather than accepting higher recurrence rates from less invasive curettage. These are exactly the kinds of surgical decision points where data-driven predictions can meaningfully change outcomes.
Bone metastases are far more common than primary bone tumors and represent a major source of morbidity in patients with advanced prostate, breast, lung, and other cancers. The standard screening tool for skeletal metastasis is whole-body bone scintigraphy with 99mTc-MDP, a nuclear medicine modality with high sensitivity but poor specificity, making it prone to false-positive findings from benign conditions such as degenerative arthritis or prior fractures. Deep learning has been applied extensively to bone scintigraphy to improve both sensitivity and specificity and to reduce radiologist workload in interpreting these scans.
Scintigraphy-based models: Papandrianos et al. developed two RGB-CNN models for classifying bone scintigraphy as metastatic, degenerative, or healthy in prostate cancer patients. Despite using small datasets (778 patients for the two-class model and 507 patients for the three-class model) without any clinical input, classification accuracy reached 97.38% for two-class and 91.42% for three-class, outperforming other standard CNN approaches. Pi et al. trained a CNN on 15,474 examinations from 13,811 patients to simultaneously analyze both anterior and posterior whole-body scan views using a spatial attention feature aggregation operator. This model achieved F1 of 0.933, accuracy of 95.00%, sensitivity of 93.17%, and specificity of 96.60%. Zhao et al. built a multi-input CNN trained on 12,222 bone scintigraphy cases to identify metastasis across cancer types, achieving AUCs of 0.988 for breast cancer, 0.955 for prostate cancer, 0.957 for lung cancer, and 0.971 for other cancers.
CT and MRI for metastasis: Chmelik et al. developed a CNN-based method for classifying and segmenting spinal metastases (both lytic and sclerotic) in whole-spine CT scans from 31 cases with 1,046 lytic and 1,135 sclerotic lesions, achieving 92% sensitivity for lesions greater than 1.4 mm3 under object-wise evaluation. Lindgren Belal et al. trained a CNN on 100 CT scans capable of automatically segmenting and calculating the volume of skeletal metastatic lesions at a performance level comparable to experienced radiologists. Fan et al. used a deep approach combining AdaBoost classification with Chan-Vese segmentation on spinal metastasis MRI in lung cancer patients, achieving 96.55% classification accuracy.
Identifying the primary tumor: A distinct and clinically valuable task is determining the origin of a skeletal metastasis when the primary cancer is unknown. Lang et al. developed a convolutional long short-term memory (CLSTM) network applied to dynamic contrast-enhanced MRI (DCE-MRI) of the spine, differentiating metastases originating from lung cancer versus other cancers across 61 patients. The model achieved 0.81 accuracy, outperforming both hotspot-based methods and radiomics approaches. When PET/CT is unavailable or too expensive, this type of model could guide the diagnostic workup toward identifying the primary cancer source.
Histopathological examination of formalin-fixed paraffin-embedded (FFPE) tissue sections stained with hematoxylin and eosin (H&E) remains the gold standard for definitive bone tumor diagnosis. In osteosarcoma, pathological review of resected tumor specimens also serves as the primary method for quantifying tumor necrosis after neoadjuvant chemotherapy, a task requiring evaluation of multiple large-format whole-slide images (WSIs) by experienced pathologists. Deep learning has been applied to automate and standardize this time-intensive process.
Necrosis classification in osteosarcoma: Arunachalam et al. trained a CNN classifier on 536 non-tumor tiles, 263 necrotic tumor tiles, and 345 viable tumor tiles annotated by two independent pathologists from 40 WSIs (1,144 total tiles). The model classified osteosarcoma WSIs into viable tumor, necrotic tumor, and non-tumor regions with an accuracy of 91.2%, and generated tumor-prediction maps that visually overlay the classified regions onto the slide. Mishra et al. developed a similar CNN achieving 92.4% accuracy, precision of 97%, and F1-score of 95%. Fu et al. used a DS-Net classifier, composed of an alignment subnetwork (ASN) and a classification network (CN) with a Siamese architecture and a novel Dual Residual Block (DRB). Using paired input data labeled by experts and trained on 1,144 images (60/20/20 split), DS-Net achieved 95.1% accuracy, 92.0% sensitivity, 96.1% specificity, 92.9% precision, and F1-score of 92.2%.
Subtype classification and sarcoma survival: Foersch et al. demonstrated that a CNN classifier could distinguish subtypes of soft tissue sarcoma from histopathological slide images alone. Critically, pathologist accuracy on the same task improved from 46.3% to 87.1% when augmented with the CNN output, representing a 40.8 percentage-point improvement driven by AI assistance rather than AI replacement. For survival analysis, CNN classifiers applied to H&E-stained slides of leiomyosarcoma achieved a mean AUC of 0.91 and accuracy of 88.9%, using class activation maps to identify survival-predictive tissue features including reduced lymphoid infiltrates, prominent intercellular matrix, intratumoral hemorrhage, and increased tumor-associated vasculature. These features align with established prognostic markers in sarcoma biology, validating that the model had learned clinically meaningful patterns rather than confounders.
The automated quantification of tumor necrosis from WSIs is particularly clinically significant. If a CNN can reliably estimate necrosis percentage at a fraction of the time required by manual pathological review, it could accelerate the generation of histologic response scores, reduce pathologist workload, and potentially enable more standardized scoring across institutions.
Despite the promising performance metrics reported across the studies covered in this review, the authors identify six concrete limitations that explain why virtually none of these models have been integrated into routine clinical bone tumor care. Each limitation is structural rather than superficial and will require coordinated effort from clinicians, data scientists, and regulators to address.
Data scarcity and annotation cost: Bone tumors have a low population incidence (2 to 3 per 100,000), which means that even major referral centers accumulate cases slowly. The annotated datasets underpinning most published models are small by deep learning standards, with many studies involving fewer than 200 patients. Obtaining labeled data from experienced oncologists is expensive, time-consuming, and dependent on individual institutional cooperation. Medical imaging data also carries strong privacy constraints, and regulatory frameworks in many countries impose significant barriers on cross-institutional data sharing. Public databases like TCGA and Cancer Genome Atlas contain abundant data for common cancers but are largely absent of bone tumor cases.
Single-task model design: Nearly all published models are designed to perform one specific task: classify lesion as benign or malignant on CT, or segment osteosarcoma on MRI, or detect bone metastasis on scintigraphy. In clinical practice, diagnosis of osteosarcoma requires synthesis of clinical history, plain radiographs, CT, MRI, biopsy histology, and often IHC biomarkers (Ki-67, SATB2, MDM2) or next-generation sequencing (NGS) results. Single-input, single-output models cannot replicate this integrative reasoning. Multi-modal, multi-task architectures that process heterogeneous data types in parallel are necessary for clinical utility but are substantially harder to train and validate.
Lung metastasis gap: Lung metastasis is the primary cause of death in osteosarcoma and other malignant bone tumor patients, yet no deep learning models have been developed specifically for detecting or characterizing pulmonary spread of primary bone tumors. Deep learning has been applied successfully to lung nodule detection in other contexts, and the authors argue that extending these methods to bone tumor lung metastases, combined with image reconstruction techniques that reduce CT radiation dose, is a high-priority research direction.
Genomics and spectroscopy frontiers: Gene expression profiling can predict tumor subtypes and prognosis, but remains costly for routine use. Deep learning models trained on genomic and transcriptomic data have shown promise in lung and breast cancer prediction, but no equivalent work exists for bone tumors. Similarly, Raman spectroscopy provides label-free molecular information and has been combined with deep learning in several cancer types, including one CNN model using 1,281 serum Raman spectra from 427 patients to identify prostate cancer bone metastases. No Raman spectroscopy plus deep learning study has been published for primary bone tumors, representing an unexplored but potentially valuable avenue. Transfer learning, in which models pretrained on abundant imaging data from common diseases are fine-tuned on small bone tumor datasets, is identified as the most practical near-term strategy to overcome data scarcity.