Multi-Distribution Mammogram Classification: Leveraging Clip for Enhanced Generalization Across Diverse Datasets

Mammography is the primary imaging modality that aids in the early detection of breast cancer, a leading cause of cancer-related mortality in women. Recent progress in vision-language models (VLMs)-based applications demonstrate promise for mammogram interpretation through classifying images as normal, benign, or malignant, as defined by the breast imaging reporting and data (BI-RADS) assessment. However, there exists limited evidence of adapting VLMs for external validation (an evaluation of generalization performance) in mammogram classification. This paper introduces multi-distribution mammogram classification with contrastive language-image pre-training (MDMC-CLIP), a strategy built by adapting OpenAI's CLIP model based on a novel strategy that captures dataset-specific diversity besides exploring the homogeneity present across datasets. It not only learns to distinguish between normal, benign, and malignant cases, but also captures subtle variations on a dataset resulting from the combination of several publicly available mammogram datasets. MDMC-CLIP demonstrates robustness against subtle variations in the existing datasets and demonstrates better generalization performance compared to a diverse set of existing VLMs.