RadAlign: Advancing Radiology Report Generation with Vision-Language Concept Alignment

RadAlign is a novel framework designed to enhance automated chest radiograph interpretation. It combines the predictive accuracy of Vision-Language Models (VLMs) with the reasoning capabilities of Large Language Models (LLMs) to bridge the gap between disease classification and radiology report generation. This repository contains the source code, model configurations, and resources to reproduce the results presented in our paper.

Table of Contents

• Introduction
• Features
• Results
• Acknowledgments

Introduction

Automated radiology report generation is a critical challenge in clinical workflows. Traditional methods focus either on disease classification accuracy or text generation but fail to integrate the two seamlessly. RadAlign overcomes these limitations by aligning visual features with medical concepts to generate detailed, interpretable, and clinically accurate radiology reports.

Key Contributions

1. Vision-Language Concept Alignment: RadAlign leverages VLMs to align visual features with text-based medical concepts, ensuring clinically relevant disease classification.
2. Retrieval-Augmented Generation: The framework retrieves similar historical cases to ground report generation in validated data, reducing hallucinations.
3. Integrated Predictive and Generative AI: By combining VLM-based classification with LLM-based reasoning, RadAlign mirrors the workflow of professional radiologists.
4. Superior Performance: Achieves an average AUC of 0.885 for classification and a GREEN score of 0.678 for report generation, outperforming state-of-the-art methods.

Features

• Disease classification with high precision, F1-score, and AUC.
• Retrieval-augmented report generation using similar historical cases.
• Explainable decision-making through concept-token attention visualization.
• Lightweight prompting mechanism for efficient report synthesis.

Results

• Classification: Average AUC of 0.885 across five common diseases.
• Report Generation: GREEN score of 0.678, indicating superior report quality with reduced hallucinations.

For detailed metrics, please refer to the paper.

Citation

If you use this repository, please cite our paper:

Coming Soon

Acknowledgments

This work was supported by Rutgers University. We also thank the creators of the MIMIC-CXR dataset for their valuable contributions to medical AI research.