HyDA: Hypernetworks for Test Time Domain Adaptation in Medical Imaging Analysis

Key Contributions

We introduce HyDA, a novel hypernetwork framework for test-time domain adaptation in medical imaging:

Hypernetwork Framework: HyDA is general purpose and applicable to any imaging modality (MRI, X-ray, etc.) and task type (classification, segmentation, etc.).
Domain-Aware Predictions: HyDA uses domain-specific embeddings to provide tailored predictions per sample.
0 Additional Data: HyDA can be applied in domain adaptation or test time adaptation settings, both of which do not require access to target domain data during training.

Abstract

Medical imaging datasets often vary due to differences in acquisition protocols, patient demographics, and imaging devices. These variations in data distribution, known as domain shift, present a significant challenge in adapting imaging analysis models for practical healthcare applications. Most current domain adaptation (DA) approaches aim either to align the distributions between the source and target domains or to learn an invariant feature space that generalizes well across all domains. However, both strategies require access to a sufficient number of examples, though not necessarily annotated, from the test domain during training. This limitation hinders the widespread deployment of models in clinical settings, where target domain data may only be accessible in real time.

In this work, we introduce HyDA, a novel hypernetwork framework that leverages domain-specific characteristics rather than suppressing them, enabling dynamic adaptation at inference time. Specifically, HyDA learns implicit domain representations and uses them to adjust model parameters on-the-fly, allowing effective interpolation to unseen domains. We validate HyDA on two clinically relevant applications—MRI-based brain age prediction and chest X-ray pathology classification—demonstrating its ability to generalize across tasks and imaging modalities.

Motivation

Most existing domain adaptation methods aim to create domain-agnostic features by suppressing domain-specific information. This approach discards valuable domain characteristics that could actually enhance model performance.

Our key insight is to leverage domain information as a prior rather than suppressing it. Instead of making features domain-invariant, we learn to utilize domain characteristics to tailor model predictions for each specific domain.

Domain Interpolation

The above t-SNE plot shows how HyDA learns meaningful domain representations. Each domain is spanned by a cluster of samples (e.g. scans from the same scanner, protocol, etc.). When we encounter samples from a new, unseen domain (red points), they naturally fall within the learned domain space as interpolations of existing domains. This representation allows HyDA to generate domain-specific parameters for each sample, enabling dynamic adaptation at test time.

Method

HyDA consists of three main components: a primary network P for medical imaging tasks, a hypernetwork h, and a domain classifier D. The domain classifier learns implicit domain representations through its encoder component D_enc.

These domain features are fed to the hypernetwork h, which generates external parameters (weights and biases) that are transferred to specific layers in the primary network P. During inference, when encountering a new domain, the domain encoder extracts domain-specific features, and the hypernetwork dynamically generates appropriate parameters for that domain.

This allows the model to interpolate between known domains and adapt to unseen target domains without requiring target domain data during training. The framework is trained using task-specific losses combined with a multi-similarity loss that encourages similar domain representations for images from the same domain.

Experiments and Results

Chest X-ray Pathology Classification

We evaluated HyDA on chest X-ray pathology classification using multiple datasets including CheXpert, NIH, and VinDr-CXR. The results demonstrate that HyDA consistently outperforms baseline methods and other domain adaptation techniques.

Chest X-ray Classification Results (AUC ↑)

Target Domain	Method	Atel.	Cardio.	Cons.	Eff.	Pneu.	Avg. (std)
-	Baseline	0.85	0.95	0.86	0.94	0.87	0.89 (0.04)
	MDAN	0.86	0.96	0.86	0.94	0.88	0.90 (0.04)
	HyDA	0.87	0.97	0.86	0.94	0.89	0.91 (0.04)
NIH	Baseline	0.70	0.81	0.76	0.86	0.77	0.78 (0.06)
	MDAN	0.67	0.89	0.76	0.86	0.77	0.79 (0.08)
	TENT	0.61	0.70	0.64	0.81	0.67	0.69 (0.07)
	HyDA	0.68	0.89	0.75	0.88	0.79	0.80 (0.08)
CheXpert	Baseline	0.81	0.86	0.73	0.87	0.74	0.80 (0.06)
	MDAN	0.77	0.76	0.71	0.84	0.72	0.76 (0.05)
	TENT	0.76	0.86	0.77	0.89	0.76	0.81 (0.06)
	HyDA	0.82	0.85	0.82	0.89	0.74	0.82 (0.05)
VinDr	Baseline	0.60	0.76	0.85	0.88	0.91	0.80 (0.11)
	MDAN	0.68	0.82	0.88	0.87	0.89	0.83 (0.08)
	TENT	0.51	0.72	0.80	0.74	0.86	0.73 (0.12)
	HyDA	0.66	0.87	0.93	0.89	0.92	0.85 (0.10)

Abbreviations: Atel: Atelectasis, Cardio: Cardiomegaly, Cons: Consolidation, Eff: Effusion, Pneu: Pneumothorax
Note: Each group compares models on the same target domain. Best results are in bold.

MRI Brain Age Prediction

For brain age prediction, we used multiple MRI datasets including ADNI, AIBL, OASIS, and others. The regression task results demonstrate HyDA's effectiveness across different evaluation protocols.

Brain Age Prediction Results (MAE ↓)

Fully Supervised (Validation MAE)

Model	CNP	NKI	ixi	OASIS	ABIDE	ADNI	AIBL	PPMI	Camcan	SLIM	Avg. (std)
Baseline	3.11	3.01	3.54	3.29	2.09	2.80	2.74	4.23	3.35	0.47	2.86 (0.96)
HyDA	2.39	2.92	3.22	3.29	1.74	3.04	2.94	3.94	3.21	0.37	2.71 (0.95)

Leave-One-Out (Test MAE)

Model	CNP	NKI	ixi	OASIS	ABIDE	ADNI	AIBL	PPMI	Camcan	SLIM	Avg. (std)
Baseline	3.36	3.90	4.41	5.40	3.25	4.31	3.56	4.15	3.50	1.44	3.73 (0.97)
HyDA	2.86	3.44	4.14	5.20	3.16	4.48	3.45	4.24	3.35	1.34	3.57 (1.00)

Note: Best scores are in bold.

Ablation Study

We conducted ablation studies to evaluate the contribution of each loss term in HyDA and to verify HyDA's robustness to the external layer configuration.

Ablation #1: Loss Terms

Ablation #2: External Layer Configuration

BibTeX

@article{serebro2025hyda, title={HyDA: Hypernetworks for Test Time Domain Adaptation in Medical Imaging Analysis}, author={Serebro, Doron and Riklin-Raviv, Tammy}, journal={International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI)}, year={2025}, publisher={Springer} }