Synthesize Realistic 3D Medical Images at Scale to Ship Pre‑Trained Models

High‑quality 3D medical imaging data is the foundation of modern radiology AI, but access to it is often constrained by data scarcity, privacy restrictions, and the high cost of expert annotation. As a result, training reliable 3D medical imaging models is frequently bottlenecked by small, narrow, and hard‑to‑share datasets, limiting model robustness and generalization.

To help teams overcome these challenges, NVIDIA introduced Medical AI for Synthetic Imaging (MAISI) in 2024—a state‑of‑the‑art generative model that synthesizes high‑resolution 3D CT volumes with pixel‑level anatomical segmentation for privacy‑preserving data augmentation and research. 

Breaking the 3D medical imaging data bottleneck

NV-Generate-MR-Brain was trained on the newly released multimodal MR-RATE dataset from University of Zurich, Medipol University Hospital, Forithmus and NVIDIA. The MR-RATE dataset builds on the highly successful CT-RATE dataset and multimodal foundation models.

Open source by design

Why image generation is essential for medical AI

Medical image synthesis has rapidly become a core capability for medical AI development. Teams use synthetic data to augment limited training sets, translate between imaging modalities such as CT and MRI, simulate rare pathologies, and enable privacy‑preserving data sharing without exposing real patient information.

By generating realistic, anatomically consistent 3D volumes—often paired with segmentation labels—synthetic data helps models generalize better when labeled examples are scarce and enables consistent benchmarking across sites, scanners, and protocols.

As clinical imaging becomes increasingly personalized, heterogeneous, and multimodal, scalable and controllable generation frameworks are no longer optional—they are essential for building robust medical AI systems.

Limitations of existing medical image synthesis approaches

Over the years, medical image synthesis methods have largely fallen into three categories: direct regression models, generative adversarial network (GAN)‑based approaches, and, more recently, diffusion models that generate images through iterative denoising.

Among these, diffusion models have emerged as the state of the art, offering improved stability and the ability to model complex anatomical distributions. However, applying diffusion models in real clinical workflows remains challenging.

Together, these challenges—limited generalization, high computational cost, and weak condition alignment—make many existing approaches difficult to deploy at scale, motivating the need for faster and more controllable 3D synthesis frameworks.

Fast, Open Source 3D Medical Image Synthesis

NV‑Generate‑CTMR is an open‑source framework from NVIDIA designed to make high‑quality 3D medical image synthesis practical for everyday research and development. Rather than treating generative modeling as a narrow, task‑specific solution, it provides a reproducible, ready‑to‑use platform for creating realistic CT and MRI volumes across a wide range of clinical scenarios.

The framework is the first open‑source medical image generation framework to support flexible voxel sizes, variable volume dimensions, and whole‑body coverage within a single model (shown in Figure below). 

This flexibility allows researchers to synthesize data that matches real clinical protocols—from small, cropped regions to full‑resolution, large‑field‑of‑view scans—without retraining separate models for each setting. In this sense, NV‑Generate‑CTMR behaves as a foundation model for medical imaging, adaptable to many downstream tasks and anatomies rather than being limited to a single organ or configuration.

Efficient, sustainable AI development

Under the hood

NV‑Generate‑CTMR contains two model architectures:

• MAISI‑v1, based on Latent Denoising Diffusion Probabilistic Models (DDPM) for stochastic image generation with better diversity 
• MAISI‑v2, based on Latent Rectified Flow, for 33x acceleration in inference speed and image generation with better quality

Details were published in two technical papers: MAISI-v2: Accelerated 3D High-Resolution Medical Image Synthesis with Rectified Flow and Region-Specific Contrastive Loss at the AAAI Conference on Artificial Intelligence in 2026; and MAISI: Medical AI for Synthetic Imaging at the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) in 2025.

Fast inference at scale

The MAISI‑v2 model in NV‑Generate‑CTMR achieves state‑of‑the‑art image quality with much faster inference compared to prior medical image generation methods, while delivering inference speeds comparable to leading video generation models. Table 1, below, breaks down the family of NV-Generate models. 

Multi-contrast generation model for brain MRI

Brain MRI is one of the most widely used applications of magnetic resonance imaging. To support this domain, we released NV-Generate-MR-Brain, a generative model built on the MAISI‑v2 architecture and trained on the newly released MR‑RATE dataset. 

In addition, NV-Generate-MR-Brain provides a ControlNet module for generation of specified anatomical structures or cross-sequence synthesis, enabling users to predict one MRI sequence based on  another. 

Real‑world applications and research adoption

Image–mask pairs with tumors generated by NV‑Generate‑CTMR have been used as augmented training data for NV Segment. Beyond NVIDIA, the models have been used or fine‑tuned by external researchers across a wide range of applications, including:

• Zero-shot anomaly detection
• Lung CT cancer classification 
• Prostate MRI lesion classification
• MR-to-CT synthesis
• Text prompted CT and MRI tumor segmentation
• Brain diffusion MRI tractography
• Brain tumor MRI synthesis
• Text-to-CT generation
• Text-to-brain MRI generation

“Synthetic, anatomically realistic neuro MR data from NV-Generate, combined with automated segmentation from NV-Segment and clinical reasoning capabilities from NV-Reason, help us design and validate AI solutions more efficiently,” said Ioannis Panagiotelis, PhD, Business Leader MR at Philips. “This enables radiologists to benefit from smarter, more explainable workflows without compromising patient privacy.”

Try it yourself: Synthesize 3D medical images

The fastest way to experience NV-Generate-CTMR is to run it yourself. 

Online demo: No GPU required, you can explore an interactive browser demo hosted by NVIDIA.

Command Line Interface (CLI): The online demo showcases core capabilities, but the full experience is available from the GitHub repository, which includes pretrained weights and ready-to-use inference scripts for generating complete 3D CT or MRI volumes locally. After cloning the repo and installing dependencies, you can launch inference with a single command:

git clone https://github.com/NVIDIA-Medtech/NV-Generate-CTMR.git


cd NV-Generate-CTMR
export MONAI_DATA_DIRECTORY="./temp_work_dir"
network="rflow"
generate_version="rflow-ct"
python -m scripts.inference \
  -t ./configs/config_network_${network}.json \
  -i ./configs/config_infer.json \
  -e ./configs/environment_${generate_version}.json \
  --random-seed 0 \
  --version ${generate_version}

This command loads the pretrained rectified flow model and synthesizes full 3D medical volumes directly to your local workspace. You can then visualize the outputs, inspect paired segmentation masks, or plug the generated data into your own training and evaluation pipelines. An example result of the code block above is shown in Figure 4, below.

Example results

Whether you’re testing ideas, augmenting datasets, or benchmarking models, NV-Generate-CTMR makes it easy to start generating realistic medical images right away.

Getting started

Code: https://github.com/NVIDIA-Medtech/NV-Generate-CTMR 

Dataset (research access): https://huggingface.co/datasets/Forithmus/MR-RATE

Model checkpoints: 

• https://huggingface.co/nvidia/NV-Generate-CT
• https://huggingface.co/nvidia/NV-Generate-MR 
• https://huggingface.co/nvidia/NV-Generate-MR-Brain