Researchers from Zhejiang University, Shanghai Jiao Tong University, and the University of Illinois Urbana-Champaign have jointly introduced Hulu-Med, a general-purpose medical vision-language model capable of interpreting medical text, 2D scans, 3D volumes, and medical videos within a single unified system. The model supports a wide range of applications, including diagnosis, consultations, and surgical guidance.
As a fully open-source medical foundation model, Hulu-Med was trained entirely on public datasets and synthetic data generated by the team. They built a 16.7-million-sample multimodal corpus covering 12 organ systems and 14 types of medical imaging. Five dedicated data-generation pipelines were developed to produce high-quality descriptions, multilingual reasoning data, and video annotations, significantly enhancing the model’s generalization capability.
Technically, Hulu-Med introduces a unified visual architecture that processes text, images, 3D slices, and video frames within a single Transformer framework. Its medically aware token-compression method reduces visual tokens by an average of 55%, substantially lowering compute costs. Training the largest 32B model required roughly 40,000 A100 GPU hours.
Across 30 public medical benchmarks, Hulu-Med ranked first in 27 of them and outperformed closed-source systems—including GPT-4o—on 16 tasks. On HealthBench, a pure-text clinical dialogue benchmark, its performance is comparable to GPT-4.1. The model also delivers strong results in multilingual understanding, rare-disease diagnosis, and surgical video analysis.
Hulu-Med is now fully open-sourced on GitHub and HuggingFace. The team plans to further expand its open data resources and advance clinical validation to build the next generation of explainable and trustworthy medical AI systems.
source：QbitAI
Source link
Like this:
Like Loading…
Нашия източник е Българо-Китайска Търговско-промишлена палaта