OmniHuman is a large-scale, multi-scene audio-visual dataset designed for fine-grained human-centric video modeling. It covers diverse real-world shooting environments and social scenarios, with rich human-human and human-object interactions. Together with the dataset, we introduce OHBench to evaluate human-centric multimodal video generation beyond conventional global quality metrics.
We construct OmniHuman, a large-scale, multi-scene audio-visual dataset. Our dataset comprises 1 million videos totaling 1,800 hours, encompassing 80,000 distinct identities. It covers a wide range of authentic shooting environments and diverse social scenarios, featuring both human-human and human-object interactions. All videos are high quality and audio-video synchronized. Hover over a video to hear its audio.
We establish OHBench, a comprehensive benchmark aligned with human perception for human-centric multimodal video generation, enabling comprehensive diagnosis of model performance in real-world scenarios. The following section details the performance scores of different cases (all videos are generated by different models) across some evaluation metrics in OHBench.
We finetuned LTX-2, the current SOTA model, using 200k samples from the OmniHuman Dataset. Notably, there is zero ID overlap between our benchmark and the training dataset. Our results show comprehensive improvements across all metrics, particularly in audio quality and identity preservation.