Being-H0: Vision-Language-Action Pretraining from Large-Scale Human Videos
Hao Luo · 罗昊
I am a fourth-year Ph.D. student at School of Computer Science, Peking University, advised by Prof. Zongqing Lu, and expect to graduate in 2027. Before starting my Ph.D., I received my B.E. in Computer Science from Peking University.
My current research focuses on Embodied AI, especially Embodied Foundation Models. Specifically, I am working on generalizable VLAs scalable with human videos and action-oriented latent representations learned with world models. I have previously worked on VLMs, particularly for egocentric video understanding. Feel free to reach out if you are interested in discussions or collaborations.
Personally, I am particularly inspired by JEPA, and believe that transition-aware and interaction-centric representations can boost progress toward truly generalizable policies with broad in-context reasoning.
Dexterous Foundation Models Scalable with Human Videos
The Being-H series explores scaling embodied foundation models with human videos.
- Being-H0 validates that thousand-hour human videos can pretrain dexterous VLAs with human priors transferable in downstream tasks.
- Being-H0.5 further scales this recipe to 35,000+ hours through unified training on human, robot, and visual-text data, enabling cross-embodiment policies.
Selected Publications
All PapersLearning from Videos — VLA
Joint-Aligned Latent Action: Towards Scalable VLA Pretraining in the Wild
Developing JEPT, JALA scales VLA pretraining by aligning hidden states with inverse dynamics to learn unified latent actions from both labeled and unannotated human videos.
Spatial-Aware VLA Pretraining through Visual-Physical Alignment from Human Videos
VIPA-VLA enables spatially grounded VLA pretraining from human videos through visual-physical alignment, addressing the gap between 2D visual perception and 3D physical action.
Learning from Videos — Visual Policy
Learning Video-Conditioned Policy on Unlabelled Data with Joint Embedding Predictive Transformer
Inspired by JEPA, JEPT learns predictive visual representations of task-relevant transitions through joint embedding prediction.
To our knowledge, it is the earliest work to bring JEPA-style learning into real policy learning.
This enables video-conditioned policy learning with unlabeled videos, reducing data cost and improving one-shot generalization.
Pre-trained Visual Dynamics Representations for Efficient Policy Learning
PVDR learns abstract visual dynamics priors from unlabeled videos and adapts them for efficient policy learning in domain-gapped tasks.
Reinforcement Learning Friendly Vision-Language Model for Minecraft
CLIP4MC learns a more RL-friendly vision-language model from large-scale internet videos, providing better reward signals for open-ended embodied tasks.
VLM
OpenMMEgo: Enhancing Egocentric Understanding for LMMs with Open Weights and Data
We enhance large multimodal models for egocentric video understanding with large-scale first-person QA data and training recipes tailored to first-person dynamics.
Unified Multimodal Understanding via Byte-Pair Visual Encoding
Building upon the visual BPE tokenizer proposed in the previous work, we further designed a complete training framework and our Being-VL-0.5 model.
VideoOrion: Tokenizing Object Dynamics in Videos
VideoOrion encodes videos with a two-branch design, using object tokens from a detect-segment-track pipeline to capture object dynamics alongside scene context.
Experience
Apr 2025 - Present
BeingBeyond Technology
Research Intern, Being Exploring-Large Models
Developing the Being-H series of embodied foundation models pretrained from human videos.
Exploring scalable VLA and latent world-action models for embodied intelligence.
Jan 2023 - Mar 2025
Beijing Academy of Artificial Intelligence (BAAI)
Research Intern, Multimodal Interaction Group
Conducted researches on video-based representation learning for policy learning and VLMs.
Honors and Services
Awards
Lingjun Pilot Scholarship, Merit Student, Academic Excellence Award, Social Work Award, all at PKU
Reviewer
ICML, NeurIPS (before 2026), ICLR, CVPR, ICCV, ECCV, BMVC
Teaching Assistant
Algorithms, PKU, Spring 2021, 2022
Deep Reinforcement Learning, PKU, Spring 2024