|
Jinyi (James) Hu
I am a third-year Ph.D. student at Tsinghua, THUNLP lab, advised by Maosong Sun. My main research interest lies in foundation models for multimodal learning and embodied AI. My long-term research goal is to build a versatile agent to interact with visual world.
During my undergraduate, I interned at Mila Québec, advised by Jian Tang, and Alibaba Damo, advised by Boxing Chen.
I am open to any possible collaboration or discussion, feel free to contact me via email.
Email /
CV /
Google Scholar /
Github
|
|
|
Education
B.E. Department of Computer Science and Technology, Tsinghua University, 2017-2021
Ph.D. Department of Computer Science and Technology, Tsinghua University, 2021-2026 (expected)
|
|
News
[2024.1] VisCPM is accepted to ICLR 2024 as a spotlight (5%).
[2023.6] VisCPM is open-source!
[2022.10] Two papers are accepted to EMNLP 2022.
[2022.4] Della is accepted to NAACL 2022.
[2020.9] One paper is accepted to NeurIPS 2020.
|
|
Publication
(*indicates equal contribution)
|
|
LEGENT: Open Platform for Embodied Agents
Zhili Hu, Zhitong Wang,
Jinyi Hu,
Shengding Hu,
An Liu, Yuge Tu, Pengkai Li, Lei Shi,
Zhiyuan Liu,
Maosong Sun
[paper]
[code]
[doc]
|
|
Large Multilingual Models Pivot Zero-Shot Multimodal Learning across Languages
Jinyi Hu,
Yuan Yao,
Chongyi Wang, Shan Wang, Yinxu Pan, Qianyu Chen, Tianyu Yu, Hanghao Wu, Yue Zhao, Haoye Zhang,
Xu Han,
Yankai Lin,
Jiao Xue,
Dahai Li,
Zhiyuan Liu,
Maosong Sun
ICLR 2024 (spotlight, top 5% )
[paper]
[code (VisCPM)]
[code (MiniCPM-V)]
|
|
RLHF-V: Towards Trustworthy MLLMs via Behavior Alignment from Fine-grained Correctional Human Feedback
Tianyu Yu,
Yuan Yao,
Haoye Zhang, Taiwen He, Yifeng Han,
Ganqu Cui,
Jinyi Hu,
Zhiyuan Liu,
Hai-Tao Zheng,
Maosong Sun
Tat-Seng Chua
CVPR 2024
[paper]
[project]
|
|
Reformulating Vision-Language Foundation Models and Datasets Towards Universal Multimodal Assistants
Tianyu Yu*,
Jinyi Hu*,
Yuan Yao,
Haoye Zhang, Yue Zhao, Chongyi Wang, Shan Wang, Yinxv Pan, Jiao Xue, Dahai Li,
Zhiyuan Liu,
Hai-Tao Zheng,
Maosong Sun
arxiv, 2023
[paper]
[code (Muffin)]
[dataset (UniMM-Chat)]
|
|
Efficient Cross-Lingual Transfer for Chinese Stable Diffusion with Images as Pivots
Jinyi Hu,
Xu Han,
Xiaoyuan Yi,
Yutong Chen,
Wenhao Li,
Zhiyuan Liu,
Maosong Sun
arxiv, 2023
[paper]
|
|
Text Reasoning and Generation
|
|
Fuse It More Deeply! A Variational Transformer with Layer-Wise Latent Variable Inference for Text Generation
Jinyi Hu,
Xiaoyuan Yi,
Wenhao Li,
Maosong Sun,
Xing Xie
NAACL 2022 (oral)
[paper]
[code (DELLA)]
|
|
Recurrence Boosts Diversity! Revisiting Recurrent Latent Variable in Transformer-Based Variational AutoEncoder for Diverse Text Generation
Jinyi Hu,
Xiaoyuan Yi,
Wenhao Li,
Maosong Sun,
Xing Xie
Findings of EMNLP 2022
[paper]
|
|
Towards interpretable natural language understanding with explanations as latent variables
Wangchunshu Zhou*,
Jinyi Hu*,
Hanlin Zhang*,
Xiaodan Liang,
Maosong Sun,
Chenyan Xiong,
Jian Tang,
NeurIPS 2020
[paper]
[code]
|
|
Evade the Trap of Mediocrity: Promoting Diversity and Novelty in Text Generation via Concentrating Attention
Wenhao Li,
Xiaoyuan Yi,
Jinyi Hu,
Maosong Sun,
Xing Xie
EMNLP 2022
[paper]
|
|
Aspect-level sentiment-controllable review generation with mutual learning framework
Huimin Chen,
Yankai Lin,
Fanchao Qi,
Jinyi Hu,
Peng Li,
Jie Zhou,
Maosong Sun
AAAI 2021
[paper]
|
|
Service
Reviewer: NeurIPS, CVPR, ACL, EMNLP, NAACL
|
Honors & Awards
- Comprehensive Scholarship, 2023
- Longhu Scholarship, 2023
- Comprehensive Scholarship, 2022
- Merit Student in Beijing, 2020
- Tang Lixin Scholarship, 2020
- Evergrande Scholarship, 2018
|
|
Miscellaneous
I usually go to the gym during the spare time. I also love NBA, Snooker and Tennis, a big fan of LeBron James, Ronnie O'Sullivan and Roger Federer.
|
|