Shiyu Huang   黄世宇 Researcher, XPENG Haidian District, Beijing, China, 100084. Email: huangsy1314@163.com [OpenRL]     [知乎]     [Google Scholar]     [GitHub]     [Linkedin]     [CV] Visitors: 11285

Biography

I am a researcher at XPeng Inc. Before that, I worked as a researcher in Zhipu AI and 4Paradigm Inc., and I am the leader of the OpenRL Lab. I received my B.E. and Ph.D. degrees (co-advised by Prof. Jun Zhu and Prof. Ting Chen) from the Department of Computer Science and Technology, Tsinghua University in July, 2017 and June, 2022. My researches focus on deep reinforcement learning, multi-agent reinforcement learning, distributed reinforcement learning, RL for robotics, LLM as agent, artificial general intelligence (AGI) and generative artificial intelligence (GAI). I have also spent time working at RealAI Inc. , Huawei Noah's Ark Lab, Tencent AI Lab, Carnegie Mellon University and Sensetime Inc. . And I am also the founder of the OpenRL Lab() and TARTRL group.

We are looking for self-motivated interns and full-timers who have a strong background in mathematics/computer science and are eager to get involved in cutting-edge, fundamental AI research. Please feel free to drop me an email if you are interested in collaborating with me.

Publications && Preprints

(* equal contribution)

• Can LLM Watermarks Robustly Prevent Unauthorized Knowledge Distillation?
  Leyi Pan, Aiwei Liu, Shiyu Huang, Yijian Lu, Xuming Hu, Lijie Wen, Irwin King, Philip S. Yu
  The 63rd Annual Meeting of the Association for Computational Linguistics (ACL), 2025
  [PDF] [Code] [BibTeX]

  @article{pan2025can,
    title={Can LLM Watermarks Robustly Prevent Unauthorized Knowledge Distillation?},
    author={Pan, Leyi and Liu, Aiwei and Huang, Shiyu and Lu, Yijian and Hu, Xuming and Wen, Lijie and King, Irwin and Yu, Philip S},
    journal={arXiv preprint arXiv:2502.11598},
    year={2025}
  }

• WaterSeeker: Pioneering Efficient Detection of Watermarked Segments in Large Documents
  Leyi Pan, Aiwei Liu, Yijian Lu, Zitian Gao, Yichen Di, Shiyu Huang, Lijie Wen, Irwin King, Philip S. Yu
  The AAAI 2025 PDLM Workshop on Preventing and Detecting LLM Misinformation, 2025
  [PDF] [Code] [BibTeX]

  @misc{pan2025waterseekerpioneeringefficientdetection,
        title={WaterSeeker: Pioneering Efficient Detection of Watermarked Segments in Large Documents},
        author={Leyi Pan and Aiwei Liu and Yijian Lu and Zitian Gao and Yichen Di and Shiyu Huang and Lijie Wen and Irwin King and Philip S. Yu},
        year={2025},
        eprint={2409.05112},
        archivePrefix={arXiv},
        primaryClass={cs.CL},
        url={https://arxiv.org/abs/2409.05112},
  }

• A Survey on Self-play Methods in Reinforcement Learning
  Ruize Zhang, Zelai Xu, Chengdong Ma, Chao Yu, Wei-Wei Tu, Shiyu Huang, Deheng Ye, Wenbo Ding, Yaodong Yang, Yu Wang
  arXiv:2408.01072, 2024
  [PDF] [BibTeX]

  @misc{zhang2024surveyselfplaymethodsreinforcement,
            title={A Survey on Self-play Methods in Reinforcement Learning},
            author={Ruize Zhang and Zelai Xu and Chengdong Ma and Chao Yu and Wei-Wei Tu and Shiyu Huang and Deheng Ye and Wenbo Ding and Yaodong Yang and Yu Wang},
            year={2024},
            eprint={2408.01072},
            archivePrefix={arXiv},
            primaryClass={cs.AI},
            url={https://arxiv.org/abs/2408.01072},
      }

• Soft-QMIX: Integrating Maximum Entropy For Monotonic Value Function Factorization
  Wentse Chen, Shiyu Huang, Jeff Schneider
  arXiv:2406.13930, 2024
  [PDF] [Code] [BibTeX]

  @misc{chen2024softqmixintegratingmaximumentropy,
        title={Soft-QMIX: Integrating Maximum Entropy For Monotonic Value Function Factorization},
        author={Wentse Chen and Shiyu Huang and Jeff Schneider},
        year={2024},
        eprint={2406.13930},
        archivePrefix={arXiv},
        primaryClass={cs.LG}
        url={https://arxiv.org/abs/2406.13930},
  }

• MQE: Unleashing the Power of Interaction with Multi-agent Quadruped Environment
  Ziyan Xiong, Bo Chen, Shiyu Huang, Wei-Wei Tu, Zhaofeng He, Yang Gao
  The 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2024
  
  [PDF] [Code] [BibTeX]

  @misc{xiong2024mqeunleashingpowerinteraction,
        title={MQE: Unleashing the Power of Interaction with Multi-agent Quadruped Environment},
        author={Ziyan Xiong and Bo Chen and Shiyu Huang and Wei-Wei Tu and Zhaofeng He and Yang Gao},
        year={2024},
        eprint={2403.16015},
        archivePrefix={arXiv},
        primaryClass={cs.RO},
        url={https://arxiv.org/abs/2403.16015},
  }

• LLMArena: Assessing Capabilities of Large Language Models in Dynamic Multi-Agent Environments
  Junzhe Chen, Xuming Hu, Shuodi Liu, Shiyu Huang, Wei-Wei Tu, Zhaofeng He, Lijie Wen
  The 62nd Annual Meeting of the Association for Computational Linguistics(ACL), 2024
  
• AutoSAT: Automatically Optimize SAT Solvers via Large Language Models
  Yiwen Sun, Xianyin Zhang, Shiyu Huang, Shaowei Cai, Bing-Zhen Zhang, Ke Wei
  arXiv:2402.10705, 2024
  
• OpenRL: A Unified Reinforcement Learning Framework
  Shiyu Huang, Wentse Chen, Yiwen Sun, Fuqing Bie, Wei-Wei Tu
  arXiv:2312.16189, 2023
  
  [PDF] [Code] [BibTeX]

  @article{huang2023openrl,
    title={OpenRL: A Unified Reinforcement Learning Framework},
    author={Huang, Shiyu and Chen, Wentse and Sun, Yiwen and Bie, Fuqing and Tu, Wei-Wei},
    journal={arXiv preprint arXiv:2312.16189},
    year={2023}
  }

• DGPO: Discovering Multiple Strategies with Diversity-Guided Policy Optimization
  Wenze Chen, Shiyu Huang, Yuan Chiang, Tim Pearce, Wei-Wei Tu, Ting Chen, Jun Zhu
  Thirty-Eighth AAAI Conference on Artificial Intelligence(AAAI), Vancouver, Canada, 2024
  
• SwiftSage: A Generative Agent with Fast and Slow Thinking for Complex Interactive Tasks
  Bill Yuchen Lin, Yicheng Fu, Karina Yang, Prithviraj Ammanabrolu, Faeze Brahman, Shiyu Huang, Chandra Bhagavatula, Yejin Choi, Xiang Ren
  Thirty-seventh Conference on Neural Information Processing Systems(NeurIPS)(Spotlight), 2023
  
• Robustness and Generalizability of Deepfake Detection: A Study with Diffusion Models
  Haixu Song, Shiyu Huang, Yinpeng Dong, Wei-Wei Tu
  arXiv:2309.02218, 2023
  
• Diverse Policies Converge in Reward-free Markov Decision Processes
  Fanqi Lin, Shiyu Huang, Wei-Wei Tu
  The 20th Pacific Rim International Conference on Artificial Intelligence(PRICAI), Jakarta, Indonesia, 2023
  
• Uncertainty quantification via a memristor Bayesian deep neural network for risk-sensitive reinforcement learning
  Yudeng Lin, Qingtian Zhang, Bin Gao, Jianshi Tang, Peng Yao, Chongxuan Li, Shiyu Huang, Zhengwu Liu, Ying Zhou, Yuyi Liu, Wenqiang Zhang, Jun Zhu and He Qian
  Nature Machine Intelligence, 2023
  
• TiZero: Mastering Multi-Agent Football with Curriculum Learning and Self-Play
  Fanqi Lin*, Shiyu Huang*, Tim Pearce, Wenze Chen and Wei-Wei Tu
  The 22nd International Conference on Autonomous Agents and Multiagent Systems(AAMAS), London, UK, 2023
  
• Learning Graph-Enhanced Commander-Executor for Multi-Agent Navigation
  Xinyi Yang, Shiyu Huang, Yiwen Sun, Yuxiang Yang, Chao Yu, Wei-Wei Tu, Huazhong Yang and Yu Wang
  The 22nd International Conference on Autonomous Agents and Multiagent Systems(AAMAS), London, UK, 2023
  
• DGPO: Discovering Multiple Strategies with Diversity-Guided Policy Optimization
  Wenze Chen, Shiyu Huang, Yuan Chiang, Ting Chen, Jun Zhu
  The 22nd International Conference on Autonomous Agents and Multiagent Systems(AAMAS) Extended Abstract, London, UK, 2023
  
• VMAPD: Generate Diverse Solutions for Multi-Agent Games with Recurrent Trajectory Discriminators
  Shiyu Huang*, Chao Yu*, Bin Wang, Dong Li, Yu Wang, Ting Chen and Jun Zhu
  IEEE Conference on Games(COG)(Best Paper Nomination), Beijing, China, 2022
  
• Ranking Cost: Building An Efficient and Scalable Circuit Routing Planner with Evolution-Based Optimization
  Shiyu Huang, Bin Wang, Dong Li, Jianye Hao, Ting Chen and Jun Zhu
  IJCAI-ECAI 2022 Workshop: The 2nd International Workshop on Heuristic Search in Industry, Vienna, Austria, 2022
  
• TiKick: Towards Playing Multi-agent Football Full Games from Single-agent Demonstrations
  Shiyu Huang*, Wenze Chen*, Longfei Zhang, Shizhen Xu, Ziyang Li, Fengming Zhu, Deheng Ye, Ting Chen and Jun Zhu
  NeurIPS-21 Workshop: 2nd Offline Reinforcement Learning Workshop
  
• Deep Reinforcement Learning with Credit Assignment for Combinatorial Optimization
  Dong Yan, Jiayi Weng, Shiyu Huang, Chongxuan Li, Yichi Zhou, Hang Su, Jun Zhu
  Pattern Recognition, 2021
  
• Off-Policy Training for Truncated TD(λ) Boosted Soft Actor-Critic
  Shiyu Huang, Bin Wang, Hang Su, Dong Li, Jianye Hao, Jun Zhu, Ting Chen
  The 18th Pacific Rim International Conference on Artificial Intelligence(PRICAI), Hanoi, Vietnam, 2021
  
• SVQN: Sequential Variational Soft Q-Learning Networks
  Shiyu Huang, Hang Su, Jun Zhu, and Ting Chen
  Eighth International Conference on Learning Representations (ICLR), Millennium Hall, Addis Ababa ETHIOPIA, 2020
  
• Combo-Action: Training Agent For FPS Game with Auxiliary Tasks (Spotlight)
  Shiyu Huang, Hang Su, Jun Zhu, and Ting Chen
  The Thirty-Third AAAI Conference on Artificial Intelligence (AAAI), Honolulu, Hawaii, USA, 2019
  
• Expecting the Unexpected: Training Detectors for Unusual Pedestrians with Adversarial Imposters
  Shiyu Huang, and Deva Ramanan
  IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, Hawaii, USA, 2017
  
• Learning to Assign Credit in Reinforcement Learning by Incorporating Abstract Relations
  Dong Yan, Shiyu Huang, Hang Su, and Jun Zhu
  AAAI-19 Workshop on Reinforcement Learning in Games
  
• Model-based Credit Assignment for Model-free Deep Reinforcement Learning
  Dong Yan, Jiayi Weng, Shiyu Huang, Chongxuan Li, Yichi Zhou, Hang Su, Jun Zhu
  

Talks

• RLHF @ Zhiyuan Community, 2023.8 [slide]
• OpenRL @ 5th BAAI Conference, 2023.6 [video]

Projects

• CogVideoX
• CogVLM2-Video
• OpenRL: Unified reinforcement learning framework.
• TiZero: RL agents for Google Research Football.
• ChatAgent: A Python-based agent framework for large language models.
• TLaunch: Launch programs on multiple hosts.
• OpenPlugin: Toolkit to manage the plugins of the large language model.

Patents

• Generation method, device, medium and computing device of diversity strategy. Shiyu Huang, Tian Tian. 2021116684627
• Method or equipment for controlling agent. Jun Zhu, Shiyu Huang, Hang Su. ZL201910078546.1

Honors & Awards

• Tung OOCL Scholarship, Tsinghua University, 2019
• Tsinghua Excellent Graduates, Tsinghua University, 2017
• Academic Excellence Award, Tsinghua University, 2014-2016

Competitions

• 2022.8
  IEEE CoG 2022 Football AI Competition:
  Track2, 3rd place
  
• 2018.8
  ViZDoom 2018 AI Competition:
  Track1, 1st place
  Track2, 2nd place
  
• 2017
  ViZDoom 2017 AI Competition:
  Track2, 2nd place
  

Services

Organizer for:
NeurIPS 2023 Workshop on New in ML

Reviewer for:
NeurIPS 2025, ICCV 2025, ICML 2025, CVPR 2025, ICLR 2025, AAAI 2025, NeurIPS 2024, ICML 2024, ICLR 2024, AAAI 2024, NeurIPS 2023, AISTATS 2023, AAAI 2023, ICLR 2023, NeurIPS 2022, ICML 2022, AISTATS 2022, AAAI 2022, ICLR 2022, NeurIPS 2021, ICML 2021, AAAI 2021, NeurIPS 2020

Teaching

2020 Spring, TA in Big Data and Machine Intelligence, instructed by Zhen Chen
2019 Fall, TA in Big Data and Machine Intelligence, instructed by Zhen Chen
2019 Spring, TA in Machine Learning, instructed by Prof. Jun Zhu