Shiyu Huang   黄世宇 Researcher, 4Paradigm Building 1, No. 66 Qinghe Middle Street, Haidian District Beijing, China, 100084. Email: huangshiyu@4paradigm.com; huangsy1314@163.com [OpenRL]         [知乎]         [Google Scholar]         [TARTRL]         [GitHub]         [Linkedin]

Biography

I am a research scientist in 4Paradigm Inc. and the leader of OpenRL Lab. Before that, I received my B.E. and Ph. D. degrees (co-advised by Prof. Jun Zhu and Prof. Ting Chen) from the Department of Computer Science and Technology, Tsinghua University in July, 2017 and June, 2022. My researches focus on deep reinforcement learning, multi-agent reinforcement learning, distributed reinforcement learning, RL for robotics, LLM as agent, artificial general intelligence (AGI) and generative artificial intelligence (GAI). I have also spent time working at RealAI Inc. , Huawei Noah's Ark Lab, Tencent AI Lab, Carnegie Mellon University and Sensetime Inc. . And I am also the founder of the OpenRL Lab and TARTRL group.

We are looking for self-motivated interns and full-timers who have a strong background in mathematics/computer science and are eager to get involved in cutting-edge, fundamental AI research. Please feel free to drop me an email if you are interested in collaborating with me.

Publications && Preprints

(* equal contribution)

• SwiftSage: A Generative Agent with Fast and Slow Thinking for Complex Interactive Tasks
  Bill Yuchen Lin, Yicheng Fu, Karina Yang, Prithviraj Ammanabrolu, Faeze Brahman, Shiyu Huang, Chandra Bhagavatula, Yejin Choi, Xiang Ren
  Thirty-seventh Conference on Neural Information Processing Systems(NeurIPS)(Spotlight), 2023
  
• Robustness and Generalizability of Deepfake Detection: A Study with Diffusion Models
  Haixu Song, Shiyu Huang, Yinpeng Dong, Wei-Wei Tu
  arXiv:2309.02218, 2023
  
• Diverse Policies Converge in Reward-free Markov Decision Processes
  Fanqi Lin, Shiyu Huang, Wei-Wei Tu
  The 20th Pacific Rim International Conference on Artificial Intelligence(PRICAI), Jakarta, Indonesia, 2023
  
• Uncertainty quantification via a memristor Bayesian deep neural network for risk-sensitive reinforcement learning
  Yudeng Lin, Qingtian Zhang, Bin Gao, Jianshi Tang, Peng Yao, Chongxuan Li, Shiyu Huang, Zhengwu Liu, Ying Zhou, Yuyi Liu, Wenqiang Zhang, Jun Zhu and He Qian
  Nature Machine Intelligence, 2023
  
• TiZero: Mastering Multi-Agent Football with Curriculum Learning and Self-Play
  Fanqi Lin*, Shiyu Huang*, Tim Pearce, Wenze Chen and Wei-Wei Tu
  The 22nd International Conference on Autonomous Agents and Multiagent Systems(AAMAS), London, UK, 2023
  
• Learning Graph-Enhanced Commander-Executor for Multi-Agent Navigation
  Xinyi Yang, Shiyu Huang, Yiwen Sun, Yuxiang Yang, Chao Yu, Wei-Wei Tu, Huazhong Yang and Yu Wang
  The 22nd International Conference on Autonomous Agents and Multiagent Systems(AAMAS), London, UK, 2023
  
• DGPO: Discovering Multiple Strategies with Diversity-Guided Policy Optimization
  Wenze Chen, Shiyu Huang, Yuan Chiang, Ting Chen, Jun Zhu
  The 22nd International Conference on Autonomous Agents and Multiagent Systems(AAMAS) Extended Abstract, London, UK, 2023
  
• VMAPD: Generate Diverse Solutions for Multi-Agent Games with Recurrent Trajectory Discriminators
  Shiyu Huang*, Chao Yu*, Bin Wang, Dong Li, Yu Wang, Ting Chen and Jun Zhu
  IEEE Conference on Games(COG)(Best Paper Nomination), Beijing, China, 2022
  
• Ranking Cost: Building An Efficient and Scalable Circuit Routing Planner with Evolution-Based Optimization
  Shiyu Huang, Bin Wang, Dong Li, Jianye Hao, Ting Chen and Jun Zhu
  IJCAI-ECAI 2022 Workshop: The 2nd International Workshop on Heuristic Search in Industry, Vienna, Austria, 2022
  
• TiKick: Towards Playing Multi-agent Football Full Games from Single-agent Demonstrations
  Shiyu Huang*, Wenze Chen*, Longfei Zhang, Shizhen Xu, Ziyang Li, Fengming Zhu, Deheng Ye, Ting Chen and Jun Zhu
  NeurIPS-21 Workshop: 2nd Offline Reinforcement Learning Workshop
  
• Deep Reinforcement Learning with Credit Assignment for Combinatorial Optimization
  Dong Yan, Jiayi Weng, Shiyu Huang, Chongxuan Li, Yichi Zhou, Hang Su, Jun Zhu
  Pattern Recognition, 2021
  
• Off-Policy Training for Truncated TD(λ) Boosted Soft Actor-Critic
  Shiyu Huang, Bin Wang, Hang Su, Dong Li, Jianye Hao, Jun Zhu, Ting Chen
  The 18th Pacific Rim International Conference on Artificial Intelligence(PRICAI), Hanoi, Vietnam, 2021
  
• SVQN: Sequential Variational Soft Q-Learning Networks
  Shiyu Huang, Hang Su, Jun Zhu, and Ting Chen
  Eighth International Conference on Learning Representations (ICLR), Millennium Hall, Addis Ababa ETHIOPIA, 2020
  
• Combo-Action: Training Agent For FPS Game with Auxiliary Tasks (Spotlight)
  Shiyu Huang, Hang Su, Jun Zhu, and Ting Chen
  The Thirty-Third AAAI Conference on Artificial Intelligence (AAAI), Honolulu, Hawaii, USA, 2019
  
• Expecting the Unexpected: Training Detectors for Unusual Pedestrians with Adversarial Imposters
  Shiyu Huang, and Deva Ramanan
  IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, Hawaii, USA, 2017
  
• Learning to Assign Credit in Reinforcement Learning by Incorporating Abstract Relations
  Dong Yan, Shiyu Huang, Hang Su, and Jun Zhu
  AAAI-19 Workshop on Reinforcement Learning in Games
  
• Model-based Credit Assignment for Model-free Deep Reinforcement Learning
  Dong Yan, Jiayi Weng, Shiyu Huang, Chongxuan Li, Yichi Zhou, Hang Su, Jun Zhu
  

Talks

• RLHF @ Zhiyuan Community, 2023.8 [slide]
• OpenRL @ 5th BAAI Conference, 2023.6 [video]

Projects

• OpenRL: Unified reinforcement learning framework.
• TiZero: RL agents for Google Research Football.
• TLaunch: Launch programs on multiple hosts.
• OpenPlugin: Toolkit to manage the plugins of the large language model.

Patents

• Generation method, device, medium and computing device of diversity strategy. Shiyu Huang, Tian Tian. 2021116684627
• Method or equipment for controlling agent. Jun Zhu, Shiyu Huang, Hang Su. ZL201910078546.1

Honors & Awards

• Tung OOCL Scholarship, Tsinghua University, 2019
• Tsinghua Excellent Graduates, Tsinghua University, 2017
• Academic Excellence Award, Tsinghua University, 2014-2016

Competitions

• 2022.8
  IEEE CoG 2022 Football AI Competition:
  Track2, 3rd place
  
• 2018.8
  ViZDoom 2018 AI Competition:
  Track1, 1st place
  Track2, 2nd place
  
• 2017
  ViZDoom 2017 AI Competition:
  Track2, 2nd place
  

Services

Organizer for:
NeurIPS 2023 Workshop on New in ML

Reviewer for:
ICLR 2024, AAAI 2024, NeurIPS 2023, AISTATS 2023, AAAI 2023, ICLR 2023, NeurIPS 2022, ICML 2022, AISTATS 2022, AAAI 2022, ICLR 2022, NeurIPS 2021, ICML 2021, AAAI 2021, NeurIPS 2020

Teaching

2020 Spring, TA in Big Data and Machine Intelligence, instructed by Zhen Chen
2019 Fall, TA in Big Data and Machine Intelligence, instructed by Zhen Chen
2019 Spring, TA in Machine Learning, instructed by Prof. Jun Zhu