UI Agent

小组介绍

UI Agent小组以人机共生为长远愿景,主攻智能体系统(Agentic System) 研究,融合大模型算法、信息可视化与人机交互技术,构建集自主感知、决策、执行与迭代于一体的智能协作体系。研究聚焦UI智能体的多模态界面理解、任务规划与交互优化,攻克复杂交互场景下智能体可靠性、泛化性与可解释性等关键难题,推动人机交互从工具使用向深度协同共生演进。团队围绕真实应用场景开展多项核心课题研究:人机协同数据分析、基于用户行为感知的主动式UI智能体、Mobile UI Agent 自动化评测平台、数据资产智能管理系统等,致力于实现智能体技术与交互设计的深度融合与落地应用。

相关发表
  • LightVA: Lightweight visual analytics with LLM agent-based task planning and execution.
    Yuheng Zhao, Junjie Wang, Linbin Xiang, Xiaowen Zhang, Zifei Guo, Cagatay Turkay, Yu Zhang, Siming Chen*.
    IEEE Transactions on Visualization and Computer Graphics, Accepted, 2024.
  • LEVA: Using large language models to enhance visual analytics.
    Yuheng Zhao, Yixing Zhang, Yu Zhang, Xinyi Zhao, Junjie Wang, Zekai Shao, Cagatay Turkay, Siming Chen*.
    IEEE Transactions on Visualization and Computer Graphics, Accepted, 2024.
  • Graph-Neural-Network-Based User Intent Understanding for Visual Analytics.
    Yue Wang, Yusheng Qi, Xiaolong Luke Zhang, Siming Chen*.
    In Proceedings of IEEE Pacific Visualization Conference (PacificVis 2024), pages 11-21, Tokyo, Japan, 2024.
  • TopicRefiner: Coherence-Guided Steerable LDA for Visual Topic Enhancement.
    Huan Yang, Jie Li, Siming Chen*.
    IEEE Trans. Vis. Comput. Graph. 30(8): 4542-4557, 2024.
  • OneLabeler: A Flexible System for Building Data Labeling Tools.
    Yu Zhang, Yun Wang, Haidong Zhang, Bin Zhu, Siming Chen, Dongmei Zhang.
    ACM CHI Conference on Human Factors in Computing Systems (ACM CHI'22), 93: 1-22, 2022.
    | Paper | pdf (8.4MB) | Video | mp4 (21.2MB) | Code | OneLabeler