会员体验
专利管家(专利管理)
工作空间(专利管理)
风险监控(情报监控)
数据分析(专利分析)
侵权分析(诉讼无效)
联系我们
交流群
官方交流:
QQ群: 891211   
微信请扫码    >>>
现在联系顾问~
热词
    • 9. 发明申请
    • System and Method for Automated Testing of Complicated Dialog Systems
    • 复杂对话系统自动测试的系统和方法
    • US20090306995A1
    • 2009-12-10
    • US12133123
    • 2008-06-04
    • Fuliang WengHua Ai
    • Fuliang WengHua Ai
    • G06Q10/00G10L15/26G10L15/00
    • G10L15/01G06Q10/103
    • Embodiments of an automated dialog system testing method and component are described. This automated testing method and system supplements real human-based testing with simulated user input and incorporates a set of evaluation measures that focus on three basic aspects of task-oriented dialog systems, namely, understanding ability, efficiency, and the appropriateness of system actions. These measures are first applied on a corpus generated between a dialog system and a group of human users to demonstrate the validity of these measures with the human users' satisfaction levels. Results generally show that these measures are significantly correlated with these satisfaction levels. A regression model is then built to predict the user satisfaction scores using these evaluation measures. The regression model is applied on a simulated dialog corpus trained from the above real user corpus, and show that the user satisfaction score estimated from the simulated dialogs do not significantly differ from the real users' satisfaction scores. These evaluation measures can then be used to assess the system performance based on the estimated user satisfaction.
    • 描述了自动对话系统测试方法和组件的实施例。 这种自动测试方法和系统通过模拟用户输入来补充真正的基于人力的测试,并结合了一套关注面向任务的对话系统的三个基本方面的评估措施,即理解能力,效率和系统操作的适用性。 这些措施首先应用于对话系统和一组人类用户之间产生的语料库,以显示这些措施与人类用户满意度的有效性。 结果一般表明,这些措施与这些满意度水平显着相关。 然后建立回归模型,以使用这些评估措施来预测用户满意度评分。 回归模型应用于从上述真实用户语料库训练的模拟对话语料库,并显示从模拟对话框估计的用户满意度评分与实际用户满意度得分无显着差异。 这些评估措施可以用于根据估计的用户满意度来评估系统性能。