Optimizing Non-Decomposable Evaluation Metrics for Neural Machine Translation

• 作 者: Shi-Qi Shen Yang Liu Mao-Song Sun
• 机 构: Department of Computer Science and Technology, Tsinghua University, Beijing 100084, China;State Key
• 出 处: 计算机科学技术学报（英文版）
• 发表日期: 2017年4期
• 关键词: neural machine translation training criterion non-decomposable evaluation metric

While optimizing model parameters with respect to evaluation metrics has recently proven to benefit end-to-end neural machine translation (NMT), the evaluation metrics used in the training are restricted to be defined at the sentence level to facilitate online leing algorithms. This is undesirable because the final evaluation metrics used in the testing phase are usually non-decomposable (i.e., they are defined at the corpus level and cannot be expressed as the sum of sentence-level metrics). To minimize the discrepancy between the training and the testing, we propose to extend the minimum risk training (MRT) algorithm to take non-decomposable corpus-level evaluation metrics into consideration while still keeping the advantages of online training. This can be done by calculating corpus-level evaluation metrics on a subset of training data at each step in online training. Experiments on Chinese-English and English-French translation show that our approach improves the correlation between training and testing and significantly outperforms the MRT algorithm using decomposable evaluation metrics.