论文部分内容阅读
以植物学作为专业领域的样本,对专业领域的新词自动化识别进行探索。研究选取《中国植物志》作为样本集,在ICTCLAS切词的基础上采用N-Gram统计的方法提取新词的候选项,然后分别按照词频(TF)、文档频率(D)和平均词频(TF/D)对新词候选项排序,取一定范围内的候选项作为识别出的新词。实验结果表明,词频TF筛选新词候选项的识别效果最好,F值为0.65。该方法能够自动产生专业领域的用户词典,具有较强的可移植性。
Using botany as a sample of professional fields, we explore the automatic identification of new words in specialized fields. In this study, “Flora of China” was selected as a sample set. Candidates of new words were extracted by N-Gram statistics based on the ICTCLAS cut-off words. Then the frequency of TFs, TFs and TFs / D) Sort new word candidates, taking a range of candidates as recognized new words. The experimental results show that the word TF filter new word candidate identification effect is best, F value is 0.65. The method can automatically generate user’s dictionary in professional field and has strong portability.