Improving Word Embeddings for Low Frequency Words by Pseudo Contexts

来源 :第十六届全国计算语言学学术会议暨第五届基于自然标注大数据的自然语言处理国际学术研讨会 | 被引量 : 0次 | 上传用户：Stephanie1121

【摘要】

：

【作者】

：

Fang Li Xiaojie Wang

【机构】

：

School of Computer,Beijing University of Posts and Telecommunications,Beijing,China

【出处】

：

第十六届全国计算语言学学术会议暨第五届基于自然标注大数据的自然语言处理国际学术研讨会

【发表日期】

：

2017年7期

【关键词】

：

Word Embedding Low Freuqcy Word

下载到本地 , 更方便阅读

下载此文赞助VIP

声明 : 本文档内容版权归属内容提供方 , 如果您对本文有版权争议 , 可与客服联系进行内容授权或下架

论文部分内容阅读

　　This paper investigates relations between word semantic den-sity and word frequency.A distributed representations based word av-erage similarity is defined as the measure of word semantic density.We find that the average similarities of low frequency words are always big-ger than that of high frequency words,when the frequency approaches to 400 around,the average similarity tends to stable.The finding keeps cor-rect with changes of the size of training corpus,dimension of distributed representations and number of negative samples in skip-gram model.It also keeps on 17 different languages.Basing on the finding,we propose a pseudo context skip-gram model,which makes use of context words of semantic nearest neighbors of target words.Experiment results show our model achieves significant performance improvements in both word similarity and analogy tasks.

其他文献

Tibetan Syllable-based Functional Chunk Boundary Identification

Tibetan syntactic functional chunk parsing is aimed at identifyingsyntactic constituents of Tibetan sentences.In this paper,based on the Tibetan syntactic functional chunk description system,we propos

会议

Tibetan Syntactic Functional ChunkChunk Boundary Recogni-tionSyllableSyntacti

Unsupervised Joint Entity Linking over Question Answering Pair with Global Knowledge

We consider the task of entity linking over question answering pair(QA-pair).In conventional approaches of entity linking,all the entities whether in one sentence or not are considered the same.We foc

会议

joint entity linkingquestion answering pairglobal knowledgein-tegral linear p

Harvest Uyghur-Chinese Aligned-Sentences Bitexts from Multilingual Sites Based on Word Embedding

Obtaining bilingual parallel data from the multilingual websites is along-standing research problem,which is very benefit for resource-scarce lan-guages.In this paper,we present an approach for obtain

会议

bilingual parallel dataword embeddingresource-scarce languages

Closed-Set Chinese Word Segmentation Based on Convolutional Neural Network Model

This paper proposes a neural model for closed-set Chinese word segmentation.The model follows the character-based approach which assigns a class label to each character,indicating its relative po-siti

会议

Chinese word segmentationDeep learningConvolutional neural networks

Improving Event Detection via Information Sharing among Related Event Types

Event detection suffers from data sparseness and label imbalance prob-lem due to the expensive cost of manual annotations of events.To address this problem,we propose a novel approach that allows for

会议

Hierarchical Gated Recurrent Neural Tensor Network for Answer Triggering

In this paper,we focus on the problem of answer triggering ad-dressed by Yang et al.(2015),which is a critical component for a real-world question answering system.We employ a hierarchical gated recur

会议

Answer TriggeringQuestion AnsweringHierarchical gated recur-rent neural tensor

Joint Extraction of Multiple Relations and Entities by using a Hybrid Neural Network

This paper proposes a novel end-to-end neural model to jointly extract entities and relations in a sentence.Unlike most exist-ing approaches,the proposed model uses a hybrid neural network to automati

会议

Information ExtractionNeural Networks

Language Model for Mongolian Polyphone Proofreading

Mongolian text proofreading is the particularly difficult task because of its unique polyphonic alphabet,morphological ambiguity and agglutinative feature,and coding errors are currently pervasive in

会议

MongolianPolyphoneAutomatic Proofreading SystemMorpho-logical Ambiguity

Collective Entity Linking on Relational Graph Model with Mentions

Given a source document with extracted mentions,entity linking callsfor map-ping the mention to an entity in reference knowledge base.Previous en-tity linking approaches mainly focus on generic statis

会议

Collective Entity LinkingEntity DisambiguationRelational Graph

Cost-aware Learning Rate for Neural Machine Translation

Neural Machine Translation(NMT)has drawn much attention due to its promising translation performance in recent years.The conventional optimiza-tion algorithm for NMT sets a unified learning rate for e

会议

Neural Machine TranslationCost-aware Learning Rate

Improving Word Embeddings for Low Frequency Words by Pseudo Contexts

其他学术论文