Language Model
单词级 n-gram 前向神经网络语言模型 A Neural Probabilistic Language Model (Bengio et al., 2001; 2003)
字符级 RNN 语言模型介绍 The Unreasonable Effectiveness of Recurrent Neural Networks
字符级 n-gram 语言模型跟 RNN 对比 The unreasonable effectiveness of Character-level Language Models
Word Embedding
review:
word2vec tutorial:
- skip-gram by Chris McCormick
- negative sample by Chris McCormick
word2vec paper:
- Efficient Estimation of Word Representations in Vector Space
- Distributed Representations of Words and Phrases and their Compositionality
embedding for downstream tasks:
-
A Unified Architecture for Natural Language Processing: Deep Neural Networks with Multitask Learning(Collobert and Weston 2008)
-
http://colah.github.io/posts/2014-07-NLP-RNNs-Representations/
distributional semantic model:
- From frequency to meaning: Vector space models of semantics
- A Brief History of Word Embeddings
- https://rare-technologies.com/making-sense-of-word2vec/
- https://ruder.io/secret-word2vec/index.html
- Glove: Global Vectors for Word Representation
- Improving Distributional Similarity with Lessons Learned from Word Embeddings
sentence embedding:
Convolutional Neural Networks for Sentence Classification
A Convolutional Neural Network for Modelling Sentences
Semi-supervised Sequence Learning
Supervised Learning of Universal Sentence Representations from Natural Language Inference Data
https://github.com/Separius/awesome-sentence-embedding
https://www.dataiku.com/product/plugins/sentence-embedding/
Are distributional representations ready for the real world?
more aboue embedding:
Enriching Word Vectors with Subword Information
https://ruder.io/word-embeddings-2017/
A Survey of Cross-lingual Word Embedding Models
visual embedding:
Machine Learning
https://cs231n.github.io/linear-classify/
An overview of gradient descent optimization algorithms
Neural Network
神经网络入门介绍 Neural Networks and Deep Learning by Nielsen
RNN:
- LSTM 介绍 Understanding LSTM Networks by Colah
- https://distill.pub/2016/augmented-rnns/
CNN:
- http://colah.github.io/posts/2014-07-Conv-Nets-Modular/
- http://colah.github.io/posts/2014-07-Understanding-Convolutions/
- http://www.wildml.com/2015/11/understanding-convolutional-neural-networks-for-nlp/
- Dilated CNN
RecNN:
Seq2Seq
From SMT to NMT
- Statistical phrase-based translation
- Recurrent continuous translation models
- Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation
- Sequence to Sequence Learning with Neural Networks
- On the properties of neural machine translation: Encoder–Decoder approaches
Attention
- Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
- Neural Machine Translation by Jointly Learning to Align and Translate
- Effective Approaches to Attention-based Neural Machine Translation
- https://distill.pub/2016/augmented-rnns/
Transformer:
- Transformer 介绍 The Illustrated Transformer by Jay Alammar
- Transformer 论文 Attention Is All You Need
- Transformer Pytorch 实现
Natural Language Processing
基于 NLTK 库的自然语言处理实践教程 The NLTK Book
基于神经网络的自然语言处理方法的历史演进 A Review of the Neural History of Natural Language Processing by Sebastian Ruder
基于神经网络的自然语言处理常用方法简介 A Primer on Neural Network Models for Natural Language Processing by Yoav Goldberg
基于深度学习的自然语言处理技术最佳实践 Deep Learning for NLP Best Practices
Pretrained language models
NLP's ImageNet moment has arrived
The Illustrated BERT, ELMo, and others
Semi-supervised Sequence Learning
https://github.com/huggingface/transformers
https://github.com/hanxiao/bert-as-service
Text Classification
Convolutional Neural Networks for Sentence Classification
http://www.wildml.com/2015/12/implementing-a-cnn-for-text-classification-in-tensorflow/
https://github.com/yoonkim/CNN_sentence
http://albertxiebnu.github.io/fasttext/
Text embedding search
https://github.com/facebookresearch/faiss
https://engineering.fb.com/data-infrastructure/faiss-a-library-for-efficient-similarity-search/
https://www.elastic.co/blog/text-similarity-search-with-vectors-in-elasticsearch
https://hanxiao.io/2019/11/22/Video-Semantic-Search-in-Large-Scale-using-GNES-and-TF-2-0/
Tools
fasttext: https://github.com/facebookresearch/fastText
textcnn: https://github.com/dennybritz/cnn-text-classification-tf
Resources
中文自然语言处理各任务最新进展 by 滴滴人工智能实验室