# Neural Network

神经网络入门介绍 [Neural Networks and Deep Learning](http://neuralnetworksanddeeplearning.com/index.html) by Nielsen

LSTM 介绍 [Understanding LSTM Networks](http://colah.github.io/posts/2015-08-Understanding-LSTMs/) by Colah

Transformer:
- Transformer 介绍 [The Illustrated Transformer](https://jalammar.github.io/illustrated-transformer/) by Jay Alammar
- Transformer 论文 [Attention Is All You Need]()
- Transformer Pytorch 实现 [](http://nlp.seas.harvard.edu/2018/04/03/attention.html)

[Dilated CNN](https://arxiv.org/abs/1610.10099)

[RecNN](https://nlp.stanford.edu/~socherr/EMNLP2013_RNTN.pdf)

[Sequence to Sequence Learning with Neural Networks](https://arxiv.org/abs/1409.3215)

Attention:
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://arxiv.org/abs/1409.0473v7)
- [Effective Approaches to Attention-based Neural Machine Translation](https://nlp.stanford.edu/pubs/emnlp15_attn.pdf)



# Natural Language Processing

基于 NLTK 库的自然语言处理实践教程 [The NLTK Book](http://www.nltk.org/book/)

# Natural Language Processing based Neural

基于神经网络的自然语言处理方法的历史演进 [A Review of the Neural History of Natural Language Processing](http://ruder.io/a-review-of-the-recent-history-of-nlp/) by Sebastian Ruder

基于神经网络的自然语言处理常用方法简介 [A Primer on Neural Network Models for Natural Language Processing](http://u.cs.biu.ac.il/~yogo/nnlp.pdf) by Yoav Goldberg

基于深度学习的自然语言处理技术最佳实践 [Deep Learning for NLP Best Practices](http://ruder.io/deep-learning-nlp-best-practices/index.html#attentionhttp://ruder.io/deep-learning-nlp-best-practices/index.html)


# Language Model

单词级 n-gram 前向神经网络语言模型 [A Neural Probabilistic Language Model](http://jmlr.org/papers/volume3/bengio03a/bengio03a.pdf) (Bengio et al., 2001; 2003)

字符级 RNN 语言模型介绍 [The Unreasonable Effectiveness of Recurrent Neural Networks](http://karpathy.github.io/2015/05/21/rnn-effectiveness/)

[好玩的文本生成](https://www.msra.cn/zh-cn/news/features/ruihua-song-20161226)

# Word Embedding

http://ruder.io/word-embeddings-2017/

word2vec paper:
- [Efficient Estimation of Word Representations in Vector Space](https://arxiv.org/abs/1301.3781)
- [Distributed Representations of Words and Phrases and their Compositionality](https://arxiv.org/abs/1310.4546)

word2vec tutorial:

- [skip-gram](http://mccormickml.com/2016/04/19/word2vec-tutorial-the-skip-gram-model/) by Chris McCormick
- [negative sample](http://mccormickml.com/2017/01/11/word2vec-tutorial-part-2-negative-sampling/) by Chris McCormick

[Glove: Global Vectors for Word Representation](https://nlp.stanford.edu/pubs/glove.pdf)

[paragraph vector](https://cs.stanford.edu/~quocle/paragraph_vector.pdf)

[A Convolutional Neural Network for Modelling Sentences](https://www.aclweb.org/anthology/P14-1062)

[Convolutional Neural Networks for Sentence Classification](https://arxiv.org/abs/1408.5882)

[Skip-Thought Vectors](https://arxiv.org/abs/1506.06726)

[A Survey of Cross-lingual Word Embedding Models](https://arxiv.org/abs/1706.04902)

# Pretrained language models

https://thegradient.pub/nlp-imagenet/

[Semi-supervised Sequence Learning](https://arxiv.org/abs/1511.01432)

# Multi-task learning

http://ruder.io/multi-task/

[A Unified Architecture for Natural Language Processing: Deep Neural Networks with Multitask Learning](https://ronan.collobert.com/pub/matos/2008_nlp_icml.pdf)(Collobert and Weston 2008)

[auxiliary task](http://ruder.io/multi-task-learning-nlp/)

# Books&Blog

[深度学习 500 问](https://github.com/scutan90/DeepLearning-500-questions)

[中文自然语言处理相关资料](https://github.com/crownpku/awesome-chinese-nlp)

[中文自然语言处理](https://chinesenlp.xyz/#/zh/) by 滴滴人工智能实验室

http://ruder.io/

http://www.hankcs.com/