commit a69505acc18871072b83d6b0109670618ffd2f90 Author: wikty Date: Thu Aug 29 23:00:49 2019 +0800 init commit diff --git a/A Convolutional Neural Network for Modelling Sentences.pdf b/A Convolutional Neural Network for Modelling Sentences.pdf new file mode 100644 index 0000000..97de22c Binary files /dev/null and b/A Convolutional Neural Network for Modelling Sentences.pdf differ diff --git a/A Neural Probabilistic Language Model.pdf b/A Neural Probabilistic Language Model.pdf new file mode 100644 index 0000000..d754453 Binary files /dev/null and b/A Neural Probabilistic Language Model.pdf differ diff --git a/A Review of the Recent History of Natural Language Processing.pdf b/A Review of the Recent History of Natural Language Processing.pdf new file mode 100644 index 0000000..e6b58cb Binary files /dev/null and b/A Review of the Recent History of Natural Language Processing.pdf differ diff --git a/A Survey of Cross-lingual Word Embedding Models.pdf b/A Survey of Cross-lingual Word Embedding Models.pdf new file mode 100644 index 0000000..8393679 Binary files /dev/null and b/A Survey of Cross-lingual Word Embedding Models.pdf differ diff --git a/A Unified Architecture for Natural Language Processing - Deep Neural Networks with Multitask Learning.pdf b/A Unified Architecture for Natural Language Processing - Deep Neural Networks with Multitask Learning.pdf new file mode 100644 index 0000000..b058d3e Binary files /dev/null and b/A Unified Architecture for Natural Language Processing - Deep Neural Networks with Multitask Learning.pdf differ diff --git a/Attention Is All You Need.pdf b/Attention Is All You Need.pdf new file mode 100644 index 0000000..2b8c574 Binary files /dev/null and b/Attention Is All You Need.pdf differ diff --git a/Convolutional Neural Networks for Sentence Classification.pdf b/Convolutional Neural Networks for Sentence Classification.pdf new file mode 100644 index 0000000..6cba3e7 Binary files /dev/null and b/Convolutional Neural Networks for Sentence Classification.pdf differ diff --git a/Distributed Representations of Words and Phrases and their Compositionality.pdf b/Distributed Representations of Words and Phrases and their Compositionality.pdf new file mode 100644 index 0000000..04281dc Binary files /dev/null and b/Distributed Representations of Words and Phrases and their Compositionality.pdf differ diff --git a/Effective Approaches to Attention-based Neural Machine Translation.pdf b/Effective Approaches to Attention-based Neural Machine Translation.pdf new file mode 100644 index 0000000..a036f8e Binary files /dev/null and b/Effective Approaches to Attention-based Neural Machine Translation.pdf differ diff --git a/Efficient Estimation of Word Representations in Vector Space.pdf b/Efficient Estimation of Word Representations in Vector Space.pdf new file mode 100644 index 0000000..aa17ab0 Binary files /dev/null and b/Efficient Estimation of Word Representations in Vector Space.pdf differ diff --git a/Neural Machine Translation by Jointly Learning to Align and Translate.pdf b/Neural Machine Translation by Jointly Learning to Align and Translate.pdf new file mode 100644 index 0000000..18beaa1 Binary files /dev/null and b/Neural Machine Translation by Jointly Learning to Align and Translate.pdf differ diff --git a/README.md b/README.md new file mode 100644 index 0000000..0efd4dc --- /dev/null +++ b/README.md @@ -0,0 +1,94 @@ +# Neural Network + +神经网络入门介绍 [Neural Networks and Deep Learning](http://neuralnetworksanddeeplearning.com/index.html) by Nielsen + +LSTM 介绍 [Understanding LSTM Networks](http://colah.github.io/posts/2015-08-Understanding-LSTMs/) by Colah + +Transformer: +- Transformer 介绍 [The Illustrated Transformer](https://jalammar.github.io/illustrated-transformer/) by Jay Alammar +- Transformer 论文 [Attention Is All You Need]() +- Transformer Pytorch 实现 [](http://nlp.seas.harvard.edu/2018/04/03/attention.html) + +[Dilated CNN](https://arxiv.org/abs/1610.10099) + +[RecNN](https://nlp.stanford.edu/~socherr/EMNLP2013_RNTN.pdf) + +[Sequence to Sequence Learning with Neural Networks](https://arxiv.org/abs/1409.3215) + +Attention: +- [Neural Machine Translation by Jointly Learning to Align and Translate](https://arxiv.org/abs/1409.0473v7) +- [Effective Approaches to Attention-based Neural Machine Translation](https://nlp.stanford.edu/pubs/emnlp15_attn.pdf) + + + +# Natural Language Processing + +基于 NLTK 库的自然语言处理实践教程 [The NLTK Book](http://www.nltk.org/book/) + +# Natural Language Processing based Neural + +基于神经网络的自然语言处理方法的历史演进 [A Review of the Neural History of Natural Language Processing](http://ruder.io/a-review-of-the-recent-history-of-nlp/) by Sebastian Ruder + +基于神经网络的自然语言处理常用方法简介 [A Primer on Neural Network Models for Natural Language Processing](http://u.cs.biu.ac.il/~yogo/nnlp.pdf) by Yoav Goldberg + +基于深度学习的自然语言处理技术最佳实践 [Deep Learning for NLP Best Practices](http://ruder.io/deep-learning-nlp-best-practices/index.html#attentionhttp://ruder.io/deep-learning-nlp-best-practices/index.html) + + +# Language Model + +单词级 n-gram 前向神经网络语言模型 [A Neural Probabilistic Language Model](http://jmlr.org/papers/volume3/bengio03a/bengio03a.pdf) (Bengio et al., 2001; 2003) + +字符级 RNN 语言模型介绍 [The Unreasonable Effectiveness of Recurrent Neural Networks](http://karpathy.github.io/2015/05/21/rnn-effectiveness/) + +[好玩的文本生成](https://www.msra.cn/zh-cn/news/features/ruihua-song-20161226) + +# Word Embedding + +http://ruder.io/word-embeddings-2017/ + +word2vec paper: +- [Efficient Estimation of Word Representations in Vector Space](https://arxiv.org/abs/1301.3781) +- [Distributed Representations of Words and Phrases and their Compositionality](https://arxiv.org/abs/1310.4546) + +word2vec tutorial: + +- [skip-gram](http://mccormickml.com/2016/04/19/word2vec-tutorial-the-skip-gram-model/) by Chris McCormick +- [negative sample](http://mccormickml.com/2017/01/11/word2vec-tutorial-part-2-negative-sampling/) by Chris McCormick + +[Glove: Global Vectors for Word Representation](https://nlp.stanford.edu/pubs/glove.pdf) + +[paragraph vector](https://cs.stanford.edu/~quocle/paragraph_vector.pdf) + +[A Convolutional Neural Network for Modelling Sentences](https://www.aclweb.org/anthology/P14-1062) + +[Convolutional Neural Networks for Sentence Classification](https://arxiv.org/abs/1408.5882) + +[Skip-Thought Vectors](https://arxiv.org/abs/1506.06726) + +[A Survey of Cross-lingual Word Embedding Models](https://arxiv.org/abs/1706.04902) + +# Pretrained language models + +https://thegradient.pub/nlp-imagenet/ + +[Semi-supervised Sequence Learning](https://arxiv.org/abs/1511.01432) + +# Multi-task learning + +http://ruder.io/multi-task/ + +[A Unified Architecture for Natural Language Processing: Deep Neural Networks with Multitask Learning](https://ronan.collobert.com/pub/matos/2008_nlp_icml.pdf)(Collobert and Weston 2008) + +[auxiliary task](http://ruder.io/multi-task-learning-nlp/) + +# Books&Blog + +[深度学习 500 问](https://github.com/scutan90/DeepLearning-500-questions) + +[中文自然语言处理相关资料](https://github.com/crownpku/awesome-chinese-nlp) + +[中文自然语言处理](https://chinesenlp.xyz/#/zh/) by 滴滴人工智能实验室 + +http://ruder.io/ + +http://www.hankcs.com/ diff --git a/RecNN.pdf b/RecNN.pdf new file mode 100644 index 0000000..3603c93 Binary files /dev/null and b/RecNN.pdf differ diff --git a/Semi-supervised Sequence Learning.pdf b/Semi-supervised Sequence Learning.pdf new file mode 100644 index 0000000..97f9a2b Binary files /dev/null and b/Semi-supervised Sequence Learning.pdf differ diff --git a/Sequence to Sequence Learning with Neural Networks.pdf b/Sequence to Sequence Learning with Neural Networks.pdf new file mode 100644 index 0000000..658feb4 Binary files /dev/null and b/Sequence to Sequence Learning with Neural Networks.pdf differ diff --git a/Skip-Thought Vectors.pdf b/Skip-Thought Vectors.pdf new file mode 100644 index 0000000..0a7f85b Binary files /dev/null and b/Skip-Thought Vectors.pdf differ diff --git a/The Illustrated Transformer – Jay Alammar.pdf b/The Illustrated Transformer – Jay Alammar.pdf new file mode 100644 index 0000000..5e96d76 Binary files /dev/null and b/The Illustrated Transformer – Jay Alammar.pdf differ diff --git a/Understanding LSTM Networks -- colah's blog.pdf b/Understanding LSTM Networks -- colah's blog.pdf new file mode 100644 index 0000000..36e599a Binary files /dev/null and b/Understanding LSTM Networks -- colah's blog.pdf differ diff --git a/glove.pdf b/glove.pdf new file mode 100644 index 0000000..64891c4 Binary files /dev/null and b/glove.pdf differ diff --git a/paragraph_vector.pdf b/paragraph_vector.pdf new file mode 100644 index 0000000..2a35752 Binary files /dev/null and b/paragraph_vector.pdf differ