Sequence to Sequence Learning with Neural Networks

Resource | v1 | created by semantic-scholar-bot |
Type Paper
Created 2014-01-01
Identifier unavailable


Deep Neural Networks (DNNs) are powerful models that have achieved excellent performance on difficult learning tasks. Although DNNs work well whenever large labeled training sets are available, they cannot be used to map sequences to sequences. In this paper, we present a general end-to-end approach to sequence learning that makes minimal assumptions on the sequence structure. Our method uses a multilayered Long Short-Term Memory (LSTM) to map the input sequence to a vector of a fixed dimensionality, and then another deep LSTM to decode the target sequence from the vector. Our main result is that on an English to French translation task from the WMT-14 dataset, the translations produced by the LSTM achieve a BLEU score of 34.8 on the entire test set, where the LSTM's BLEU score was penalized on out-of-vocabulary words. Additionally, the LSTM did not have difficulty on long sentences. For comparison, a phrase-based SMT system achieves a BLEU score of 33.3 on the same dataset.


about Computer science

Computer science is the study of computation and information. Computer science deals with theory of c...

relates to Learning representations by back-propagating errors

We describe a new learning procedure, back-propagation, for networks of neurone-like units. The proce...

relates to Neural Machine Translation by Jointly Learning to Align and Translate

Neural machine translation is a recently proposed approach to machine translation. Unlike the traditi...

relates to Long Short-Term Memory

Learning to store information over extended time intervals by recurrent backpropagation takes a very...

relates to ImageNet Classification with Deep Convolutional Neural Networks

We trained a large, deep convolutional neural network to classify the 1.2 million high-resolution ima...

relates to K

The aim of the paper is to show the effect of our times on mental health disorder. Organizational and...

Edit details Edit relations Attach new author Attach new topic Attach new resource
0.0 /10
useless alright awesome
from 0 reviews
Write comment Rate resource Tip: Rating is anonymous unless you also write a comment.
Resource level 0.0 /10
beginner intermediate advanced
Resource clarity 0.0 /10
hardly clear sometimes unclear perfectly clear
Reviewer's background 0.0 /10
none basics intermediate advanced expert
Comments 0
Currently, there aren't any comments.