A NEURAL MACHINE LANGUAGE TRANSLATION SYSTEM FROM GERMAN TO ENGLISH
M
Department of Computer Science and Engineering, Anna University CEG campus, Chennai India
Abstract Machine Translation is a Natural Language Processing,attainingsignificantattention to fullyautomate the system that can translate source content into the target languages. The traditional systems focus on the word level context while neglecting the sentence level context most often, resulting in meaningless translation. This paper focuses on the Transformer based MT which is used for translation integrated with Fuzzy Semantic Representation (FSR) for handling the occurrence of rare words in the sentence level context. In the existing method there is a mismatch of translation but the proposed system is more superior due to the sentence context inclusion. The algorithms proposed in this paper can be used for multilingualtranslationsystems. TheFSRgrouprarewords together and represent sentence level context as Latent Topic Representation (LTR) using convolutional neural network. In particular, our algorithm can achieve a significant performance for WMT En De bilingual parallel corpus and is used for translation handling the Out of Vocabulary words using clustering of <UNK> tags. The model performance is enhanced with hyper parameter optimization obtaining a significantly high BLEU score, and significantlylowTERscorewithabe
Key Words: Natural Language Processing, Neural Machine Translation(NMT), Transformer MT, Fuzzy Semantic Representation(FSR), Latent Topic Representation(LTR), Convolutional Neural Network (CNN).
I. INTRODUCTION
Machine translation (MT) refers to fully automated software which translates source language into target language. Humans may use machine translation (MT) to help them render speech and text into another language, or the MT software may operate without human intervention. Allowing a model to automatically (soft )searchforpartsof a sourcesentencethat are relevant to predictingatargetword[1].
Automatic or machine translation is one of the most challenging AI tasks given the fluidity of human language. The classical approach of the rule based machine translationsystemswereusedfortranslation,whichwere replaced by statistical methods in the 1990s. In recent times, deep neural network models achieve state of the artresultsforthelanguagetranslationthatisaptlynamed
as NeuralMachineTranslation(NMT).
Neural machine translation, or NMT in short, uses the neural network models for machine translation. NMT approaches are more powerful at capturing context beyond phrase boundaries and were shown to better exploit available training data[4]. The key benefit for translation is that a single system can be trained directly on source and target text, thereby not requiring the pipeline of specialized systems which are used in statistical machine translation(SMT). As such, NMT systems are said to be end to end systems as only one transformer likemodelisrequiredforthetranslation.
Transformer is an end to end model with encoder decoder architecture for seq seq translation. The encoder consists of a set of encoding layers that processes the input iteratively one layer after another and the decoder consists of a set of decoding layers that does the same thing to the output of the encoder. Each of the encoder layers function is to process its input to generate word encodings (a.k.a word embeddings), containing information about which parts of the input sentences are relevant with each other. It passes its set of word encodings to the next encoder layer as inputs to get a better translation. Each decoder layer takes the output of the encoder layer as input by taking all the word encodings and processes them, using their incorporated contextual information to generate an output sequence[12].
To achieve this, each of the encoder and decoder layers makesuseofanattentionmechanism(teacherenforcing), which for each input word weighs the relevance of every other input word and draws information (context) from them accordingly to produce the output sentence. Each layerofdecoderalsohasanadditionalattentionweightto draw additional information from the outputs of previous decoders, before the decoder layer draws information from the encodings. Both the encoder and decoder layers have a feed forward neural network for additional processing of the outputs, and contain residual connections and layer normalization steps[13].
Asinthecurrentscenariothereexistseverallanguagesin the world. All the languages can’t be understood, so there isaneedforalanguagetranslatorbetweenthesourceand target language. Natural Language Translation, which is a
International Research Journal of Engineering and Technology (IRJET) e ISSN:2395 0056
Volume: 09 Issue: 04 | Apr 2022 www.irjet.net p ISSN:2395 0072
NLPtask,istheprocessofconvertingsourcelanguageinto the target language using a Seq Seq Encoder Decoder model[14]. A machine translator is a NLP task which solves the problem of taking the input sentence of the source language and giving it to the Neural Machine Translator (NMT) for processing the language and giving the output of the target language. Neural machine translation (NMT) is an approach to machine translation that uses an artificial neural network to predict the likelihood of a sequence of words, typically modeling entire sentences in a single integrated model. Topic modeling techniques, such as Latent Dirichlet Allocation (LDA) is very widely used for to capture semantics of the sentence[2]. This is an effective approach for language translation.
The proposed system uses a Fuzzy Semantic Representation (FSR) of rare words to overcome the <UNK> tag in the sentence along with the Latent Topic Representation (LTR) using CNN to obtain the sentence level context. FSR and LTR are integrated with the NMT model to predict the target sentence given the source sentence. Thus an efficient machine translation is generatedusingtheTransformerbasedencoder decoder MachineTranslator[15].
II. RECENT WORKS
Kehai Chen et.al[3] have discussed sentence level context as a latent topic representation and designs a topic attention to integrate source sentence level topic context information into both Attention based and Transformer based NMT. LTR is represented using a Convolutional Neural Network (CNN). The performance of NMT improves by modeling source topics and translations jointly. This study explored the dependence of source topic information on its sentence level context and proposed a topic attention that integrated latent topic representations into the existing NMT architecture to improve translation prediction. To enhance target word prediction it needs to exploit explicit source topic information. It focuses on sentence level context to learn translation knowledge instead of document level context information, multi source information, multi model informationorsyntaxinformation.
Muyun Yang , Shujie Liu, Kehai Chen , et.al (2020) proposes to build a fuzzy semantic representation (FSR) method for rare words through a hierarchical clustering method to group rare words together, and will be integrated with the help of encoder decoder framework. Chinese to English sentence translation tasks confirm a significant improvement in the translation quality for the proposed method. To best preserve the semantic information,thisarticleproposesahierarchicalclustering embedding approach to fuzzy semantic representation (FSR)ofrarewordsinNMT.
Xing Wang, Zhaopeng Tu, and Min Zhang (2015) have explained a general framework forincorporating the SMT word knowledge into NMT to alleviate above word level limitations. The NMT decoder makes more accurate word predictionthatistobetranslatedby referringtotheSMT word recommendations in both training and testing phases. The SMT word predictions are used as prior knowledgetoadjusttheNMTwordgenerationprobability, which unitizes a neural network based classifier to digest thediscretewordknowledge.
RuiWang,MasaoUtiyama,AndrewFinchand et.al.(2018) havediscussedthe4goalsforsentencelevelNMTdomain adaptation.First:TheNMT’sinternalsentenceembedding isexploitedandthesentenceembeddingsimilarityisused to select out of domain sentences which are close to the in domain corpus. Second: propose three sentence weighting methods, i.e., sentence weighting, domain weighting and batch weighting, to balance the data distribution during NMT training. Third: In addition propose dynamic training methods to adjust the sentence selection and weighting during NMT training. Fourth: To solve the multi domain problem in a real world NMT scenario where the domain distributions of training and testing data often mismatch, proposed a multi domain sentence weighting method to balance the domain distributions of training data and match the domain distributionsoftrainingandtestingdata.
HaipengSun,RuiWang,KehaiChenandet.al(2020)have discussed about empirical relationships between unsupervised neural machine translation (UNMT) and unsupervised bilingual word embedding (UBWE) pre training and cross lingual masked language model (CMLM) pre training. So a novel UNMT structure with cross lingual language representation agreement to capture the interaction between UBWE / CMLM and UNMTduringUNMTtraining.
Kehai Chen , Tiejun Zhao, Muyun Yang and et.al (2018) havegivenanovelneuralapproachtosourcedependence based context representation for translation prediction. The proposed model is capable of not only encoding source long distance dependencies but also capturing functional similarities to better predict translations. It is incorporated into phrase based translation (PBT) and hierarchicalphrase basedtranslation(HPBT)models. The evaluationmetricsusedisCaseInsensitive BLEU 4metric which outperforms all the baseline systems with better translation. NMT still performs better than the proposed modelwithasignificantlygreaterBLEUscore[5].
The methodologies and architectures employed in the existingsystemsformachinetranslationwerestudiedand the issues in the existing systems include: inadequate trainingofmodels,insufficientdata,somemodelsarebuilt for a specific language alone and are not applicable for
© 2022, IRJET | Impact Factor value: 7.529 | ISO 9001:2008 Certified Journal | Page3095
other common languages like English. Some models translate sentences that resemble machine generated sentencesthatdonotsoundlikethelanguagethathumans usetospeak[6][7].MachineTranslationisneededtosolve this problem and the existing methods and related works that were also studied. Transformer based NMT are more naturalandlessartificialwhencomparedtoothermodels.
III. PROPOSED SYSTEM
The transformer is an end to end system having encoder decoder architecture. The encoder consists of a set of encoding layers that processes the input sentence iteratively one layer after another to produce sentence vectors along with the rare words with the help of FSR module and the decoder consists of a set of decoding layers that does the same thing to the output sentence vector of the encoder along with the topic vector to get a moreaccuratetranslation.
The function of each encoder layer is to process its input language sentence (English) to generate encodings in vector form referred to as embeddings, containing information about which parts of the inputs are relevant toeachother.Itpassesitssetofencodings(obtainedfrom the previous encoding layer) to the next encoder layer as inputs. Each decoder layer does the opposite of encoder, taking all the encodings obtained from the encoder and processes them, using their incorporated contextual information to generate an output sequence. To achieve this, each encoder and decoder layer makes use of an attention mechanism (assigns weight to the input word embeddingsthatistobetranslated),whichforeachinput, weighs the relevance of every other input and draws contextual information from them accordingly to produce the output. Each decoder layer also has an additional attention mechanism which draws information from the outputsofpreviousdecoders(assignsweighttotheoutput word embeddings that is being translated), before the decoder layer draws information from the output of encodings. Figure 1 shows the transformer integrating LTR for sentence context along with LTR to handle rare words which are OOV. Both the encoder and decoder layers have a feed forward neural network for additional processing of the outputs, and contain residual connectionsandlayernormalizationsteps.
Figure 1: Transformer Architecture with FSR + LTR
The modules required to build an end to end Encoder Decoder system for translation is described here. From figure1thehighlevelblock diagramwherethedataset of bilingual parallel English German corpus is preprocessed togetacleansentencebyremovingspecialcharactersand punctuation marks. The cleaned dataset is splitted into separatefilesofGermansentencesandEnglish sentences. The words which are OOV are given to the FSR most commonly related together under a universal <UNK> tag. The embedding module converts the sentence sequence into its corresponding vector representation. The positional encoding module uses sinusoidal functions to indicate the position of the words in the sentence at both source and target side. The context of the sentence is handled as Latent Topic Representation which uses CNN to extract the context of the sentence. The transformer encoder and decoder which uses a self attention mechanism to translate. Finally the decoded vector is passed to the linear classifier and softmax functions. Finally the google translator is integrated into our NMT systemtocheck thecorrectnessofthepredictedsentence usingthesentencepredictionmodule.
and Technology
A. TRANSFORMER ENCODER DECODER
Each encoder consists of two major components: a self attention mechanism and a feed forward neural network. The self attention mechanism takes in a set of input encodings from the previous encoder and weighs their relevance to each other to generate a set of output encodings. The feed forward neural network then further processeseachoutputencodingindividually.Theseoutput encodings are finally passed to the next encoder as its input, as well as the decoders. The first encoder takes positional information and embeddings of the input sequence as its input, rather than encodings. The positionalinformationisnecessaryfortheTransformerto make use of the order of the sequence, because no other partoftheTransformermakesuseofthis[8].
Each decoder consists of three major components: a self attention mechanism, an attention mechanism over the encodings, and a feed forward neural network. The decoder functions in a similar fashion to the encoder, but an additional attention mechanism is inserted which instead draws relevant information from the encodings generatedbytheencoders.Likethefirstencoder,thefirst decoder takes positional information and embeddings of the output sequence as its input, rather than encodings. Sincethetransformershouldnotusethecurrentorfuture output to predict an output though, the output sequence must be partially masked to prevent this reverse information flow. The last decoder is followed by a final linear transformation and softmax layer, to produce the outputprobabilitiesoverthevocabulary[9].
Algorithm : Continuous Bag of Words - Word2Vec model
Input: Onehotvectorofinput/outputsentenceofsizeV
Output: WordVectorRepresentation dependenceofone wordonotherwordsinsentence
Begin 1. Padding bits are added to the one hot vector to make equilengthsentence
2. Wvn is the weight matrix that maps input X to the hiddenlayer(V*Ndimensionalmatrix)
3.Wnvistheweightmatrixthatmapshiddenlayeroutput tothefinaloutputlayer (N*Vdimensionalmatrix)
4. Hidden layer neurons copy the weighted sum of inputs tothenextlayer
5. Non linearity Activation function of Softmax is used in theOutputlayer
End
The encoder receives a source sentence x and encodes each prefix using a recurrent neural network that recursively combines embeddings xj for each word
position j:
where f is a non linear function. Reverse encodings hj are computed similarly to represent suffixes of the sentence. These vector representations are stacked to form hj, a representation of the whole sentence focused on position j. The decoder predicts each target word yi sequentially according to the distribution
wheresi isahiddendecoderstatesummarizingtheprefix of the translation generated so far, ci is a summary of the entire input sequence, and g is another non linear function. Encoder and decoder parameters are jointly optimized to maximize the log likelihood of a training corpus[5].
B. FSR - FUZZY SEMANTIC REPRESENTATION
Algorithm : Embedding Tree Construction
Begin
1.TraintoolkitWord2VecisusedforMonolingualData
2. Words OOV but appears >3 times are Clustered into M classesbyk means
3. Class Embedding = Mean(Word Embeddings in the class)
4.Eachclasshas<UNKj>containingremainingwordsthat belongtothisclass(CosineSimilarity)
5.<UNKj>embedding=Mean(Remainingwordsinclass) End
The3 levelembeddingtreeisconstructedbasedontheK means clustering and is categorised based on the class whichfitsitintowhichisasshowninfig.2
Figure 2 : Classification of words in embedding tree
Algorithm : FSR Generation
Begin
1.InputVectorisPathfrom roottotherare word,todeal withdatasparseness
2. Differentiate class information and word information forRarewords
3.ComputesSourceandTargetRarewordInputvector
End
International Research Journal of Engineering and Technology (IRJET) e ISSN:2395 0056 Volume: 09 Issue: 04 | Apr 2022 www.irjet.net p ISSN:2395 0072
Fortheencoder,withpreviousencoderstatehi−1,previous inputwordembeddingExi−1,encodercomputesthesource rare word input vector Exunk with attention mechanism as
results.
where Ep2 and Ep1 are the embeddings of word and class node in the path, respectively, and Ux is a random initializedweightmatrix.Similartotheencoder,thetarget rare word input vector Eyunk is based on previous decoder state si−1, previous predicted word embedding Ey
,previouscontextvectorc
,andweightmatrixUy
Thewordswhichareoutofvocabulary(OOV)aredifficult totranslateandneedsspecialattentionforthosewordsin a sentence. TheFSRmodule takescareofdata sparseness wherethewordswhichare
outofdictionarycanbetranslatedwithbetteraccuracyby Differentiate class information and word information for Rarewords.
To integrate the proposed FSR into the existing transformer based NMT for improving target translation, especially translations of rare words. Specifically, the proposed FSR is only applied to the input representation layer,whoseoutputisthesummationofwordembeddings andpositionalembeddingsintheinputsentence.Takethe source sentence as an example, if this word xj is not in vocabulary, the learned Exunk is used to replace the originalOOVvector; ifonewordxj isin vocabulary,itswordvectorisxj
The xj is, then, used to replace the existing xj to learn the inputrepresentation.
C. LTR - LATENT TOPIC REPRESENTATION
Figure 3 shows all the layers of CNN model with (D X L) dimensions of the sentence and extracts the meaning of the sentence (context) for getting better translation
© 2022, IRJET
Impact Factor value: 7.529
Figure 3 : CNN model for LTR
The layers of CNN extracts the semantics of the sentence based on the topics which are being defined initially. The semantics closely matched with the most appropriate topic.
Algorithm : Extracting Key Information
INPUT : {T1, T2, T3,..., Tm} is M Topics X={x1, x2,x3,...,xj}isinputsentence(Wordembedded)
OUTPUT:
Topicdistributionofasentence topiccontextvector Begin
1. J*D input matrix layer is passed to 2M Convolution Filtersfor3consecutiverowswhereDisthedimensionof thewordvectorandJisthesentenceLength
2. The convolution Layer performs M row wise max operationstogenerateMaxPoolingLayer
3.MaxPoolingLayerundergoesTanhfunctiontogenerate OutputLayer
4.FinallyTismappedto{Key,Value}pair End
Max Pooling Layer: This takes row wise maximum over pairsofconsecutiverowsofL:
Pm =max(L2m−1,L2m),1≤m≤M
TheresultingoutputfeaturemapisP: P={P1,P2,...,PM}.
Note that a row pair wise max pooling operation is applied to L2m−1 and L2m−1 to gain a D dimensional topic
ISO 9001:2008
Journal
3098
International Research Journal of Engineering and Technology (IRJET) e ISSN:2395
Volume: 09 Issue: 04 | Apr 2022 www.irjet.net p ISSN:2395 0072
feature vector Pm. Compared with the max pooling of the sentence classification task, we use a D dimensional topic feature vector Pm to represent a topic in the input sentence instead of a topic feature value. As we all know, the vector representation has a better ability to encode word or topic information. Meanwhile, the vector representations of topics are also easier integrated into theexistingNMTarchitecture.
Output layer:This applies the tanh function over Pm to obtainaLTRTm:
T
The T is our proposed LTRs, which will later be used to learn the topic context vector for NMT. The model parametersforlearningTareasthefollowing:
TABLE 1 : LOSS AND ACCURACY SCORES
TheCNN model layersextractsthekeyinformationasthe topiccontextvectorfortheNMTtoworkeffectively.
D. DATASET DESCRIPTION
WMT '14 is open source data available on the Stanford site. The modality of the dataset is text sentences. The trainingdataconsistsof4.5Millionsentencepairsofboth German and English sentence pairs. The training and validation is splitted in the ratio of 80% and 20% respectively.Thedatasetisofrichtextformatandmustbe preprocessed so that it can be fed into the model for training. The dataset also includes the vocabularies of wordswhichoccurmorefrequently,saytop50Kfrequent words. The dictionary is formed by extracting from alignmentdata[10][11].
IV. EVALUATION
The translated sentences were evaluated using Training accuracy,BLEUandTERmetrics.Thelossandaccuracyof training is considered as one of the most important steps towards evaluation. It is evident from Table 1 that accuracyincreasesandlossdecreaseswiththeincreasein epochs.Thelossiscalculatedbasedonthecrossentropy.
Cross entropy Loss= ∑ i=1 ∑ j=1 yi,j log(pi , j)
Accuracy = No. of correct
Total no. of
= 1 Accuracy
© 2022, IRJET
Impact Factor value: 7.529
Bilingual Evaluation Understudy (BLEU) is a score for comparing target sentence translation with one or more reference translations. The formula for calculating BLEU Scoreisgivenas:
BLEU = BP . exp( ∑ Wn log pn )
Brevity Penalty (BP) = 1 if c > r e(1 r/c) if c <= r
wherer countofwordsinareferencetranslation; c countofwordsinacandidatetranslation.
Figure 4 shows that the training loss decreases as the numberoftrainingepochsincreases.Attherangeof15to 25 epochs the loss doesn't decrease significantly and reaches the optimal point with a loss of almost 3% only. Further increasing the epochs there is no further improvementandthetrainingtimealsoincreaseswiththe increaseinthenumberofepochs.
Figure 4: Graph for Training Loss Score vs Epochs
Figure5showsthatthetrainingaccuracyincreasesasthe numberoftrainingepochsincreases.Attherangeof15to 25 epochs the accuracy doesn't increase significantly and reachestheoptimalpointwithanaccuracyofalmost98%. Further increasing the epochs there is no further improvementandthetrainingtimealsoincreaseswiththe increaseinthenumberofepochs.
ISO 9001:2008
3099
International
Fromthefigure7,showstheBLUE
metricfor5
from German to English
are obtained based on the above
formula (Brevity Penalty) for the translatedsentences.
Figure 5: Graph for Training Accuracy Score vs Epochs
Figure 7: BLEU score for 5 sample translations
Translation
the
of
required to
into one of the references. The formulaforcalculatingTERScoreisgivenas:
FOR
Figure 6: Bar Plot for Loss and Accuracy with Epochs
From
From
of
from
the
for
to English
From the graph above the Testcase
has
thesentencelengthislong.Soourmodelislimited
Figure 8: TER score for 5 sample translations
The performance of the system is evaluated based on the BLEUscoreandtheTERscore.Theaimistomaximizethe BLEU score so that the predicted translation which is closely related to the reference translation (human interference) and to minimize the TER score so that the error in the translation becomes less. For sample 5 test cases are evaluated using the above two scores and this canbeextendedtotheentirecorpusifrequired.
Fortraining,thetotaltimetakenbythebatchesperepoch istabulatedbelow.Eachepochhas6batchesofsize100to betrainedandthesumofallthetimeofbatchesperepoch is calculated. The GPU with RAM of 25 GB is used to run the model and the time taken per epoch is given in seconds (sec). The training time for every 5 epochs increases by a slight margin. From the training time for 50000 training sentences the time for every epoch increases slightly and the average time taken for every epoch is almost 45 seconds. To minimize the time per epoch,afasterGPUwithincreasedRAMsizeisneeded.
Thetotaltimetakenbythebatchesperepochistabulated below.Eachepochhas6batchesofsize100to be trained and the sum of all the time of batches per epoch is calculatedandshowninTable4.TheGPUwithRAMof25 GBisusedtorunthemodelandthetimetakenperepoch is given in seconds (sec). The training time for every 5 epochsincreasesbyaslightmargin.
:
TRAINING
The plotting of Training Time for different epochs is shown in fig. 9, for every 5 epochs the time is almost constant but the time is largely dependent on the corpus size.Thetimetakenfortrainingisdirectlyproportionalto thecorpussizeandthenumberofepochs.Themajorityof time is spent in model training for highly accurate translation, so the optimal number of epochs must be maintainedforbetterefficiency.
Figure 9: Bar Plot of Training Time for different epochs
V. RESULTS
from the above tables we have the translated sentencesoutputbythemodelwiththegoogletranslators output.Theendtoendsystemforthetranslationofsimple german sentences to english sentences is obtained. Thus the sample test cases with the Transformer based NMT along with FSR to handle OOV words and LTR to handle sentence context has better translation BLUE scores and minimal TER scoresincomparison withall other baseline models. The preprocessing of sentences is done with the help of regular expression by removing the special characters from the sentence, for both german as well as english sentences along with <start> and <end> tag to denotethestartingandendingofsentence.Thewordsina sentence, which has been preprocessed are tagged with numbers based on the ordering given by the dictionary, for both german as well as english sentences. The Transformer model along with FSR and LTR is trained with 25 epochs to get optimal results for the translation. The training along with the accuracy and loss for each batch in epochs are done. The WordtoVec model (pretrainedmodel)isusedtoobtainthewordembeddings (encoded vector). The German to English translation for the trained data is tested and compared with the google’s translation for validating the translated output. The attentiongraphisplottedwhichissimilartoheatmap(i.e) the brightest color is taken as the matched translated word. The transformer model uses the self attention mechanismforthetranslation.
International Research Journal of Engineering and Technology (IRJET) e ISSN:2395 0056
Volume: 09 Issue: 04 | Apr 2022 www.irjet.net p ISSN:2395 0072
VI. CONCLUSION
Inthiswork,buildinganendtoendLanguageTranslation system which uses a Transformer based NMT to convert German Language sentences to English Language sentences.Thecontextofa sentenceistakenintoaccount sothatthe targetsentencetranslation will beof the same meaning of the source sentence. The Self Attention mechanism used in the transformer predicts a better translatedsentence.TherarewordswhichareOOVinthe sentence are handled using FSR with the help of hierarchical clustering which is integrated with the transformermodel.Thecontextofthesentenceactsasthe LTR which uses CNN isalso integrated within the system. The Sequence to Sequence system for the dataset takes a significanttimeformodelingthesystem.Theexperiments indicated that the proposed NMT system achieves good performance with a better BLEU score compared to conventional MT systems and minimizes the TER significantly.
VII. FUTURE WORKS
The future work for this project lies in the module of machine translator, where more attention is needed to generate more accurate translation as in accordance with thehumantranslatedoutputratherthangoogletranslated output. This work will be related to multiple reference translationusingtheGANmodelsothatitachievesaclose match with human translation. These translation systems can be applied to many other applications like chatBot systems,Helpdeskapplicationsetc.
Thus we have translated sentences from German to EnglishwiththehelpofGoogleTranslationforthelimited sentence sizes. In the near future we are planning to extend this system for lang sentences along with multilingual translations from any to any known scripted languageswiththehelpofgoogletranslationpackage.
REFERENCES
[1] Bahdanau, Dzmitry, Kyunghyun Cho, and Yoshua Bengio. "Neural machine translation by jointly learning to align and translate." arXiv preprint arXiv:1409.0473(2014).
[2] Cao,Ziqiang,etal."Anovelneuraltopicmodelandits supervised extension." Twenty Ninth AAAI ConferenceonArtificialIntelligence.2015.
[3] Chen, Kehai, et al. "A neural approach to source dependence based context model for statistical machine translation." IEEE/ACM Transactions on Audio, Speech, and Language Processing 26.2 (2017): 266 280.
[4] Chen W., Matusov E., Khadivi S., and Peter J., “Guided alignment training for topic aware neural machine translation,” in The 38th Annual Meeting and Symposium of the Antenna Measurement Techniques Association,vol.1,Austin,Texas,November2016,pp. 121 134.
[5] Chitnis R. and DeNero J., “Variable length word encodings for neural translation models,” in Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, 2015, pp. 2088 2093.
[6] Chung J., Cho K., and Bengio Y., “A character level decoder without explicit segmentation for neural machine translation,” in Proceedings of the 54th Annual Meeting on Association for Computational Linguistics.Associationfor Computational Linguistics, 2016,pp.1693 1703.
[7] Fadaee M., Bisazza A., and Monz C., “Learning topic sensitivewordrepresentations,”inProceedingsofthe 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). Vancouver, Canada: Association for Computational Linguistics,Jul.2017,pp.441 447.
[8] Haipeng Sun,Rui Wang, Kehai Chen, Masao Utiyama, Eiichiro Sumita, and Tiejun Zhao."Unsupervised Neural Machine Translation With Cross Lingual Language Representation Agreement." IEEE/ACM Trans on audio,speech,and language processing,Vol. 28,2020.
[9] Jean S., Cho K., Memisevic R.,and Bengio Y.,“On Using Very Large Target vocabulary for neural machine translation,”inProc.53rdAnnu.Meet.Assoc.Comput. Linguistics 7th Int. Joint Conf. Natural Lang. Process., Jul.2015,pp.1 10.
[10] Kehai Chen, Rui Wang, Masao Utiyama, Eiichiro Sumita, and Tiejun Zhao."Neural Machine Translation with Sentence level Topic Context."IEEE/ACM Trans on audio,speech,and language processing, doi:10.1109/TASLP.2019.2937190.
[11] Kim Y., “Convolutional neural networks for sentence classification,” in Proceedings of the 2014 Conference onEmpiricalMethodsinNaturalLanguageProcessing (EMNLP). Doha, Qatar: Association for Computational Linguistics,Oct.2014,pp.1746 1751.
[12] MuyunYang,ShujieLiu,KehaiChen,HongyangZhang, EnboZhao,andTiejunZhao."AHierarchicalClustering Approach to Fuzzy Semantic Representation of Rare WordsinNeural MachineTranslation."IEEETrans.on fuzzylogic,Vol.28,no.5,May2020.
© 2022, IRJET | Impact Factor value: 7.529 | ISO 9001:2008 Certified Journal | Page3102
International Research Journal of Engineering and Technology (IRJET) e ISSN:2395 0056
Volume: 09 Issue: 04 | Apr 2022 www.irjet.net p ISSN:2395 0072
[13] Rui Wang, Masao Utiyama, Andrew Finch, Lemao Liu, Kehai Chen, and Eiichiro Sumita."Sentence Selection and Weighting for Neural Machine Translation Domain Adaptation."IEEE/ACM Trans. on audio, speech, and language processing, doi: 10.1109/TASLP.2018.2837223.
[14] Snover,Matthew,etal."Astudyoftranslationeditrate with targeted human annotation." Proceedings of association for machine translation in the Americas. Vol.200.No.6.2006.
[15] Xing Wang, Zhaopeng Tu, and Min Zhang" Incorporating Statistical Machine Translation Word Knowledge into Neural Machine Translation".Journal oflatexclassfiles,Vol.14,no.8,August2015.