Nikolay Bogoychev

About me

I am a Postdoc at The University of Edinburgh Institute for Languages, Cognition and Computation, with Kenneth Heafield. I completed my PhD in the same institute with Adam Lopez and Kenneth Heafield.

I like languages, logographic writing systems, game theory and GPUs. I (try to) make things run faster and enjoy (premature) optimization. In my spare time I learn languages and play football. Please check out my blog where I post random things about Chinese characters, code and life.

LinkedIN Github StackOverflow Google Scholar CV

Research

My research lies at the intersection of high performance computing and NLP. I am interested in GPGPU, low level CPU GEMM routines, NLP, in particular Neural Machine Translation, transformers, word embeddings, transfer learning and multilinguality. Lately, I have been working on optimising Neural Machine Translation inference, both on CPUs and GPUs.

Projects

I am an avid supporter and believer in Open Source software and have contributed to various projects, among which:

Consulting

I have consulted for the following companies:

Education

2019: Received a PhD from The University of Edinburgh ILCC.
Thesis: Fast machine translation on parallel and massively parallel hardware [pdf] [bib]

2014: Graduated from The University of Edinburgh with a First class Bachelor's degree in Artificial Intelligence & Computer science.

Publications

2021

Nikolay Bogoychev Not all parameters are born equal: Attention is mostly what you need In Proceedings of the Fourth BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP, Punta Cana, Dominican Republic [pdf] [bib] [blog] [poster]

Nikolay Bogoychev and Pinzhen Chen The Highs and Lows of Simple Lexical Domain Adaptation Approaches for Neural Machine Translation In Proceedings of the Second Workshop on Insights from Negative Results in NLP, Punta Cana, Dominican Republic [pdf] [bib] [blog] [presentation]

Nikolay Bogoychev, Jelmer Van der Linde, Kenneth Heafield TranslateLocally: Blazing-fast translation running on the local CPU Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, Punta Cana, Dominican Republic [pdf] [bib] [blog] [demo]

2020

Pinzhen Chen, Nikolay Bogoychev, Ulrich Germann Character Mapping and Ad-hoc Adaptation: Edinburgh’s IWSLT 2020 Open Domain Translation System. In Proceedings of the 17th International Conference on Spoken Language Translation, Seattle, USA [pdf] [bib]

Nikolay Bogoychev, Roman Grundkiewicz, Alham Fikri Aji, Maximiliana Behnke, Kenneth Heafield, Sidharth Kashyap, Emmanouil-Ioannis Farsarakis, Mateusz Chudyk Edinburgh’s Submissions to the 2020 Machine Translation Efficiency Task. In Proceedings of the Fourth Workshop on Neural Generation and Translation, Seattle, USA [pdf] [bib]

Pinzhen Chen, Nikolay Bogoychev, Kenneth Heafield, Faheem Kirefu Parallel Sentence Mining by Constrained Decoding. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Seattle, USA [pdf] [bib]

Alham Fikri Aji, Nikolay Bogoychev, Kenneth Heafield In Neural Machine Translation, What Does Transfer Learning Transfer? In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Seattle, USA [pdf] [bib]

2019

Nikolay Bogoychev and Rico Sennrich Domain, Translationese and Noise in Synthetic Data for Neural Machine Translation. arxiv preprint. [pdf] [bib]

Young Jin Kim, Marcin Junczys-Dowmunt, Hany Hassan, Alham Fikri Aji, Kenneth Heafield, Roman Grundkiewicz, Nikolay Bogoychev From Research to Production and Back: Ludicrously Fast Neural Machine Translation. In Proceedings of the 3rd Workshop on Neural Generation and Translation, Hong Kong. [pdf] [bib]

Alham Fikri Aji, Kenneth Heafield, Nikolay Bogoychev Combining Global Sparse Gradients with Local Gradients in Distributed Neural Network Training. In Proceedings of EMNLP, Hong Kong. [pdf] [bib]

Rachel Bawden, Nikolay Bogoychev, Ulrich Germann, Roman Grundkiewicz, Faheem Kirefu, Antonio Valerio Miceli Barone, Alexandra Birch The University of Edinburgh's Submissions to the WMT19 News Translation Task. In Proceedings of the Fourth Conference on Machine Translation, Florence, Italy [pdf] [bib]

Nikolay Bogoychev Fast machine translation on parallel and massively parallel hardware. PhD Dissertation, Edinburgh, United Kingdom. [pdf] [bib]

Lushi Chen, Abeer Aldayel, Nikolay Bogoychev, Tao Gong Similar Minds Post Alike: Assessment of Suicide Risk Using a Hybrid Model. In Proceedings of the Sixth Workshop on Computational Linguistics and Clinical Psychology, Minneapolis, Minnesota, USA. [pdf] [bib]

2018

Nikolay Bogoychev, Marcin Junczys-Dowmunt, Kenneth Heafield, Alham Fikri Aji Accelerating Asynchronous Stochastic Gradient Descent for Neural Machine Translation. In Proceedings of EMNLP, Brussels, Belgium. [pdf] [bib]

Barry Haddow, Nikolay Bogoychev, Denis Emelin, Ulrich Germann, Roman Grundkiewicz, Kenneth Heafield, Antonio Valerio Miceli Barone, Rico Sennrich The University of Edinburghs Submissions to the WMT18 News Translation Task In Proceedings of the Third Conference on Machine Translation, Brussels, Belgium. [pdf] [bib]

Marcin Junczys-Dowmunt, Roman Grundkiewicz, Tomasz Dwojak, Hieu Hoang, Kenneth Heafield, Tom Neckermann, Frank Seide, Ulrich Germann, Alham Fikri Aji, Nikolay Bogoychev, André F. T. Martins, Alexandra Birch (2018). Marian: Fast Neural Machine Translation in C++. Proceedings of ACL, Sydney, Australia, System Demonstrations. [pdf] [bib]

2016

Nikolay Bogoychev and Adam Lopez N-gram language models for massively parallel devices. In Proceedings of ACL, Berlin, Germany. [pdf] [bib]

Nikolay Bogoychev and Hieu Hoang Fast and highly parallelizable phrase table for statistical machine translation. In Proceedings of WMT, Berlin, Germany. [pdf] [bib]

Hieu Hoang, Nikolay Bogoychev, Lane Schwartz and Marcin Junczys-Dowmunt Fast, Scalable Phrase-Based SMT Decoding. In Proceedings of AMTA 2016, Austin, Texas [pdf] [bib]

2015

Barry Haddow, Matthias Huck, Alexandra Birch, Nikolay Bogoychev, Philipp Koehn The Edinburgh/JHU Phrase-based Machine Translation Systems for WMT 2015. In Proceedings of WMT, Lisboa, Portugal. [pdf] [bib]

2014

Alexandra Birch, Matthias Huck, Nadir Durrani, Nikolay Bogoychev, Philipp Koehn Edinburgh SLT and MT System Description for the IWSLT 2014 Evaluation. In Proceedings of IWSLT, Lake Tahoe, USA. [pdf] [bib]

The University of Edinburgh
School of Informatics
Institute for Languages, Cognition and Computation