吉永 直樹 / Naoki Yoshinaga, PhD

Associate Professor (Tenured) and The University of Tokyo Excellent Young Researcher

Institute of Industrial Science, the University of Tokyo
Department of Information and Communication Engineering, the University of Tokyo

Ee-503, 4-6-1 Komaba, Meguro-Ku, Tokyo, 153-8505, Japan
+81-3-5452-6239
ynaga (at) iis [guess from URL]

Curriculum Vitae

Full CV

Research Interests

We're studying various aspects on natural language processing (NLP) and computational linguistics (CL), especially

• Making the fastest NLP algorithmis even faster [EMNLP-09, ACL-10, COLING-10, COLING-14]
• NLP in the wild; robust NLP with non-textual factors [IJCNLP-13, ACL-13, ACL-17 SRW, NAACL-19, F. EMNLP-20]
• NLP for breaking language barriers [CoNLL-15, ACL-17, NAACL-19, CoNLL-19, F. EMNLP-20, NAACL-21]
• Mining from time-series social-media text [EMNLP-12, IJCAI-16, IJCAI-19]
• Information visualization for NLP [PacificVis-11, IUI-16, PacificVis-18]

I like to design novel and important NLP tasks [EMNLP-12, ACL-13, COLING-14, IJCAI-16, IJCAI-19], rather than solving classic tasks on worn-out datasets.

Research Grants

• Grant-in-Aid for Scientific Research (B) (21H03494; 2021-2024): 17,420,000 JPY (Principal Investigator)
• NII CRIS Collaborative Research operated by NII CRIS and LINE Corporation (2020-2022): 18,000,000 JPY (Principal Investigator)
• NII CRIS Contract Research 2019 (2019-2020): 2,499,000 JPY (Principal Investigator)
• Grant-in-Aid for Scientific Research (B) (16H02905; 2016-2019)
• U-Tokyo Excellent Young Researchers Start-up (2016-2018): 6,000,000 JPY (Principal Investigator)
• Grant-in-Aid for Young Scientists (B) (16K16109; 2016-2018): 3,900,000 JPY (Principal Investigator)
• IIS Sentei Kenkyuu (2016-2017): 2,000,000 JPY (Principal Investigator)
• Grant-in-Aid for JSPS Fellows (2005-2008): 3,400,000 JPY (Principal Investigator)
• Grant-in-Aid for JSPS Fellows (2002-2005): 3,000,000 JPY (Principal Investigator)

Softwares for NLP/CL

• opal: A scalable kernel-based online learner (polynomial kernel is supported).
• pecco: An efficient classifier for a model pre-trained with conjunctive features (or polynomial kernel).
• cedar: An efficient, updatable trie implementation based on double array.
• yakmo: A robust, efficient alternative k-means clustering.
• J.DepP: Very fast dependency parsers with the state-of-the-art accuracy for Japanese.
• RenTAL (no longer maintained): A grammar compiler from Lexicalized TAG to HPSG-style grammar.

Note that some of the above softwares are no longer experimental codes; they are substantially elaborated from the original codes to have a better performance (2-5x speed-up, or +1% in accuracy etc.; track History section of each software). Those who want to reproduce the experimental results may want to use the oldest release of the softwares.

Selected Publications (with links to presentations, codes, and datasets)

• Exploratory Model Analysis Using Data-Driven Neuron Representations
  BlackBoxNLP Workshop. Joint work with D. Oba and M. Toyoda
  TL;DR: We establish a methodology for exploratory, hypothesis-free model analysis of neural NLP models.
• Fine-grained Typing of Emerging Entities in Microblogs --- code & data (TBD)
  Findings of EMNLP-21 (long). Joint work with S. Akasaki and M. Toyoda
  TL;DR: We tackle a novel task of typing emerging entities in microblogs.
• Speculative Sampling in Variational Autoencoders for Dialogue Response Generation --- code & data (TBD)
  Findings of EMNLP-21 (short). Joint work with S. Shoetsu, M. Toyoda, and M. Kitsuregawa
  TL;DR: We redundantly pick latent variables in the training of variational models to obtain better latent space.
• Context-aware Decoder for Neural Machine Translation using a Target-side Document-Level Language Model -- code & data (TBD)
  NAACL-21 (long, acceptance rate 28%). Joint work with A. Sugiyama
  TL;DR: We utilize PMI computed by a document-level LM to perform context-aware decoding with a sentence-level NMT model.
• Robust Backed-off Estimation of Out-of-Vocabulary Embeddings --- code
  Findings of EMNLP-20 (long, acceptance rate 37%). Joint work with N. Fukuda and M. Kitsuregawa
  TL;DR: Inspired by two processes of creating words, we propose a simple word-based method to estimate OOV embeddings.
• Vocabulary Adaptation for Domain Adaptation in Neural Machine Translation --- code
  Findings of EMNLP-20 (long, acceptance rate 37%). Joint work with S. Sato, J. Sakuma, M. Toyoda, M. Kitsuregawa
  TL;DR: We transplant target-domain vocabularies to source-domain NMT model for effective fine tuning.
• uBLEU: Uncertainty-Aware Automatic Evaluation Method for Open-Domain Dialogue Systems --- slide / code
  ACL-20 SRW (acceptance rate 36%). Joint work with Y. Tsuta and M. Toyoda
  TL;DR: Our υBLEU exhibits higher correlation with human judgement than RUBER (AAAI-18).
• Multilingual model using cross-task embedding projection --- slide / code
  CoNLL-19 (oral, acceptance rate: 22%). Joint work with J. Sakuma
  TL;DR: Our locally-linear mapping optimizes multilingual models based on cross-lingual word embeddings to any tasks.
• On the Relation between Position Information and Sentence Length in Neural Machine Translation --- poster / code
  CoNLL-19 (acceptance rate: 22%). Joint work with M. Neishi
  TL;DR: We revealed that Transformer is bad at handling inputs of unseen lengths, and fixed it using RNN as relative position encoder.
• Early Discovery of Emerging Entities in Microblogs --- slide / poster / data
  IJCAI-19 (acceptance rate: 13.7%). Joint work with S. Akasaki and M. Toyoda
  TL;DR: We tackle a novel task of detecting emerging entities from Twitter timelines using timely distant supervision.
• Learning to Describe Unknown Phrases with Local and Global Contexts --- slide / code & data
  NAACL-19 (long, oral; acceptance rate 26%). Joint work with S. Ishiwatari, H. Hayashi, G. Neubig, S. Sato, M. Toyoda, and M. Kitsuregawa
  TL;DR: Our LOG-CaD can explain unknown phrases; evaluated on new Wikipedia datasets.
• Modeling Personal Biases in Language Use by Inducing Personalized Word Embeddings (journal ver.) --- slide
  NAACL-19 (short, oral; acceptance rate 21%). Joint work with D. Oba, S. Sato, S, Akasaki, and M. Toyoda
  TL;DR: We enable to model personalized usage of words, accompanied with analysis and applications.
• Information Integrated Visualization System for Heavy Rainfall Risk Analysis
  PacificViz-18 (poster). Joint work with M. Itoh, T. Sagara, U. Suzuki, K. Umemoto, M. Toyoda, K. Zettsu, and Y. Kidawara
  TL;DR: We integrate and visualize social sensor data to analyze heavy rainfall risks.
• A Bag of Useful Tricks for Practical Neural Machine Translation: Embedding Layer Initialization and Large Batch Size --- poster / code
  WAT-17 (oral). Joint work with M. Neishi, J. Sakuma, S. Tohda, S. Ishiwatari and Masashi Toyoda
  TL;DR: We confirmed the effectiveness of CBoW-based embedding layer initialization and Large Batch Size in NMT training.
• Modeling Situations in Neural Chat Bots --- poster
  ACL-17 SRW (acceptance rate 36%). Joint work with S. Sato, M. Toyoda and M. Kitsuregawa
  TL;DR: We enable to model and incorporate user profiles and time in open-domain dialogue systems.
• Chunk-based Decoder for Neural Machine Translation --- poster
  ACL-17 (long; acceptance rate 26%). Joint work with S. Ishiwatari, J. Yao, S. Liu, M. Li, M. Zhou, M. Kitsuregawa and W. Jia
  Inspired by phrase-based SMT, we propose chunk-based decoding in NMT.
• Ordering Concepts Based on Common Attribute Intensity --- slide / poster / code & data
  IJCAI-16 (acceptance rate <25%). Joint work with T. Iwanari, N. Kaji, T. Nishina, M. Toyoda and M. Kitsuregawa; a follow-up paper at COLING-16 (Demo) --- poster / software & data
  TL;DR: We induce your sense of values from your writings; novel task.
• Spatio-temporal Event Visualization from a Geo-parsed Microblog Stream --- poster / demo by Prof. Itoh
  IUI-16 (poster). Joint work with M. Itoh and M. Toyoda
  TL;DR: We visualize spatio-temporal tweets on word-clouds in the sky.
• Accurate Cross-lingual Projection between Count-based Word Vectors by Exploiting Translatable Context Pairs --- poster
  CoNLL-15 (short, acceptance rate <30%). Joint work with S. Ishiwatari, N. Kaji, M. Toyoda and M. Kitsuregawa
  TL;DR: We incorporate surface-based dimensional correspondences into count-based word vectors.
• A Self-adaptive Classifier for Efficient Text-stream Processing --- poster / software
  COLING-14 (acceptance rate 32%). Joint work with M. Kitsuregawa
  TL;DR: We propose a method of accelerating NLP classifiers when the processed text becomes redundant.
• Modeling User Leniency and Product Popularity for Sentiment Classification (journal ver.) --- poster
  IJCNLP-13. Joint work with W. Gao, N. Kaji, and M. Kitsuregawa
  TL;DR: We enable to model annotation and selection biases in sentiment analysis for unseen users and targets.
• Predicting and Eliciting Addressee's Emotion in Online Dialogue --- poster
  ACL-13 (long; acceptance rate 26%). Joint work with T. Hasegawa, N. Kaji, and M. Toyoda
  TL;DR: We enable to model emotions in corpus-based open-dialogue systems.
• Identifying Constant and Unique Relations by using Time-Series Text --- slide
  EMNLP-12 (oral; acceptance rate 17%). Joint work with Y. Takaku, N. Kaji, and M. Toyoda
  Use of massive Web in knowledge acquisition introduces enormous contradictions; solution provided.
• Analysis and Visualization of Temporal Changes in Bloggers' Activities and Interests --- demo / demo by Prof. Itoh
  PacificVis-12 (acceptance rate 34%). Joint work with M. Itoh, M. Toyoda, and M. Kitsuregawa
  TL;DR: We visualize dependencies in weblogs to understand the state of the world.
• Kernel Slicing: Scalable Online Training with Conjunctive Features --- slide / software & data
  COLING-10 (oral; acceptance rate 19%). Joint work with M. Kitsuregawa
  The kernel slicing generalizes kernel splitting (ACL 2008) to pack computations in online learning with polynomial kernel.
• Efficient Staggered Decoding for Sequence Labeling --- software
  ACL-10 (long; acceptance rate 25%). Joint work with N. Kaji, Y. Fujiwara, and M. Kitsuregawa
  TL;DR: We made structured prediction scalable to the number of classes.
• Polynomial to Linear: Efficient Classification with Conjunctive Features --- poster / software & data (journal ver.)
  EMNLP-09 (acceptance rate 34%). Joint work with M. Kitsuregawa
  TL;DR: We speed up testing of non-linear classifier with polynomial kernel.
• Boosting Precision and Recall of Hyponymy Relation Acquisition from Hierarchical Layouts in Wikipedia --- software
  LREC-08. Joint work with A. Sumida and K. Torisawa
  TL;DR: We developed a method of quickly obtaining a large-scale hyponymy relations from Wikipedia.
• Open-Domain Attribute-Value Acquisition from Semi-Structured Texts --- demo
  ISWC-07 workshop, OntoLex. Joint work with K. Torisawa; a follow-up paper on writing-support environment
  TL;DR: We developed a method of extracting attributes and their values for a given object using unsupervised wrapper induction.
• Improving the Accuracy of Subcategorizations Acquired from Corpora
  ACL-04 SRW (acceptance rate 28%)
• A Debug Tool for Practical Grammar Development
  ACL-03 (poster; acceptance rate 43%). Joint work with A. Yakushiji, K. Tateisi, Y. Miyao, and J. Tsujii
• Comparison between CFG Filtering Techniques for LTAG and HPSG (journal ver.)
  ACL-03 (poster; acceptance rate 43%). Joint work with Y. Miyao, K. Torisawa, and J. Tsujii
• Grammar Conversion from LTAG to HPSG (journal ver.) --- software
  ESSLLI 2011 Student Session (oral; acceptance rate 26%). Joint work with Y. Miyao.

More publications

Awards