Keisuke Sakaguchi

Contact Information

Tohoku University
6-6-05 Aramaki Aza Aoba, Aoba-ku, Sendai
Miyagi, 980-8579, Japan
URL: https://keisuke-sakaguchi.github.io
Email: keisuke.sakaguchi@tohoku.ac.jp
GitHub: https://github.com/keisks

Professional Experience

RIKEN Center for Advanced Intelligence Project Tokyo, Japan
Visiting Scientist 09/2022 -- present
Tohoku University Miyagi, Japan
Associate Professor 07/2022 -- present
Allen Institute for AI (AI2) Seattle, WA, USA
Research Scientist 08/2018 -- 06/2022
IBM T. J. Watson Research Center Yorktown Heights, NY, USA
Research Intern 05/2017 -- 08/2017
Educational Testing Service (ETS) Princeton, NJ, USA
Research Intern 05/2014 -- 08/2014
Nara Institute of Science and Technology (NAIST) Nara, Japan
JSPS Research Fellow (DC1) 04/2013 -- 08/2013
Microsoft Research Asia Beijing, China
Research Intern 08/2012 -- 11/2012
Yahoo Japan Corporation Tokyo, Japan
Research Intern 08/2011 -- 09/2011
IBM Japan Systems Engineering Co. Ltd. Chiba, Japan
IT Infrastructure Engineer 04/2008 -- 03/2011
RIKEN Brain Science Institute Saitama, Japan
Technical Assistant 12/2006 -- 03/2008

Education

Johns Hopkins University Baltimore, MD, USA
Ph.D. in Computer Science 08/2013 -- 08/2018
Advisor(s): Prof. Benjamin Van Durme, Matt Post
Nara Institute of Science and Technology (NAIST) Nara, Japan
M.E. in Computer and Information Science 04/2011 -- 03/2013
Advisor(s): Prof. Yuji Matsumoto, Mamoru Komachi
University of Essex Colchester, UK
M.A. in Psycholinguistics and Neurolinguistics (with distinction) 10/2005 -- 09/2006
Advisor(s): Prof. Harald Clahsen
Waseda University Tokyo, Japan
B.A. in Literature (major in Philosophy) 4/2001 -- 03/2005
Advisor(s): Marito Sato

Honors and Awards

(Student) Best Paper Award at AACL-IJCNLP Student Research Workshop 2022
AAAI Outstanding Paper (Single Best Paper) Award 2020
ACL Outstanding Paper Award 2017
NAIST Excellent Student Scholarship 2013
University of Essex: graduate with distinction in Psycho(&Neuro)linguistics 2006

Professional activities

Area Chair:

ARR 2023-current (ACL Rolling Review), EMNLP 2023 (Commonsense Reasoning), ACL 2023 (area: Semantics: Sentence-level Semantics, Textual Inference, and Other Areas), EACL 2023 (area: Language Resource and Evaluation), EMNLP 2022 (area: NLP Applications), StarSEM 2022 (area: Commonsense reasoning and natural language understanding)

Reviewer (Journals):

Transactions of the Association for Computational Linguistics (TACL) 2019-current. Computational Linguistics 2018. International Journal of Artificial Intelligence in Education (AIED) 2023

Reviewer (Conferences):

International Conference on Learning Representations (ICLR) 2024, Conference on Neural Information Processing Systems (NeurIPS) 2023, 2024. ACL Rolling Review (ARR) 2021, 2022, 2023. Annual Meeting of the Association for Computational Linguistics (ACL) 2018, 2019, 2020, 2021. Annual Meeting of the North American Chapter of the ACL (NAACL) 2018, 2019, 2021, 2022. Conference on Empirical Methods on Natural Language Processing (EMNLP) 2015, 2016, 2017, 2018, 2019, 2020, 2021. AAAI Conference on Artificial Intelligence 2020, 2021. International Joint Conferences on Artificial Intelligence (IJCAI) 2019, 2020, 2021. Conference on Computational Natural Language Learning (CoNLL) 2019, 2020. Conference of the European Chapter of the Association for Computational Linguistics (EACL) 2021, Asia-Pacific Chapter of the Association for Computational Linguistics (AACL) 2020. International Joint Conference on Natural Language Processin (IJCNLP) 2017. International Conference on Computational Linguistics (COLING) 2014, 2016, 2018, 2020. International Conference on Spoken Language Translation (IWSLT) 2020.

Reviewer (Workshops):

Methods for Optimizing and Evaluating Neural Language Generation (NeuralGen) 2019. Workshop on Innovative Use of NLP for Building Educational Applications (BEA) 2014, 2015, 2016, 2017, 2019, 2020. Workshop on Noisy User-generated Text (W-NUT) 2018, 2019 2020.

Organizer:

NII Shonan Meeting (2024) Empowering Local Open Source LLMs.


Invited Talks

RealTime QA: What's the Answer Right Now? 12/03/2023
2023 BrainLink X-Lab Day, Korean Federation of Science and Technology Societies (KOFST), Republic of Korea
Twist Decoding: Diverse Generators Guide Each Other 08/07/2023
CWRU-TOHOKU Joint Workshop, Sendai, Japan
Large Language Models: What is happening now? 04/05/2023
Sungkyunkwan University, Republic of Korea (online)
Large Language Models: What will happen next? 10/25/2022
2022 BrainLink X-Lab Day, Korean Federation of Science and Technology Societies (KOFST), Republic of Korea
Large Language Models: What will happen next? 09/27/2022
AIST AI Research Center, Japan
Robust Text Correction for Grammar and Fluency 09/27/2018
Microsoft Research, Redmond, WA
Robust Text Correction for Grammar and Fluency 03/30/2018
Allen Institute for Artificial Intelligence, Seattle, WA
Grammatical Error Correction with Neural Reinforcement Learning 02/22/2018
Grammarly Research, Online
Robust Text Correction for Grammar and Fluency 02/09/2018
Amazon Research, Boston, MA
Evaluating SMT evaluation 11/27/2013
Rakuten Institute of Technology, NY

Teaching

Natural Language Processing 2024 Fall
Tohoku University, Graduate School of Information Science
Information and Data Literacy 2024 Spring
Tohoku University, Graduate School of Information Science
Natural Language Processing 2023 Fall
Tohoku University, Graduate School of Information Science
Information and Data Literacy 2023 Spring
Tohoku University, Graduate School of Information Science
Introduction to Electrical, Information and Physics Engineering 2023 Spring
Tohoku University, School of Engineering
Natural Language Processing 2022 Fall
Tohoku University, Graduate School of Information Science

Publications

  1. A Multimodal Dialogue System to Lead Consensus Building with Emotion-Displaying
    Shinnosuk Nozue, Yuto Nakano, Shoji Moriya, Tomoki Ariyama, Kazuma Kokuta, Suchun Xie, Kai Sato, Shusaku Sone, Ryohei Kamei, Reina Akama, Yuichiroh Matsubayashi, and Keisuke Sakaguchi
    Proceedings of the 25th Annual Meeting of the Special Interest Group on Discourse and Dialogue. pp. 669–673, Sep 2024
  2. J-UniMorph: Japanese Morphological Annotation through the Universal Feature Schema
    Kosuke Matsuzaki, Masaya Taniguchi, Kentaro Inui, and Keisuke Sakaguchi
    Proceedings of the 21st SIGMORPHON workshop on Computational Research in Phonetics, Phonology, and Morphology. pp. 7–19, Jun 2024
  3. Beam Decoding with Controlled Patience
    Jungo Kasai, Keisuke Sakaguchi, Ronan Le Bras, Dragomir Radev, Yejin Choi, and Noah A. Smith
    Proceedings of 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation. May 2024
  4. PlaSma: Procedural Knowledge Models for Language-based Planning and Re-Planning
    Faeze Brahman, Chandra Bhagavatula, Valentina Pyatkin, Jena D. Hwang, Xiang Lorraine Li, Hirona Jacqueline Arai, Soumya Sanyal, Keisuke Sakaguchi, Xiang Ren, and Yejin Choi
    The Twelfth International Conference on Learning Representations. May 2024
  5. RealTime QA: What’s the Answer Right Now?
    Jungo Kasai, Keisuke Sakaguchi, Yoichi Takahashi, Ronan Le Bras, Akari Asai, Xinyan Velocity Yu, Dragomir Radev, Noah A. Smith, Yejin Choi, and Kentaro Inui
    Thirty-seventh Conference on Neural Information Processing Systems Datasets and Benchmarks Track. Dec 2023
  6. Evaluating GPT in Japanese Bar Examination: Insights and Limitations
    Jungmin Choi, Jungo Kasai, and Keisuke Sakaguchi
    Dec 2023
  7. Test-time Augmentation for Factual Probing
    Go Kamoda, Benjamin Heinzerling, Keisuke Sakaguchi, and Kentaro Inui
    Findings of the Association for Computational Linguistics: EMNLP 2023. pp. 3650–3661, Dec 2023
  8. I2D2: Inductive Knowledge Distillation with NeuroLogic and Self-Imitation
    Chandra Bhagavatula, Jena D Hwang, Doug Downey, Ronan Le Bras, Ximing Lu, Keisuke Sakaguchi, Swabha Swayamdipta, Peter West, and Yejin Choi
    Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). pp. 9614–9630, Jul 2023
  9. ELQA: A Corpus of Metalinguistic Questions and Answers about English
    Shabnam Behzad, Keisuke Sakaguchi, Nathan Schneider, and Amir Zeldes
    Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). pp. 2031–2047, Jul 2023
  10. Evaluating GPT-4 and ChatGPT on Japanese Medical Licensing Examinations
    Jungo Kasai, Yuhei Kasai, Keisuke Sakaguchi, Yutaro Yamada, and Dragomir Radev
    arXiv. 2023
  11. An Analysis of GPT-3’s Performance in Grammatical Error Correction
    Steven Coyne, and Keisuke Sakaguchi
    arXiv. 2023
  12. Causal schema induction for knowledge discovery
    Michael Regan, Jena D. Hwang, Keisuke Sakaguchi, and James Pustejovsky
    arXiv. 2023
  13. Do Deep Neural Networks Capture Compositionality in Arithmetic Reasoning?
    Keito Kudo, Yoichi Aoki, Tatsuki Kuribayashi, Ana Brassard, Masashi Yoshikawa, Keisuke Sakaguchi, and Kentaro Inui
    Proceedings of the 2023 Conference of the European Chapter of the Association for Computational Linguistics. pp. 1351–1362, May 2023
  14. Empirical Investigation of Neural Symbolic Reasoning Strategies
    Yoichi Aoki, Keito Kudo, Tatsuki Kuribayashi, Ana Brassard, Masashi Yoshikawa, Keisuke Sakaguchi, and Kentaro Inui
    Findings of the Association for Computational Linguistics: EACL 2023 . pp. 1154–1162, May 2023
    :trophy: Best Paper Award at AACL-IJCNLP 2022 Student Research Workshop
  15. Twist Decoding: Diverse Generators Guide Each Other
    Jungo Kasai, Keisuke Sakaguchi, Ronan Le Bras, Hao Peng, Ximing Lu, Dragomir Radev, Yejin Choi, and Noah A. Smith
    Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing (EMNLP). pp. 4909–4923, Dec 2022
  16. Can Machines Learn Morality? The Delphi Experiment
    Liwei Jiang, Jena D. Hwang, Chandra Bhagavatula, Ronan Le Bras, Jenny Liang, Jesse Dodge, Keisuke Sakaguchi, Maxwell Forbes, Jon Borchardt, Saadia Gabriel, Yulia Tsvetkov, Oren Etzioni, Maarten Sap, Regina Rini, and Yejin Choi
    arXiv. 2022
  17. Towards Automated Document Revision: Grammatical Error Correction, Fluency Edits, and Beyond
    Masato Mita, Keisuke Sakaguchi, Masato Hagiwara, Tomoya Mizumoto, Jun Suzuki, and Kentaro Inui
    arXiv. 2022
  18. Bidimensional Leaderboards: Generate and Evaluate Language Hand in Hand
    Jungo Kasai, Keisuke Sakaguchi, Ronan Le Bras, Lavinia Dunagan, Jacob Morrison, Alexander R. Fabbri, Yejin Choi, and Noah A. Smith
    Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. pp. 3540–3557, Jul 2022
  19. Transparent Human Evaluation for Image Captioning
    Jungo Kasai, Keisuke Sakaguchi, Lavinia Dunagan, Jacob Morrison, Ronan Le Bras, Yejin Choi, and Noah A. Smith
    Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. pp. 3464–3478, Jul 2022
  20. Interscript: A dataset for interactive learning of scripts through error feedback
    Niket Tandon, Aman Madaan, Peter Clark, Keisuke Sakaguchi, and Yiming Yang
    The AAAI-22 Workshop on Interactive Machine Learning. 2022
  21. Improving Neural Model Performance through Natural Language Feedback on Their Explanations
    Aman Madaan, Niket Tandon, Dheeraj Rajagopal, Yiming Yang, Peter Clark, Keisuke Sakaguchi, and Eduard H. Hovy
    arXiv. 2021
  22. GrammarTagger: A Multilingual, Minimally-Supervised Grammar Profiler for Language Education
    Masato Hagiwara, Joshua Tanner, and Keisuke Sakaguchi
    arXiv. 2021
  23. proScript: Partially Ordered Scripts Generation
    Keisuke Sakaguchi, Chandra Bhagavatula, Ronan Le Bras, Niket Tandon, Peter Clark, and Yejin Choi
    Findings of the Association for Computational Linguistics: EMNLP 2021. pp. 2138–2149, Nov 2021
  24. WinoGrande: An Adversarial Winograd Schema Challenge at Scale
    Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi
    Commun. ACM. pp. 99–106, Aug 2021
  25. COMET-ATOMIC 2020: On Symbolic and Neural Commonsense Knowledge Graphs
    Jena D. Hwang, Chandra Bhagavatula, Ronan Le Bras, Jeff Da, Keisuke Sakaguchi, Antoine Bosselut, and Yejin Choi
    Proceedings of the AAAI Conference on Artificial Intelligence. pp. 6384–6392, May 2021
  26. A Dataset for Tracking Entities in Open Domain Procedural Text
    Niket Tandon, Keisuke Sakaguchi, Bhavana Dalvi, Dheeraj Rajagopal, Peter Clark, Michal Guerquin, Kyle Richardson, and Eduard Hovy
    Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). pp. 6408–6417, Nov 2020
  27. Uncertain Natural Language Inference
    Tongfei Chen, Zhengping Jiang, Adam Poliak, Keisuke Sakaguchi, and Benjamin Van Durme
    Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. pp. 8772–8779, Jul 2020
  28. The Universal Decompositional Semantics Dataset and Decomp Toolkit
    Aaron Steven White, Elias Stengel-Eskin, Siddharth Vashishtha, Venkata Subrahmanyan Govindarajan, Dee Ann Reisinger, Tim Vieira, Keisuke Sakaguchi, Sheng Zhang, Francis Ferraro, Rachel Rudinger, Kyle Rawlins, and Benjamin Van Durme
    Proceedings of the 12th Language Resources and Evaluation Conference. pp. 5698–5707, May 2020
  29. Abductive Commonsense Reasoning
    Chandra Bhagavatula, Ronan Le Bras, Chaitanya Malaviya, Keisuke Sakaguchi, Ari Holtzman, Hannah Rashkin, Doug Downey, Wen-tau Yih, and Yejin Choi
    International Conference on Learning Representations. 2020
  30. WinoGrande: An Adversarial Winograd Schema Challenge at Scale
    Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi
    Proceedings of the AAAI Conference on Artificial Intelligence. pp. 8732–8740, Apr 2020
    :trophy: Outstanding Paper Award (Single Best Paper)
  31. WIQA: A dataset for “What if...” reasoning over procedural text
    Niket Tandon, Bhavana Dalvi, Keisuke Sakaguchi, Peter Clark, and Antoine Bosselut
    Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). pp. 6076–6085, Nov 2019
  32. Efficient Online Scalar Annotation with Bounded Support
    Keisuke Sakaguchi, and Benjamin Van Durme
    Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). pp. 208–218, Jul 2018
  33. Grammatical Error Correction with Neural Reinforcement Learning
    Keisuke Sakaguchi, Matt Post, and Benjamin Van Durme
    Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 2: Short Papers). pp. 366–372, Nov 2017
  34. GEC into the future: Where are we going and how do we get there?
    Keisuke Sakaguchi, Courtney Napoles, and Joel Tetreault
    Proceedings of the 12th Workshop on Innovative Use of NLP for Building Educational Applications. pp. 180–187, Sep 2017
  35. Error-repair Dependency Parsing for Ungrammatical Texts
    Keisuke Sakaguchi, Matt Post, and Benjamin Van Durme
    Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). pp. 189–195, Jul 2017
    :trophy: Outstanding Paper Award (1.5% of the submissions)
  36. JFLEG: A Fluency Corpus and Benchmark for Grammatical Error Correction
    Courtney Napoles, Keisuke Sakaguchi, and Joel Tetreault
    Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers. pp. 229–234, Apr 2017
  37. Robsut Wrod Reocginiton via Semi-Character Recurrent Neural Network
    Keisuke Sakaguchi, Kevin Duh, Matt Post, and Benjamin Van Durme
    Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence. pp. 3281–3287, 2017
  38. Universal Decompositional Semantics on Universal Dependencies
    Aaron Steven White, Drew Reisinger, Keisuke Sakaguchi, Tim Vieira, Sheng Zhang, Rachel Rudinger, Kyle Rawlins, and Benjamin Van Durme
    Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. pp. 1713–1723, Nov 2016
  39. There’s No Comparison: Reference-less Evaluation Metrics in Grammatical Error Correction
    Courtney Napoles, Keisuke Sakaguchi, and Joel Tetreault
    Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. pp. 2109–2115, Nov 2016
  40. Phrase Structure Annotation and Parsing for Learner English
    Ryo Nagata, and Keisuke Sakaguchi
    Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). pp. 1837–1847, Aug 2016
  41. Reassessing the Goals of Grammatical Error Correction: Fluency Instead of Grammaticality
    Keisuke Sakaguchi, Courtney Napoles, Matt Post, and Joel Tetreault
    Transactions of the Association for Computational Linguistics. pp. 169–182, 2016
  42. GLEU Without Tuning
    Courtney Napoles, Keisuke Sakaguchi, Matt Post, and Joel R. Tetreault
    arXiv. 2016
  43. Ground Truth for Grammatical Error Correction Metrics
    Courtney Napoles, Keisuke Sakaguchi, Matt Post, and Joel Tetreault
    Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 2: Short Papers). pp. 588–593, Jul 2015
  44. Effective Feature Integration for Automated Short Answer Scoring
    Keisuke Sakaguchi, Michael Heilman, and Nitin Madnani
    Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. pp. 1049–1054, May 2015
  45. Efficient Elicitation of Annotations for Human Evaluation of Machine Translation
    Keisuke Sakaguchi, Matt Post, and Benjamin Van Durme
    Proceedings of the Ninth Workshop on Statistical Machine Translation. pp. 1–11, Jun 2014
  46. Discriminative Approach to Fill-in-the-Blank Quiz Generation for Language Learners
    Keisuke Sakaguchi, Yuki Arase, and Mamoru Komachi
    Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). pp. 238–242, Aug 2013
  47. NAIST at 2013 CoNLL Grammatical Error Correction Shared Task
    Ippei Yoshimoto, Tomoya Kose, Kensuke Mitsuzawa, Keisuke Sakaguchi, Tomoya Mizumoto, Yuta Hayashibe, Mamoru Komachi, and Yuji Matsumoto
    Proceedings of the Seventeenth Conference on Computational Natural Language Learning: Shared Task. pp. 26–33, Aug 2013
  48. NAIST at the NLI 2013 Shared Task
    Tomoya Mizumoto, Yuta Hayashibe, Keisuke Sakaguchi, Mamoru Komachi, and Yuji Matsumoto
    Proceedings of the Eighth Workshop on Innovative Use of NLP for Building Educational Applications. pp. 134–139, Jun 2013
  49. Construction of English MWE Dictionary and its Application to POS Tagging
    Yutaro Shigeto, Ai Azuma, Sorami Hisamoto, Shuhei Kondo, Tomoya Kose, Keisuke Sakaguchi, Akifumi Yoshimoto, Frances Yung, and Yuji Matsumoto
    Proceedings of the 9th Workshop on Multiword Expressions. pp. 139–144, Jun 2013
  50. Joint English Spelling Error Correction and POS Tagging for Language Learners Writing
    Keisuke Sakaguchi, Tomoya Mizumoto, Mamoru Komachi, and Yuji Matsumoto
    Proceedings of COLING 2012. pp. 2357–2374, Dec 2012
  51. NAIST at the HOO 2012 Shared Task
    Keisuke Sakaguchi, Yuta Hayashibe, Shuhei Kondo, Lis Kanashiro, Tomoya Mizumoto, Mamoru Komachi, and Yuji Matsumoto
    Proceedings of the Seventh Workshop on Building Educational Applications Using NLP. pp. 281–288, Jun 2012