publications

See also Google Scholar, Semantic Scholar.

2025
  1. Nature MI
    Investigating machine moral judgement through the Delphi experiment
    Liwei Jiang, Jena D. Hwang, Chandra Bhagavatula, Ronan Le Bras, Jenny T. Liang, Sydney Levine, Jesse Dodge, Keisuke Sakaguchi, Maxwell Forbes, Jack Hessel, Jon Borchardt, Taylor Sorensen, Saadia Gabriel, Yulia Tsvetkov, Oren Etzioni, Maarten Sap, Regina Rini, Yejin Choi
    Nature Machine Intelligence 2025
  2. arXiv
    Weight-based Analysis of Detokenization in Language Models: Understanding the First Stage of Inference Without Inference
    Go Kamoda, Benjamin Heinzerling, Tatsuro Inaba, Keito Kudo, Keisuke Sakaguchi, Kentaro Inui
    arXiv 2025
  3. arXiv
    FinchGPT: a Transformer based language model for birdsong analysis
    Kosei Kobayashi, Kosuke Matsuzaki, Masaya Taniguchi, Keisuke Sakaguchi, Kentaro Inui, Kentaro Abe
    arXiv 2025
    2024
    1. EMNLP
      First Heuristic Then Rational: Dynamic Use of Heuristics in Language Model Reasoning
      Yoichi Aoki, Keito Kudo, Tatsuki Kuribayashi, Shusaku Sone, Masaya Taniguchi, Keisuke Sakaguchi, Kentaro Inui
      Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing Nov 2024
    2. COLM
      ACORN: Aspect-wise Commonsense Reasoning Explanation Evaluation
      Ana Brassard, Benjamin Heinzerling, Keito Kudo, Keisuke Sakaguchi, Kentaro Inui
      First Conference on Language Modeling 2024
    3. arXiv
      Think-to-Talk or Talk-to-Think? When LLMs Come Up with an Answer in Multi-Step Reasoning
      Keito Kudo, Yoichi Aoki, Tatsuki Kuribayashi, Shusaku Sone, Masaya Taniguchi, Ana Brassard, Keisuke Sakaguchi, Kentaro Inui
      arXiv Dec 2024
    4. arXiv
      Self-Training Meets Consistency: Improving LLMs’ Reasoning With Consistency-Driven Rationale Evaluation
      Jaehyeok Lee, Keisuke Sakaguchi, JinYeong Bak
      arXiv Nov 2024
    5. BEA
      Towards Automated Document Revision: Grammatical Error Correction, Fluency Edits, and Beyond
      Masato Mita, Keisuke Sakaguchi, Masato Hagiwara, Tomoya Mizumoto, Jun Suzuki, Kentaro Inui
      Proceedings of the 19th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2024) Jun 2024
    6. SigDial
      A Multimodal Dialogue System to Lead Consensus Building with Emotion-Displaying
      Shinnosuk Nozue, Yuto Nakano, Shoji Moriya, Tomoki Ariyama, Kazuma Kokuta, Suchun Xie, Kai Sato, Shusaku Sone, Ryohei Kamei, Reina Akama, Yuichiroh Matsubayashi, Keisuke Sakaguchi
      Proceedings of the 25th Annual Meeting of the Special Interest Group on Discourse and Dialogue Sep 2024
    7. arXiv
      Empirical Analysis of Large Vision-Language Models against Goal Hijacking via Visual Prompt Injection
      Subaru Kimura, Ryota Tanaka, Shumpei Miyawaki, Jun Suzuki, Keisuke Sakaguchi
      arXiv Aug 2024
    8. arXiv
      LLM-jp: A Cross-organizational Project for the Research and Development of Fully Open Japanese LLMs
      LLM-jp, :, Akiko Aizawa, Eiji Aramaki, Bowen Chen, Fei Cheng, Hiroyuki Deguchi, Rintaro Enomoto, Kazuki Fujii, Kensuke Fukumoto, Takuya Fukushima, Namgi Han, Yuto Harada, Chikara Hashimoto, Tatsuya Hiraoka, Shohei Hisada, Sosuke Hosokawa, Lu Jie, Keisuke Kamata, Teruhito Kanazawa, Hiroki Kanezashi, Hiroshi Kataoka, Satoru Katsumata, Daisuke Kawahara, Seiya Kawano, Atsushi Keyaki, Keisuke Kiryu, Hirokazu Kiyomaru, Takashi Kodama, Takahiro Kubo, Yohei Kuga, Ryoma Kumon, Shuhei Kurita, Sadao Kurohashi, Conglong Li, Taiki Maekawa, Hiroshi Matsuda, Yusuke Miyao, Kentaro Mizuki, Sakae Mizuki, Yugo Murawaki, Akim Mousterou, Ryo Nakamura, Taishi Nakamura, Kouta Nakayama, Tomoka Nakazato, Takuro Niitsuma, Jiro Nishitoba, Yusuke Oda, Hayato Ogawa, Takumi Okamoto, Naoaki Okazaki, Yohei Oseki, Shintaro Ozaki, Koki Ryu, Rafal Rzepka, Keisuke Sakaguchi, Shota Sasaki, Satoshi Sekine, Kohei Suda, Saku Sugawara, Issa Sugiura, Hiroaki Sugiyama, Hisami Suzuki, Jun Suzuki, Toyotaro Suzumura, Kensuke Tachibana, Yu Takagi, Kyosuke Takami, Koichi Takeda, Masashi Takeshita, Masahiro Tanaka, Kenjiro Taura, Arseny Tolmachev, Nobuhiro Ueda, Zhen Wan, Shuntaro Yada, Sakiko Yahata, Yuya Yamamoto, Yusuke Yamauchi, Hitomi Yanaka, Rio Yokota, Koichiro Yoshino
      arXiv e-prints Jul 2024
    9. arXiv
      The Curse of Popularity: Popular Entities have Catastrophic Side Effects when Deleting Knowledge from Language Models
      Ryosuke Takahashi, Go Kamoda, Benjamin Heinzerling, Keisuke Sakaguchi, Kentaro Inui
      arXiv Jun 2024
    10. MORPHON
      J-UniMorph: Japanese Morphological Annotation through the Universal Feature Schema
      Kosuke Matsuzaki, Masaya Taniguchi, Kentaro Inui, Keisuke Sakaguchi
      Proceedings of the 21st SIGMORPHON workshop on Computational Research in Phonetics, Phonology, and Morphology Jun 2024
    11. LREC-COLING
      Beam Decoding with Controlled Patience
      Jungo Kasai, Keisuke Sakaguchi, Ronan Le Bras, Dragomir Radev, Yejin Choi, Noah A. Smith
      Proceedings of 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation May 2024
    12. ICLR
      PlaSma: Procedural Knowledge Models for Language-based Planning and Re-Planning
      Faeze Brahman, Chandra Bhagavatula, Valentina Pyatkin, Jena D. Hwang, Xiang Lorraine Li, Hirona Jacqueline Arai, Soumya Sanyal, Keisuke Sakaguchi, Xiang Ren, Yejin Choi
      The Twelfth International Conference on Learning Representations May 2024
      2023
      1. NeurIPS
        RealTime QA: What’s the Answer Right Now?
        Jungo Kasai, Keisuke Sakaguchi, Yoichi Takahashi, Ronan Le Bras, Akari Asai, Xinyan Velocity Yu, Dragomir Radev, Noah A. Smith, Yejin Choi, Kentaro Inui
        Thirty-seventh Conference on Neural Information Processing Systems Datasets and Benchmarks Track Dec 2023
      2. EMNLP Findings
        Test-time Augmentation for Factual Probing
        Go Kamoda, Benjamin Heinzerling, Keisuke Sakaguchi, Kentaro Inui
        Findings of the Association for Computational Linguistics: EMNLP 2023 Dec 2023
      3. ACL
        I2D2: Inductive Knowledge Distillation with NeuroLogic and Self-Imitation
        Chandra Bhagavatula, Jena D Hwang, Doug Downey, Ronan Le Bras, Ximing Lu, Keisuke Sakaguchi, Swabha Swayamdipta, Peter West, Yejin Choi
        Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) Jul 2023
      4. ACL
        ELQA: A Corpus of Metalinguistic Questions and Answers about English
        Shabnam Behzad, Keisuke Sakaguchi, Nathan Schneider, Amir Zeldes
        Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) Jul 2023
      5. arXiv
        Evaluating GPT-4 and ChatGPT on Japanese Medical Licensing Examinations
        Jungo Kasai, Yuhei Kasai, Keisuke Sakaguchi, Yutaro Yamada, Dragomir Radev
        arXiv 2023
      6. arXiv
        An Analysis of GPT-3’s Performance in Grammatical Error Correction
        Steven Coyne, Keisuke Sakaguchi
        arXiv 2023
      7. arXiv
        Causal schema induction for knowledge discovery
        Michael Regan, Jena D. Hwang, Keisuke Sakaguchi, James Pustejovsky
        arXiv 2023
      8. EACL
        Do Deep Neural Networks Capture Compositionality in Arithmetic Reasoning?
        Keito Kudo, Yoichi Aoki, Tatsuki Kuribayashi, Ana Brassard, Masashi Yoshikawa, Keisuke Sakaguchi, Kentaro Inui
        Proceedings of the 2023 Conference of the European Chapter of the Association for Computational Linguistics May 2023
      9. EACL Findings
        Empirical Investigation of Neural Symbolic Reasoning Strategies
        Yoichi Aoki, Keito Kudo, Tatsuki Kuribayashi, Ana Brassard, Masashi Yoshikawa, Keisuke Sakaguchi, Kentaro Inui
        Findings of the Association for Computational Linguistics: EACL 2023 May 2023
      1. Jxiv
        Evaluating GPT in Japanese Bar Examination: Insights and Limitations
        Jungmin Choi, Jungo Kasai, Keisuke Sakaguchi
        Dec 2023
      2022
      1. EMNLP
        Twist Decoding: Diverse Generators Guide Each Other
        Jungo Kasai, Keisuke Sakaguchi, Ronan Le Bras, Hao Peng, Ximing Lu, Dragomir Radev, Yejin Choi, Noah A. Smith
        Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing (EMNLP) Dec 2022
      2. arXiv
        Can Machines Learn Morality? The Delphi Experiment
        Liwei Jiang, Jena D. Hwang, Chandra Bhagavatula, Ronan Le Bras, Jenny Liang, Jesse Dodge, Keisuke Sakaguchi, Maxwell Forbes, Jon Borchardt, Saadia Gabriel, Yulia Tsvetkov, Oren Etzioni, Maarten Sap, Regina Rini, Yejin Choi
        arXiv 2022
      3. NAACL
        Bidimensional Leaderboards: Generate and Evaluate Language Hand in Hand
        Jungo Kasai, Keisuke Sakaguchi, Ronan Le Bras, Lavinia Dunagan, Jacob Morrison, Alexander R. Fabbri, Yejin Choi, Noah A. Smith
        Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies Jul 2022
      4. NAACL
        Transparent Human Evaluation for Image Captioning
        Jungo Kasai, Keisuke Sakaguchi, Lavinia Dunagan, Jacob Morrison, Ronan Le Bras, Yejin Choi, Noah A. Smith
        Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies Jul 2022
      5. IMLW@AAAI
        Interscript: A dataset for interactive learning of scripts through error feedback
        Niket Tandon, Aman Madaan, Peter Clark, Keisuke Sakaguchi, Yiming Yang
        The AAAI-22 Workshop on Interactive Machine Learning 2022
        2021
        1. arXiv
          Improving Neural Model Performance through Natural Language Feedback on Their Explanations
          Aman Madaan, Niket Tandon, Dheeraj Rajagopal, Yiming Yang, Peter Clark, Keisuke Sakaguchi, Eduard H. Hovy
          arXiv 2021
        2. arXiv
          GrammarTagger: A Multilingual, Minimally-Supervised Grammar Profiler for Language Education
          Masato Hagiwara, Joshua Tanner, Keisuke Sakaguchi
          arXiv 2021
        3. EMNLP Findings
          proScript: Partially Ordered Scripts Generation
          Keisuke Sakaguchi, Chandra Bhagavatula, Ronan Le Bras, Niket Tandon, Peter Clark, Yejin Choi
          Findings of the Association for Computational Linguistics: EMNLP 2021 Nov 2021
        4. CACM
          WinoGrande: An Adversarial Winograd Schema Challenge at Scale
          Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, Yejin Choi
          Commun. ACM Aug 2021
        5. AAAI
          COMET-ATOMIC 2020: On Symbolic and Neural Commonsense Knowledge Graphs
          Jena D. Hwang, Chandra Bhagavatula, Ronan Le Bras, Jeff Da, Keisuke Sakaguchi, Antoine Bosselut, Yejin Choi
          Proceedings of the AAAI Conference on Artificial Intelligence May 2021
          2020
          1. EMNLP
            A Dataset for Tracking Entities in Open Domain Procedural Text
            Niket Tandon, Keisuke Sakaguchi, Bhavana Dalvi, Dheeraj Rajagopal, Peter Clark, Michal Guerquin, Kyle Richardson, Eduard Hovy
            Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) Nov 2020
          2. ACL
            Uncertain Natural Language Inference
            Tongfei Chen, Zhengping Jiang, Adam Poliak, Keisuke Sakaguchi, Benjamin Van Durme
            Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics Jul 2020
          3. LREC
            The Universal Decompositional Semantics Dataset and Decomp Toolkit
            Aaron Steven White, Elias Stengel-Eskin, Siddharth Vashishtha, Venkata Subrahmanyan Govindarajan, Dee Ann Reisinger, Tim Vieira, Keisuke Sakaguchi, Sheng Zhang, Francis Ferraro, Rachel Rudinger, Kyle Rawlins, Benjamin Van Durme
            Proceedings of the 12th Language Resources and Evaluation Conference May 2020
          4. ICLR
            Abductive Commonsense Reasoning
            Chandra Bhagavatula, Ronan Le Bras, Chaitanya Malaviya, Keisuke Sakaguchi, Ari Holtzman, Hannah Rashkin, Doug Downey, Wen-tau Yih, Yejin Choi
            International Conference on Learning Representations 2020
          5. AAAI
            WinoGrande: An Adversarial Winograd Schema Challenge at Scale
            Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, Yejin Choi
            Proceedings of the AAAI Conference on Artificial Intelligence Apr 2020
            2019
            1. EMNLP
              WIQA: A dataset for “What if...” reasoning over procedural text
              Niket Tandon, Bhavana Dalvi, Keisuke Sakaguchi, Peter Clark, Antoine Bosselut
              Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) Nov 2019
              2018
              1. ACL
                Efficient Online Scalar Annotation with Bounded Support
                Keisuke Sakaguchi, Benjamin Van Durme
                Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) Jul 2018
                2017
                1. IJCNLP
                  Grammatical Error Correction with Neural Reinforcement Learning
                  Keisuke Sakaguchi, Matt Post, Benjamin Van Durme
                  Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 2: Short Papers) Nov 2017
                2. BEA
                  GEC into the future: Where are we going and how do we get there?
                  Keisuke Sakaguchi, Courtney Napoles, Joel Tetreault
                  Proceedings of the 12th Workshop on Innovative Use of NLP for Building Educational Applications Sep 2017
                3. ACL
                  Error-repair Dependency Parsing for Ungrammatical Texts
                  Keisuke Sakaguchi, Matt Post, Benjamin Van Durme
                  Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) Jul 2017
                4. EACL
                  JFLEG: A Fluency Corpus and Benchmark for Grammatical Error Correction
                  Courtney Napoles, Keisuke Sakaguchi, Joel Tetreault
                  Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers Apr 2017
                5. AAAI
                  Robsut Wrod Reocginiton via Semi-Character Recurrent Neural Network
                  Keisuke Sakaguchi, Kevin Duh, Matt Post, Benjamin Van Durme
                  Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence 2017
                  2016
                  1. EMNLP
                    Universal Decompositional Semantics on Universal Dependencies
                    Aaron Steven White, Drew Reisinger, Keisuke Sakaguchi, Tim Vieira, Sheng Zhang, Rachel Rudinger, Kyle Rawlins, Benjamin Van Durme
                    Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing Nov 2016
                  2. EMNLP
                    There’s No Comparison: Reference-less Evaluation Metrics in Grammatical Error Correction
                    Courtney Napoles, Keisuke Sakaguchi, Joel Tetreault
                    Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing Nov 2016
                  3. ACL
                    Phrase Structure Annotation and Parsing for Learner English
                    Ryo Nagata, Keisuke Sakaguchi
                    Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) Aug 2016
                  4. TACL
                    Reassessing the Goals of Grammatical Error Correction: Fluency Instead of Grammaticality
                    Keisuke Sakaguchi, Courtney Napoles, Matt Post, Joel Tetreault
                    Transactions of the Association for Computational Linguistics 2016
                  5. arXiv
                    GLEU Without Tuning
                    Courtney Napoles, Keisuke Sakaguchi, Matt Post, Joel R. Tetreault
                    arXiv 2016
                    2015
                    1. ACL
                      Ground Truth for Grammatical Error Correction Metrics
                      Courtney Napoles, Keisuke Sakaguchi, Matt Post, Joel Tetreault
                      Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 2: Short Papers) Jul 2015
                    2. NAACL
                      Effective Feature Integration for Automated Short Answer Scoring
                      Keisuke Sakaguchi, Michael Heilman, Nitin Madnani
                      Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies May 2015
                      2014
                      1. WMT
                        Efficient Elicitation of Annotations for Human Evaluation of Machine Translation
                        Keisuke Sakaguchi, Matt Post, Benjamin Van Durme
                        Proceedings of the Ninth Workshop on Statistical Machine Translation Jun 2014
                        2013
                        1. ACL
                          Discriminative Approach to Fill-in-the-Blank Quiz Generation for Language Learners
                          Keisuke Sakaguchi, Yuki Arase, Mamoru Komachi
                          Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) Aug 2013
                        2. CoNLL
                          NAIST at 2013 CoNLL Grammatical Error Correction Shared Task
                          Ippei Yoshimoto, Tomoya Kose, Kensuke Mitsuzawa, Keisuke Sakaguchi, Tomoya Mizumoto, Yuta Hayashibe, Mamoru Komachi, Yuji Matsumoto
                          Proceedings of the Seventeenth Conference on Computational Natural Language Learning: Shared Task Aug 2013
                        3. BEA
                          NAIST at the NLI 2013 Shared Task
                          Tomoya Mizumoto, Yuta Hayashibe, Keisuke Sakaguchi, Mamoru Komachi, Yuji Matsumoto
                          Proceedings of the Eighth Workshop on Innovative Use of NLP for Building Educational Applications Jun 2013
                        4. MWE
                          Construction of English MWE Dictionary and its Application to POS Tagging
                          Yutaro Shigeto, Ai Azuma, Sorami Hisamoto, Shuhei Kondo, Tomoya Kose, Keisuke Sakaguchi, Akifumi Yoshimoto, Frances Yung, Yuji Matsumoto
                          Proceedings of the 9th Workshop on Multiword Expressions Jun 2013
                          2012
                          1. COLING
                            Joint English Spelling Error Correction and POS Tagging for Language Learners Writing
                            Keisuke Sakaguchi, Tomoya Mizumoto, Mamoru Komachi, Yuji Matsumoto
                            Proceedings of COLING 2012 Dec 2012
                          2. BEA
                            NAIST at the HOO 2012 Shared Task
                            Keisuke Sakaguchi, Yuta Hayashibe, Shuhei Kondo, Lis Kanashiro, Tomoya Mizumoto, Mamoru Komachi, Yuji Matsumoto
                            Proceedings of the Seventh Workshop on Building Educational Applications Using NLP Jun 2012