Details of the Researcher

PHOTO

Jun Suzuki
Section
Center for Language AI Research
Job title
Professor
Degree

  • 博士（工学）（奈良先端科学技術大学院大学）

  • 修士（工学）（慶應義塾大学）

Research History 8

  • 2024/04 - Present
    大学共同利用機関法人 情報・システム研究機構 国立情報学研究所 客員教授／大規模言語モデル研究開発センター 科学主幹

  • 2023/10 - Present
    Tohoku University　Center for Language AI Research　Director, Professor

  • 2017/04 - Present
    RIKEN　Center for Advanced Intelligence Project　Visiting Resercher

  • 2020/07 - 2023/09
    Tohoku University

  • 2020/04 - 2022/04
    Google LLC　Visiting Researcher

  • 2018/04 - 2020/06
    Tohoku University

  • 2018/04 - 2020/03
    Nippon Telegraph and Telephone Corporation　Communication Science Laboratories　Research Professor

  • 2001/04 - 2018/03
    Nippon Telegraph and Telephone Corporation　Communication Science Laboratories

Show all Show first 5

Committee Memberships 11

  • ACL Rolling Review　Editor-in-Chief

    2023/03 - Present

  • 言語処理学会　理事

    2022/04 - Present

  • 言語処理学会 代議員

    2018/04 - 2022/03

  • 言語処理学会　年次大会 プログラム委員（NLP2018. NLP2019：大会賞担当，NLP2020：副委員長，NLP2021：委員長, NLP2022：アドバイザー）

    2017 - 2022/03

  • 情報処理学会　東北支部 庶務幹事

    2020 - 2022

  • EMNLP-2018 Machine Learning Area Chair

    - 2018

  • ACL-2018 Machine Learning Senior Area Chair

    - 2018

  • Computational Linguistics Journal Editorial Board

    2015/01 - 2017/12

  • IJCNLP-2017 Machine Learning Area Chair

    - 2017

  • 言語処理学会　言語処理学会論文誌 編集委員

    2013/08 - 2014/08

  • ACL-2009 Statistical Machine Learning Methods Area Chair

    - 2009

Show all ︎Show first 5

Professional Memberships 4

  • 人工知能学会

    2020/06 - Present

  • Association for Computational Linguistics

    2003 - Present

  • INFORMATION PROCESSING SOCIETY OF JAPAN

    2001 - Present

  • THE ASSOCIATION FOR NATURAL LANGUAGE PROCESSING

    2001 - Present

Research Interests 10

  • 説明可能AI

  • 文法誤り訂正

  • Language Generation

  • Compact Modeling

  • Machine Translation

  • Word Embeddings

  • Deep Learning

  • Semi-supervised Learning

  • Machine Learning

  • Natural Language Processing

Research Areas 2

  • Informatics / Intelligent informatics / Machine Learning

  • Informatics / Intelligent informatics / Natural Language Procesing

Awards 53

  1. 言語処理学会第30回年次大会, 委員特別賞

    2024/03　言語処理学会　大規模視覚言語モデルに関する指示追従能力の検証

  2. 言語処理学会第30回年次大会, 委員特別賞

    2024/03　言語処理学会　大規模言語モデル事前学習の安定化

  3. 言語処理学会第30回年次大会 委員特別賞

    2024/03　言語処理学会　Integrated Gradientsにおける理想の積分ステップ数はインスタンス毎に異なる

  4. 言語処理学会第30回年次大会 優秀賞

    2024/03　言語処理学会　InstructDoc: 自然言語指示に基づく視覚的文書理解

  5. ACL Student Research Workshop (ACL-SRW 2023) Best Paper Award

    2023/07　Assessing Chain-of-Thought Reasoning against Lexical Negation: A Case Study on Syllogism

  6. 言語処理学会2022年度最優秀論文賞

    2023/03　言語処理学会　負例を厳選した対話応答選択による対話応答生成システムの評価

  7. 言語処理学会第29回年次大会, 委員特別賞

    2023/03　言語処理学会　計算資源が限られた複数組織での出力選択による協働の検討

  8. 言語処理学会第29回年次大会, 高電社賞

    2023/03　言語処理学会　文単位のNbest候補制約に基づく文書翻訳

  9. 言語処理学会第29回年次大会, 日本電気賞

    2023/03　言語処理学会　計算資源が限られた複数組織での出力選択による協働の検討

  10. 言語処理学会第29回年次大会, 言語資源賞

    2023/03　言語処理学会　日本語日常対話コーパスの構築

  11. 言語処理学会第28回年次大会 委員特別賞

    2022/03　言語処理学会　JParaCrawl v3.0: 大規模日英対訳コーパス

  12. 言語処理学会第28回年次大会 優秀賞

    2022/03　言語処理学会　論述リビジョンのためのメタ評価基盤

  13. 言語処理学会第28回年次大会 優秀賞

    2022/03　言語処理学会　Transformerを多層にする際の勾配消失問題と解決法について

  14. 言語処理学会第28回年次大会 最優秀賞

    2022/03　言語処理学会　ニューラル言語モデルの効率的な学習に向けた代表データ集合の獲得

  15. 言語処理学会 2021年度論文賞

    2022/03　言語処理学会　訓練事例の影響の軽量な推定

  16. AAMT Nagao Prize

    2021/06　Asia-Pacific Association for Machine Translation (AAMT)

  17. 言語処理学会 第27回年次大会 委員特別賞

    2021/03　言語処理学会　対話システムの矛盾応答の生成に対する脆弱性の分析

  18. 言語処理学会 第27回年次大会 委員特別賞

    2021/03　言語処理学会　オープンドメイン質問応答における解答可能性判別の役割

  19. 言語処理学会 第27回年次大会 スポンサー賞（デンソーアイティーラボラトリ賞）

    2021/03　言語処理学会　単語埋め込みの決定的縮約

  20. 言語処理学会 第27回年次大会 優秀賞

    2021/03　言語処理学会　単語埋め込みの決定的縮約

  21. 言語処理学会 論文誌 最優秀論文賞

    2021/03　言語処理学会　論述構造解析におけるスパン分散表現

  22. 第3回対話システムライブコンペティション 優秀賞

    2020/12　人工知能学会　ILYS aoba bot: 大規模ニューラル応答生成モデルとルールベースを統合した雑談対話システム

  23. 人工知能学会 第34回全国大会 全国大会優秀賞

    2020/11　人工知能学会　単語埋め込みのノルムと方向ベクトルを区別した文間最適輸送コスト

  24. 言語処理学会 第26回年次大会 優秀賞

    2020/03　言語処理学会　大規模疑似データを用いた高性能文法誤り訂正モデルの構築

  25. 言語処理学会 第26回年次大会 優秀賞

    2020/03　言語処理学会　テキストを通して世界を見る:機械読解における常識的推論のための画像説明文の評価

  26. 言語処理学会 第26回年次大会 優秀賞

    2020/03　言語処理学会　擬似タグと線形移動ベクトルを用いた単一モデルによる擬似モデルアンサンブル

  27. 言語処理学会 第26回年次大会 最優秀賞

    2020/03　人工知能学会　ベクトル⻑に基づく自己注意機構の解析

  28. 言語処理学会 第26回年次大会 最優秀賞

    2020/03　言語処理学会　超球面上での最適輸送に基づく文類似性尺度

  29. 人工知能学会 音声・言語理解と対話処理研究会（SLUD）第87回研究会 第10回対話システムシンポジウム 若手萌芽ポスター賞

    2019/12　人工知能学会　負例を厳選した対話応答選択テストセット構築の試みと分析

  30. NLP若手の会 (YANS) 第14回シンポジウム 奨励賞受賞

    2019/08　NLP若手の会　文法誤り訂正を拡張した新タスクの提案

  31. NLP若手の会 (YANS) 第14回シンポジウム 奨励賞受賞

    2019/08　NLP若手の会　文ベクトルの最適輸送に基づく類似性尺度

  32. 情報処理学会 第241回自然言語処理研究会 優秀研究賞

    2019/08　情報処理学会　クイズ解答タスクにおける大規模ラベルなしコーパスの利用: 言語モデルとデータ拡張

  33. ICLR-2019 Outstanding Reviewers

    2019/05　International Conference on Representation Learning

  34. 言語処理学会 第25回年次大会 優秀賞

    2019/03　言語処理学会　ExpertとImitatorの混合ネットワークによる大規模半教師あり学習

  35. NeurIPS-2018 Top 200 reviewers

    2018/12

  36. 人工知能学会 言語・音声理解と対話処理研究会 対話システムライブコンペティション 優秀賞

    2018/11　人工知能学会　Zunkobot: 複数の知識モジュールを統合した雑談対話システム

  37. NLP若手の会（YANS）第13回シンポジウム 奨励賞

    2018/08　NLP若手の会　サブワードに基づく単語ベクトルの再構築

  38. 言語処理学会第 24 回年次大会 優秀賞

    2018/03　言語処理学会　ニューラルヘッドライン生成における誤生成問題の改善

  39. 言語処理学会第 23 回年次大会 最優秀賞

    2017/03　言語処理学会　単語出現頻度予測機能付きRNNエンコーダデコーダモデル

  40. NAACL:HLT 2016 Best reviewers

    2016/06　Association of Computational Linguistics

  41. 言語処理学会第 22 回年次大会 最優秀賞

    2016/03　言語処理学会　単語分散表現に対する縮約モデリング

  42. ACL 2015 outstanding reviewers

    2015/07　Association of Computational Linguistics

  43. 情報処理学会 自然言語処理研究会 優秀研究賞

    2015/05　情報処理学会　逐次最適解更新による頑健な単語分散表現の学習方式

  44. 言語処理学会論文誌 論文賞

    2015/03　言語処理学会　単語並べ替えと冠詞生成の同時逐次処理：日英機械翻訳への適用

  45. 人工知能学会 現場イノベーション賞 銀賞

    2014/06　人工知能学会　しゃべってコンシェルにおける日本語質問応答技術の実用化

  46. 言語処理学会第 20 回年次大会 優秀賞

    2014/03　言語処理学会　オラクル要約の列挙

  47. 言語処理学会第 20 回年次大会 最優秀賞

    2014/03　言語処理学会　大規模素性集合に対する教師あり縮約モデリング

  48. 第40回 UBI 研究発表会 優秀論文賞

    2014/01　情報処理学会　Boostingを用いた環境変化に頑健なWi-Fi屋内位置推定手法の提案

  49. 言語処理学会第17回年次大会 最優秀発表論文賞

    2011/03　言語処理学会　L1正則化特徴選択に基づく大規模データ・特徴集合に適した半教師あり学習

  50. 言語処理学会第16回年次大会 優秀発表論文賞

    2010/03　言語処理学会　大規模ラベルなしデータを利用した係り受け解析の性能検証

  51. 言語処理学会第14回年次大会 優秀発表論文賞

    2008/03　言語処理学会　大規模ラベルなしデータを利用した言語解析器の性能検証

  52. 言語処理学会第13回年次大会 最優秀発表論文賞

    2007/03　言語処理学会　データの分布特性を利用した半教師あり学習：言語解析ヘの適用

  53. 言語処理学会第12回年次大会 最優秀発表論文賞

    2006/03　言語処理学会　誤り最小化に基づく条件付き確立場の学習：言語解析への適用

Show all ︎Show 5

Papers 154

  1. STEP: Staged Parameter-Efficient Pre-training for Large Language Models.

    Kazuki Yano, Takumi Ito, Jun Suzuki 0001

    NAACL (Short Papers)　374-384　2025

    DOI： 10.18653/v1/2025.naacl-short.32 　

  2. Drop-Upcycling: Training Sparse Mixture of Experts with Partial Re-initialization.

    Taishi Nakamura, Takuya Akiba, Kazuki Fujii, Yusuke Oda, Rio Yokota, Jun Suzuki 0001

    ICLR　2025

  3. VDocRAG: Retrieval-Augmented Generation over Visually-Rich Documents.

    Ryota Tanaka, Taichi Iki, Taku Hasegawa, Kyosuke Nishida, Kuniko Saito, Jun Suzuki 0001

    CVPR　24827-24837　2025

    DOI： 10.1109/CVPR52734.2025.02312 　

  4. Predicting Fine-tuned Performance on Larger Datasets Before Creating Them.

    Toshiki Kuramoto, Jun Suzuki 0001

    COLING (Industry)　204-212　2025

  5. Can Input Attributions Explain Inductive Reasoning in In-Context Learning?

    Mengyu Ye, Tatsuki Kuribayashi, Goro Kobayashi, Jun Suzuki 0001

    ACL (Findings)　21199-21225　2025

  6. Deterministic Compression of Word Embeddings.

    Yuki Nakamura, Jun Suzuki 0001, Takumi Ito, Kentaro Inui

    IEEE Access　13　39248-39262　2025

    DOI： 10.1109/ACCESS.2025.3546226 　

  7. Evaluating Model Alignment with Human Perception: A Study on Shitsukan in LLMs and LVLMs. Peer-reviewed

    Daiki Shiono, Ana Brassard, Yukiko Ishizuki, Jun Suzuki 0001

    COLING　11428-11444　2025

  8. MQM-Chat: Multidimensional Quality Metrics for Chat Translation. Peer-reviewed

    Yunmeng Li, Jun Suzuki 0001, Makoto Morishita, Kaori Abe, Kentaro Inui

    COLING　3283-3299　2025

  9. Document-level Translation with LLM Reranking: Team-J at WMT 2024 General Translation Task. Peer-reviewed

    Keito Kudo, Hiroyuki Deguchi, Makoto Morishita, Ryo Fujii, Takumi Ito, Shintaro Ozaki, Koki Natsumi, Kai Sato, Kazuki Yano, Ryosuke Takahashi, Subaru Kimura, Tomomasa Hara, Yusuke Sakai 0010, Jun Suzuki 0001

    Proceedings of the Ninth Conference on Machine Translation(WMT)　210-226　2024

    Publisher: Association for Computational Linguistics

    DOI： 10.18653/v1/2024.wmt-1.14 　

  10. Detecting Response Generation Not Requiring Factual Judgment. Peer-reviewed

    Ryohei Kamei, Daiki Shiono, Reina Akama, Jun Suzuki 0001

    Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies: Student Research Workshop　116-123　2024

    Publisher: Association for Computational Linguistics

    DOI： 10.18653/v1/2024.naacl-srw.13 　

  11. Pruning Multilingual Large Language Models for Multilingual Inference. Peer-reviewed

    Hwichan Kim, Jun Suzuki 0001, Tosho Hirasawa, Mamoru Komachi

    Findings of the Association for Computational Linguistics: EMNLP 2024　9921-9942　2024

    Publisher: Association for Computational Linguistics

    DOI： 10.18653/v1/2024.findings-emnlp.580 　

  12. The Impact of Integration Step on Integrated Gradients. Peer-reviewed

    Masahiro Makino, Yuya Asazuma, Shota Sasaki, Jun Suzuki 0001

    Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics　279-289　2024

    Publisher: Association for Computational Linguistics

  13. Towards Automated Document Revision: Grammatical Error Correction, Fluency Edits, and Beyond. Peer-reviewed

    Masato Mita, Keisuke Sakaguchi, Masato Hagiwara, Tomoya Mizumoto, Jun Suzuki 0001, Kentaro Inui

    Proceedings of the 19th Workshop on Innovative Use of NLP for Building Educational Applications(BEA)　251-265　2024

    Publisher: Association for Computational Linguistics

  14. STEP: Staged Parameter-Efficient Pre-training for Large Language Models. Peer-reviewed

    Kazuki Yano, Takumi Ito, Jun Suzuki 0001

    Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics　607-614　2024

    Publisher: Association for Computational Linguistics

    DOI： 10.18653/v1/2024.acl-srw.50 　

  15. InstructDoc: A Dataset for Zero-Shot Generalization of Visual Document Understanding with Instructions. Peer-reviewed

    Ryota Tanaka, Taichi Iki, Kyosuke Nishida, Kuniko Saito, Jun Suzuki 0001

    Thirty-Eighth AAAI Conference on Artificial Intelligence(AAAI)　19071-19079　2024

    Publisher: AAAI Press

    DOI： 10.1609/aaai.v38i17.29874 　

  16. A Large Collection of Model-generated Contradictory Responses for Consistency-aware Dialogue Systems Peer-reviewed

    Shiki Sato, Reina Akama, Jun Suzuki, Kentaro Inui

    Findings of the Association for Computational Linguistics ACL 2024　16047-16062　2024

    Publisher: Association for Computational Linguistics

    DOI： 10.18653/v1/2024.findings-acl.949 　

  17. Achievements and Challenges in Japanese Question Answering: Insights from Quiz Competition Results Peer-reviewed

    Tomoki Ariyama, Jun Suzuki, Masatoshi Suzuki, Ryota Tanaka, Reina Akama, Kyosuke Nishida

    Journal of Natural Language Processing　31　(1)　47-78　2024

    Publisher: Association for Natural Language Processing

    DOI： 10.5715/jnlp.31.47 　

    ISSN： 1340-7619

    eISSN： 2185-8314

  18. Aoba_v3 bot: a multimodal chatbot system combining rules and various response generation models Peer-reviewed

    Shoji Moriya, Daiki Shiono, Riki Fujihara, Yosuke Kishinami, Subaru Kimura, Shusaku Sone, Reina Akama, Yuta Matsumoto, Jun Suzuki, Kentaro Inui

    Advanced Robotics　37　(21)　1-14　2023/08/01

    Publisher: Informa UK Limited

    DOI： 10.1080/01691864.2023.2240883 　

    ISSN： 0169-1864

    eISSN： 1568-5535

  19. Reducing the Cost: Cross-Prompt Pre-Finetuning for Short Answer Scoring. Peer-reviewed

    Hiroaki Funayama, Yuya Asazuma, Yuichiroh Matsubayashi, Tomoya Mizumoto, Jun Suzuki, Kentaro Inui

    The 24th International Conference on Artificial Intelligence in Education (AIED2023)　78-89　2023/07

    DOI： 10.1007/978-3-031-36272-9_7 　

  20. B2T Connection: Serving Stability and Performance in Deep Transformers. Peer-reviewed

    Sho Takase, Shun Kiyono, Sosuke Kobayashi, Jun Suzuki

    ACL (Findings)　3078-3095　2023/07

    Publisher: Association for Computational Linguistics

    DOI： 10.18653/v1/2023.findings-acl.192 　

  21. Hunt for Buried Treasures: Extracting Unclaimed Embodiments from Patent Specifications. Peer-reviewed

    Chikara Hashimoto, Gautam Kumar, Shuichiro Hashimoto, Jun Suzuki

    ACL (industry)　25-36　2023/07

    Publisher: Association for Computational Linguistics

    DOI： 10.18653/v1/2023.acl-industry.3 　

  22. Examining the effect of whitening on static and contextualized word embeddings. Peer-reviewed

    Shota Sasaki, Benjamin Heinzerling, Jun Suzuki, Kentaro Inui

    Inf. Process. Manag.　60　(3)　103272-103272　2023/05

    DOI： 10.1016/j.ipm.2023.103272 　

  23. Extracting representative subset from extensive text data for training pre-trained language models Peer-reviewed

    Jun Suzuki, Heiga Zen, Hideto Kazawa

    Information Processing & Management　60　(3)　103249-103249　2023/05

    DOI： 10.1016/j.ipm.2022.103249 　

  24. Refactoring Programs Using Large Language Models with Few-Shot Examples.

    Atsushi Shirafuji, Yusuke Oda, Jun Suzuki 0001, Makoto Morishita, Yutaka Watanobe

    30th Asia-Pacific Software Engineering Conference(APSEC)　151-160　2023

    Publisher: IEEE

    DOI： 10.1109/APSEC60848.2023.00025 　

  25. SKIM at WMT 2023 General Translation Task. Peer-reviewed

    Keito Kudo, Takumi Ito, Makoto Morishita, Jun Suzuki

    WMT　128-136　2023

  26. Use of an AI-powered Rewriting Support Software in Context with Other Tools: A Study of Non-Native English Speakers. Peer-reviewed

    Takumi Ito, Naomi Yamashita, Tatsuki Kuribayashi, Masatoshi Hidaka, Jun Suzuki, Ge Gao, Jack Jamieson, Kentaro Inui

    UIST　45-13　2023

    DOI： 10.1145/3586183.3606810 　

  27. Assessing Step-by-Step Reasoning against Lexical Negation: A Case Study on Syllogism. Peer-reviewed

    Mengyu Ye, Tatsuki Kuribayashi, Jun Suzuki, Goro Kobayashi, Hiroaki Funayama

    EMNLP　14753-14773　2023

  28. Investigating the Effectiveness of Multiple Expert Models Collaboration. Peer-reviewed

    Ikumi Ito, Takumi Ito, Jun Suzuki, Kentaro Inui

    EMNLP (Findings)　14393-14404　2023

  29. A Challenging Multimodal Video Summary: Simultaneously Extracting and Generating Keyframe-Caption Pairs from Video. Peer-reviewed

    Keito Kudo, Haruki Nagasawa, Jun Suzuki, Nobuyuki Shimizu

    EMNLP　7380-7402　2023

    Publisher: Association for Computational Linguistics

    DOI： 10.18653/v1/2023.emnlp-main.457 　

  30. NT5 at WMT 2022 General Translation Task. Peer-reviewed

    Makoto Morishita, Keito Kudo, Yui Oka, Katsuki Chousa, Shun Kiyono, Sho Takase, Jun Suzuki

    WMT　318-325　2022/12

  31. Domain Adaptation of Machine Translation with Crowdworkers. Peer-reviewed

    Makoto Morishita, Jun Suzuki, Masaaki Nagata

    EMNLP (Industry Track)　606-618　2022/12

  32. Chat Translation Error Detection for Assisting Cross-lingual Communications Peer-reviewed

    Yunmeng Li, Jun Suzuki, Makoto Morishita, Kaori Abe, Ryoko Tokuhisa, Ana Brassard, Kentaro Inui

    the 3rd Workshop on Evaluation and Comparison of NLP Systems (Eval4NLP)　88-95　2022/11

    DOI： 10.18653/v1/2022.eval4nlp-1.9 　

  33. Bipartite-play Dialogue Collection for Practical Automatic Evaluation of Dialogue Systems. Peer-reviewed

    Shiki Sato, Yosuke Kishinami, Hiroaki Sugiyama, Reina Akama, Ryoko Tokuhisa, Jun Suzuki

    AACL/IJCNLP 2022 (Student Research Workshop)　8-16　2022/11

  34. Target-Guided Open-Domain Conversation Planning. Peer-reviewed

    Yosuke Kishinami, Reina Akama, Shiki Sato, Ryoko Tokuhisa, Jun Suzuki, Kentaro Inui

    COLING　660-668　2022/10

  35. Prompt Sensitivity of Language Model for Solving Programming Problems. Peer-reviewed

    Atsushi Shirafuji, Takumi Ito, Makoto Morishita, Yuki Nakamura, Yusuke Oda, Jun Suzuki, Yutaka Watanobe

    SoMeT　346-359　2022/09

    DOI： 10.3233/FAIA220264 　

  36. N-best Response-based Analysis of Contradiction-awareness in Neural Response Generation Models. Peer-reviewed

    Shiki Sato, Reina Akama, Hiroki Ouchi, Ryoko Tokuhisa, Jun Suzuki, Kentaro Inui

    SIGDIAL　637-644　2022/09

  37. Balancing Cost and Quality: An Exploration of Human-in-the-Loop Frameworks for Automated Short Answer Scoring. Peer-reviewed

    Hiroaki Funayama, Tasuku Sato, Yuichiroh Matsubayashi, Tomoya Mizumoto, Jun Suzuki, Kentaro Inui

    The 23rd International Conference on Artificial Intelligence in Education (AIED2022)　465-476　2022/07

    Publisher: Springer

    DOI： 10.1007/978-3-031-11644-5_38 　

  38. Diverse Lottery Tickets Boost Ensemble from a Single Pretrained Model Peer-reviewed

    Sosuke Kobayashi, Shun Kiyono, Jun Suzuki, Kentaro Inui

    2022 Challenges and Perspectives in Creating Large Language Models, Proceedings of the Workshop　42-50　2022/05

  39. Scene-Text Aware Image and Text Retrieval with Dual-Encoder. Peer-reviewed

    Shumpei Miyawaki, Taku Hasegawa, Kyosuke Nishida, Takuma Kato, Jun Suzuki

    Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics: Student Research Workshop　422-433　2022/05

    Publisher: Association for Computational Linguistics

    DOI： 10.18653/v1/2022.acl-srw.34 　

  40. JParaCrawl v3.0: A Large-scale English-Japanese Parallel Corpus. Peer-reviewed

    Makoto Morishita, Katsuki Chousa, Jun Suzuki, Masaaki Nagata

    LREC　6704-6710　2022/04

  41. Evaluating Dialogue Response Generation Systems via Response Selection with Well-chosen False Candidates Peer-reviewed

    Sato Shiki, Akama Reina, Ouchi Hiroki, Suzuki Jun, Inui Kentaro

    Journal of Natural Language Processing　29　(1)　53-83　2022/03

    Publisher: The Association for Natural Language Processing

    DOI： 10.5715/jnlp.29.53 　

    ISSN： 1340-7619

    eISSN： 2185-8314

    More details Close

    Developing an automatic evaluation framework for open-domain dialogue response generation systems that can validate the effects of daily system improvements at a low cost is necessary. However, existing metrics commonly used for automatic response generation evaluation, such as bilingual evaluation understudy (BLEU), correlate poorly with human evaluation. This poor correlation arises from the nature of dialogue, i.e., several acceptable responses to an input context. To address this issue, we focus on evaluating response generation systems via response selection. In this task, for a given context, systems select an appropriate response from a set of response candidates. Because the systems can only select specific candidates, evaluation via response selection can mitigate the effect of the above-mentioned nature of dialogue. Generally, false response candidates are randomly sampled from other unrelated dialogues, resulting in two issues: (a) unrelated false candidates and (b) acceptable utterances marked as false. General response selection test sets are unreliable owing to these issues. Thus, this paper proposes a method for constructing response selection test sets with well-chosen false candidates. Experiments demonstrate that evaluating systems via response selection with well-chosen false candidates correlates more strongly with human evaluation compared with commonly used automatic evaluation metrics such as BLEU.

  42. Instance-Based Neural Dependency Parsing Peer-reviewed

    Hiroki Ouchi, Jun Suzuki, Sosuke Kobayashi, Sho Yokoi, Tatsuki Kuribayashi, Masashi Yoshikawa, Kentaro Inui

    Transactions of the Association for Computational Linguistics　9　1493-1507　2021/12/17

    Publisher: MIT Press - Journals

    DOI： 10.1162/tacl_a_00439 　

    eISSN： 2307-387X

    More details Close

    Abstract Interpretable rationales for model predictions are crucial in practical applications. We develop neural models that possess an interpretable inference process for dependency parsing. Our models adopt instance-based inference, where dependency edges are extracted and labeled by comparing them to edges in a training set. The training edges are explicitly used for the predictions; thus, it is easy to grasp the contribution of each edge to the predictions. Our experiments show that our instance-based models achieve competitive accuracy with standard neural models and have the reasonable plausibility of instance-based explanations.

  43. SHAPE : Shifted Absolute Position Embedding for Transformers. Peer-reviewed

    Shun Kiyono, Sosuke Kobayashi, Jun Suzuki, Kentaro Inui

    Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing　3309-3321　2021/11

    Publisher: Association for Computational Linguistics

    More details Close

    Position representation is crucial for building position-aware representations in Transformers. Existing position representations suffer from a lack of generalization to test data with unseen lengths or high computational cost. We investigate shifted absolute position embedding (SHAPE) to address both issues. The basic idea of SHAPE is to achieve shift invariance, which is a key property of recent successful position representations, by randomly shifting absolute positions during training. We demonstrate that SHAPE is empirically comparable to its counterpart while being simpler and faster.

  44. Subword-Based Compact Reconstruction for Open-Vocabulary Neural Word Embeddings. Peer-reviewed

    Shota Sasaki, Jun Suzuki, Kentaro Inui

    IEEE/ACM Transactions on Audio, Speech and Language Processing　29　3551-3564　2021/11

    Publisher: Institute of Electrical and Electronics Engineers ({IEEE})

    DOI： 10.1109/TASLP.2021.3125133 　

  45. Phenomenon-wise Evaluation Dataset Towards Analyzing Robustness of Machine Translation Models Peer-reviewed

    Fujii Ryo, Mita Masato, Abe Kaori, Hanawa Kazuaki, Morishita Makoto, Suzuki Jun, Inui Kentaro

    Journal of Natural Language Processing　28　(2)　450-478　2021/06

    Publisher: The Association for Natural Language Processing

    DOI： 10.5715/jnlp.28.450 　

    ISSN： 1340-7619

    More details Close

    <p>Neural Machine Translation (NMT) has shown drastic improvement in its quality when translating clean input such as text from the news domain. However, existing studies suggest that NMT still struggles with certain kinds of input with considerable noise, such as User-Generated Contents (UGC) on the Internet. To make better use of NMT for cross-cultural communication, one of the most promising directions is to develop a translation model that correctly handles these informal expressions. Though its importance has been recognized, it is still not clear as to what creates the large performance gap between the translation of clean input and that of UGC. To answer the question, we present a new dataset, PheMT, for evaluating robustness of MT systems against specific linguistic phenomena in Japanese-English translation. We provide more fine-grained error analysis about the behavior of the models with the accuracy and relative drop in translation quality on the contrastive dataset specifically designed for each phenomenon. Our experiments with the dataset revealed that not only our in-house models but even widely used off-the-shelf systems are greatly disturbed by the presence of certain phenomena. </p>

  46. Efficient Estimation of Influence of a Training Instance Peer-reviewed

    Sosuke Kobayashi, Sho Yokoi, Jun Suzuki, Kentaro Inui

    Journal of Natural Language Processing　28　(2)　573-597　2021/06

    Publisher: The Association for Natural Language Processing

    DOI： 10.5715/jnlp.28.573 　

    ISSN： 1340-7619

    More details Close

    Understanding the influence of a training instance on a neural network model leads to improving interpretability. However, it is difficult and inefficient to evaluate the influence, which shows how a model's prediction would be changed if a training instance were not used. In this paper, we propose an efficient method for estimating the influence. Our method is inspired by dropout, which zero-masks a sub-network and prevents the sub-network from learning each training instance. By switching between dropout masks, we can use sub-networks that learned or did not learn each training instance and estimate its influence. Through experiments with BERT and VGGNet on classification datasets, we demonstrate that the proposed method can capture training influences, enhance the interpretability of error predictions, and cleanse the training dataset for improving generalization.

  47. Context-aware Neural Machine Translation with Mini-batch Embedding. Peer-reviewed

    Makoto Morishita, Jun Suzuki, Tomoharu Iwata, Masaaki Nagata

    EACL-2021　2513-2521　2021/04

    Publisher: Association for Computational Linguistics

  48. A Data-Oriented Approach for Closed-Book Question Answering Peer-reviewed

    Suzuki Masatoshi, Matsuda Koji, Ouchi Hiroki, Suzuki Jun, Inui Kentaro

    Journal of Natural Language Processing　28　(1)　3-25　2021/03

    Publisher: The Association for Natural Language Processing

    DOI： 10.5715/jnlp.28.3 　

    ISSN： 1340-7619

  49. An Empirical Study of Span Representations in Argumentation Structure Parsing. Peer-reviewed

    Tatsuki Kuribayashi, Hiroki Ouchi, Naoya Inoue, Paul Reisert, Toshinori Miyoshi, Jun Suzuki, Kentaro Inui

    Journal of Natural Language Processing　27　(4)　4691-4698　2020/12

    Publisher: The Association for Natural Language Processing

    DOI： 10.18653/v1/p19-1464 　

    ISSN： 1340-7619

  50. PheMT: A Phenomenon-wise Dataset for Machine Translation Robustness on User-Generated Contents. International-journal Peer-reviewed

    Ryo Fujii, Masato Mita, Kaori Abe, Kazuaki Hanawa, Makoto Morishita, Jun Suzuki, Kentaro Inui

    COLING-2020　5929-5943　2020/12

    Publisher: International Committee on Computational Linguistics

    DOI： 10.18653/v1/2020.coling-main.521 　

    More details Close

    Neural Machine Translation (NMT) has shown drastic improvement in its quality when translating clean input, such as text from the news domain. However, existing studies suggest that NMT still struggles with certain kinds of input with considerable noise, such as User-Generated Contents (UGC) on the Internet. To make better use of NMT for cross-cultural communication, one of the most promising directions is to develop a model that correctly handles these expressions. Though its importance has been recognized, it is still not clear as to what creates the great gap in performance between the translation of clean input and that of UGC. To answer the question, we present a new dataset, PheMT, for evaluating the robustness of MT systems against specific linguistic phenomena in Japanese-English translation. Our experiments with the created dataset revealed that not only our in-house models but even widely used off-the-shelf systems are greatly disturbed by the presence of certain phenomena.

  51. Seeing the world through text: Evaluating image descriptions for commonsense reasoning in machine reading comprehension International-journal Peer-reviewed

    Diana Galvan-Sosa, Jun Suzuki, Kyosuke Nishida, Koji Matsuda, Kentaro Inui

    the Second Workshop Beyond Vision and LANguage: inTEgrating Real-world kNowledge (LANTERN 2020)　2020/11

  52. Efficient Estimation of Influence of a Training Instance International-journal Peer-reviewed

    Sosuke Kobayashi, Sho Yokoi, Jun Suzuki, Kentaro Inui

    Proceedings of SustaiNLP: Workshop on Simple and Efficient Natural Language Processing　2020/11

    Publisher: Association for Computational Linguistics

    DOI： 10.18653/v1/2020.sustainlp-1.6 　

  53. Tohoku-AIP-NTT at WMT 2020 News Translation Task. Peer-reviewed

    Shun Kiyono, Takumi Ito, Ryuto Konno, Makoto Morishita, Jun Suzuki

    WMT-2020　145-155　2020/11

    Publisher: Association for Computational Linguistics

  54. Langsmith: An Interactive Academic Text Revision System. Peer-reviewed

    Takumi Ito, Tatsuki Kuribayashi, Masatoshi Hidaka, Jun Suzuki, Kentaro Inui

    EMNLP-2020 (Dem Track)　216-226　2020/11

    Publisher: Association for Computational Linguistics

    DOI： 10.18653/v1/2020.emnlp-demos.28 　

    More details Close

    Despite the current diversity and inclusion initiatives in the academic community, researchers with a non-native command of English still face significant obstacles when writing papers in English. This paper presents the Langsmith editor, which assists inexperienced, non-native researchers to write English papers, especially in the natural language processing (NLP) field. Our system can suggest fluent, academic-style sentences to writers based on their rough, incomplete phrases or sentences. The system also encourages interaction between human writers and the computerized revision system. The experimental results demonstrated that Langsmith helps non-native English-speaker students write papers in English. The system is available at https://emnlp-demo.editor. langsmith.co.jp/.

  55. Word Rotator's Distance. International-journal Peer-reviewed

    Sho Yokoi, Ryo Takahashi, Reina Akama, Jun Suzuki, Kentaro Inui

    Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP-2020)　2944-2960　2020/11

    Publisher: Association for Computational Linguistics

    DOI： 10.18653/v1/2020.emnlp-main.236 　

    More details Close

    A key principle in assessing textual similarity is measuring the degree of semantic overlap between two texts by considering the word alignment. Such alignment-based approaches are intuitive and interpretable; however, they are empirically inferior to the simple cosine similarity between general-purpose sentence vectors. To address this issue, we focus on and demonstrate the fact that the norm of word vectors is a good proxy for word importance, and their angle is a good proxy for word similarity. Alignment-based approaches do not distinguish them, whereas sentence-vector approaches automatically use the norm as the word importance. Accordingly, we propose a method that first decouples word vectors into their norm and direction, and then computes alignment-based similarity using earth mover's distance (i.e., optimal transport cost), which we refer to as word rotator's distance. Besides, we find how to grow the norm and direction of word vectors (vector converter), which is a new systematic approach derived from sentence-vector estimation methods. On several textual similarity datasets, the combination of these simple proposed methods outperformed not only alignment-based approaches but also strong baselines. The source code is available at https://github.com/eumesy/wrd

  56. Filtering Noisy Dialogue Corpora by Connectivity and Content Relatedness. International-journal Peer-reviewed

    Reina Akama, Sho Yokoi, Jun Suzuki, Kentaro Inui

    Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP-2020)　941-958　2020/11

    Publisher: Association for Computational Linguistics

    DOI： 10.18653/v1/2020.emnlp-main.68 　

    More details Close

    Large-scale dialogue datasets have recently become available for training neural dialogue agents. However, these datasets have been reported to contain a non-negligible number of unacceptable utterance pairs. In this paper, we propose a method for scoring the quality of utterance pairs in terms of their connectivity and relatedness. The proposed scoring method is designed based on findings widely shared in the dialogue and linguistics research communities. We demonstrate that it has a relatively good correlation with the human judgment of dialogue quality. Furthermore, the method is applied to filter out potentially unacceptable utterance pairs from a large-scale noisy dialogue corpus to ensure its quality. We experimentally confirm that training data filtered by the proposed method improves the quality of neural dialogue agents in response generation.

  57. A Self-Refinement Strategy for Noise Reduction in Grammatical Error Correction. International-journal Peer-reviewed

    Masato Mita, Shun Kiyono, Masahiro Kaneko, Jun Suzuki, Kentaro Inui

    Findings of the Association for Computational Linguistics: EMNLP 2020　267-280　2020/11

    Publisher: Association for Computational Linguistics

    DOI： 10.18653/v1/2020.findings-emnlp.26 　

    More details Close

    Existing approaches for grammatical error correction (GEC) largely rely on supervised learning with manually created GEC datasets. However, there has been little focus on verifying and ensuring the quality of the datasets, and on how lower-quality data might affect GEC performance. We indeed found that there is a non-negligible amount of "noise" where errors were inappropriately edited or left uncorrected. To address this, we designed a self-refinement method where the key idea is to denoise these datasets by leveraging the prediction consistency of existing models, and outperformed strong denoising baseline methods. We further applied task-specific techniques and achieved state-of-the-art performance on the CoNLL-2014, JFLEG, and BEA-2019 benchmarks. We then analyzed the effect of the proposed denoising method, and found that our approach leads to improved coverage of corrections and facilitated fluency edits which are reflected in higher recall and overall performance.

  58. Instance-Based Learning of Span Representations: A Case Study through Named Entity Recognition. International-journal Peer-reviewed

    Hiroki Ouchi, Jun Suzuki, Sosuke Kobayashi, Sho Yokoi, Tatsuki Kuribayashi, Ryuto Konno, Kentaro Inui

    Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL-2020)　6452-6459　2020/07

    Publisher: Association for Computational Linguistics

    DOI： 10.18653/v1/2020.acl-main.575 　

    More details Close

    Interpretable rationales for model predictions play a critical role in practical applications. In this study, we develop models possessing interpretable inference process for structured prediction. Specifically, we present a method of instance-based learning that learns similarities between spans. At inference time, each span is assigned a class label based on its similar spans in the training set, where it is easy to understand how much each training instance contributes to the predictions. Through empirical analysis on named entity recognition, we demonstrate that our method enables to build models that have high interpretability without sacrificing performance.

  59. Encoder-Decoder Models Can Benefit from Pre-trained Masked Language Models in Grammatical Error Correction. International-journal Peer-reviewed

    Masahiro Kaneko, Masato Mita, Shun Kiyono, Jun Suzuki, Kentaro Inui

    Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL-2020)　4248-4254　2020/07

    Publisher: Association for Computational Linguistics

    DOI： 10.18653/v1/2020.acl-main.391 　

    More details Close

    This paper investigates how to effectively incorporate a pre-trained masked language model (MLM), such as BERT, into an encoder-decoder (EncDec) model for grammatical error correction (GEC). The answer to this question is not as straightforward as one might expect because the previous common methods for incorporating a MLM into an EncDec model have potential drawbacks when applied to GEC. For example, the distribution of the inputs to a GEC model can be considerably different (erroneous, clumsy, etc.) from that of the corpora used for pre-training MLMs; however, this issue is not addressed in the previous methods. Our experiments show that our proposed method, where we first fine-tune a MLM with a given GEC corpus and then use the output of the fine-tuned MLM as additional features in the GEC model, maximizes the benefit of the MLM. The best-performing model achieves state-of-the-art performances on the BEA-2019 and CoNLL-2014 benchmarks. Our code is publicly available at: https://github.com/kanekomasahiro/bert-gec.

  60. Single Model Ensemble using Pseudo-Tags and Distinct Vectors. International-journal Peer-reviewed

    Ryosuke Kuwabara, Jun Suzuki, Hideki Nakayama

    Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL-2020)　3006-3013　2020/07

    Publisher: Association for Computational Linguistics

    DOI： 10.18653/v1/2020.acl-main.271 　

    More details Close

    Model ensemble techniques often increase task performance in neural networks; however, they require increased time, memory, and management effort. In this study, we propose a novel method that replicates the effects of a model ensemble with a single model. Our approach creates K-virtual models within a single parameter space using K-distinct pseudo-tags and K-distinct vectors. Experiments on text classification and sequence labeling tasks on several datasets demonstrate that our method emulates or outperforms a traditional model ensemble with 1/K-times fewer parameters.

  61. Evaluating Dialogue Generation Systems via Response Selection. International-journal Peer-reviewed

    Shiki Sato, Reina Akama, Hiroki Ouchi, Jun Suzuki, Kentaro Inui

    Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL-2020)　593-599　2020/07

    Publisher: Association for Computational Linguistics

    DOI： 10.18653/v1/2020.acl-main.55 　

    More details Close

    Existing automatic evaluation metrics for open-domain dialogue response generation systems correlate poorly with human evaluation. We focus on evaluating response generation systems via response selection. To evaluate systems properly via response selection, we propose the method to construct response selection test sets with well-chosen false candidates. Specifically, we propose to construct test sets filtering out some types of false candidates: (i) those unrelated to the ground-truth response and (ii) those acceptable as appropriate responses. Through experiments, we demonstrate that evaluating systems via response selection with the test sets developed by our method correlates more strongly with human evaluation, compared with widely used automatic evaluation metrics such as BLEU.

  62. Language Models as an Alternative Evaluator of Word Order Hypotheses: A Case Study in Japanese. International-journal Peer-reviewed

    Tatsuki Kuribayashi, Takumi Ito, Jun Suzuki, Kentaro Inui

    Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL-2020)　488-504　2020/07

    Publisher: Association for Computational Linguistics

    DOI： 10.18653/v1/2020.acl-main.47 　

    More details Close

    We examine a methodology using neural language models (LMs) for analyzing the word order of language. This LM-based method has the potential to overcome the difficulties existing methods face, such as the propagation of preprocessor errors in count-based methods. In this study, we explore whether the LM-based method is valid for analyzing the word order. As a case study, this study focuses on Japanese due to its complex and flexible word order. To validate the LM-based method, we test (i) parallels between LMs and human word order preference, and (ii) consistency of the results obtained using the LM-based method with previous linguistic studies. Through our experiments, we tentatively conclude that LMs display sufficient word order knowledge for usage as an analysis tool. Finally, using the LM-based method, we demonstrate the relationship between the canonical word order and topicalization, which had yet to be analyzed by large-scale experiments.

  63. Preventing Critical Scoring Errors in Short Answer Scoring with Confidence Estimation. International-journal Peer-reviewed

    Hiroaki Funayama, Shota Sasaki, Yuichiroh Matsubayashi, Tomoya Mizumoto, Jun Suzuki, Masato Mita, Kentaro Inui

    ACL-2020 (Student Research Workshop)　237-243　2020/07

    Publisher: Association for Computational Linguistics

    DOI： 10.18653/v1/2020.acl-srw.32 　

  64. Embeddings of Label Components for Sequence Labeling: A Case Study of Fine-grained Named Entity Recognition. International-journal Peer-reviewed

    Takuma Kato, Kaori Abe, Hiroki Ouchi, Shumpei Miyawaki, Jun Suzuki, Kentaro Inui

    ACL-2020 (Student Research Workshop)　222-229　2020/07

    Publisher: Association for Computational Linguistics

    DOI： 10.18653/v1/2020.acl-srw.30 　

    More details Close

    In general, the labels used in sequence labeling consist of different types of elements. For example, IOB-format entity labels, such as B-Person and I-Person, can be decomposed into span (B and I) and type information (Person). However, while most sequence labeling models do not consider such label components, the shared components across labels, such as Person, can be beneficial for label prediction. In this work, we propose to integrate label component information as embeddings into models. Through experiments on English and Japanese fine-grained named entity recognition, we demonstrate that the proposed method improves performance, especially for instances with low-frequency labels.

  65. Massive Exploration of Pseudo Data for Grammatical Error Correction. International-journal Peer-reviewed

    Shun Kiyono, Jun Suzuki, Tomoya Mizumoto, Kentaro Inui

    IEEE ACM Trans. Audio Speech Lang. Process.　28　2134-2145　2020/07

    DOI： 10.1109/TASLP.2020.3007753 　

  66. JParaCrawl: A Large Scale Web-Based English-Japanese Parallel Corpus International-journal Peer-reviewed

    Makoto Morishita, Jun Suzuki, Masaaki Nagata

    LREC-2020　3603-3609　2020/05

    Publisher: European Language Resources Association

    More details Close

    Recent machine translation algorithms mainly rely on parallel corpora. However, since the availability of parallel corpora remains limited, only some resource-rich language pairs can benefit from them. We constructed a parallel corpus for English-Japanese, for which the amount of publicly available parallel corpora is still limited. We constructed the parallel corpus by broadly crawling the web and automatically aligning parallel sentences. Our collected corpus, called JParaCrawl, amassed over 8.7 million sentence pairs. We show how it includes a broader range of domains and how a neural machine translation model trained with it works as a good pre-trained model for fine-tuning specific domains. The pre-training and fine-tuning approaches achieved or surpassed performance comparable to model training from the initial state and reduced the training time. Additionally, we trained the model with an in-domain dataset and JParaCrawl to show how we achieved the best performance with them. JParaCrawl and the pre-trained models are freely available online for research purposes.

  67. Assisting authors to convert raw products into polished prose Peer-reviewed

    Ito, T., Kuribayashi, T., Kobayashi, H., Brassard, A., Hagiwara, M., Suzuki, J., Inui, K.

    Journal of Cognitive Science　21　(1)　2020/03

  68. Diamonds in the Rough: Generating Fluent Sentences from Early-Stage Drafts for Academic Writing Assistance. Peer-reviewed

    Takumi Ito, Tatsuki Kuribayashi, Hayato Kobayashi, Ana Brassard, Masato Hagiwara, Jun Suzuki, Kentaro Inui

    Proceedings of the 12th International Conference on Natural Language Generation (INLG-2020)　40-53　2019/10

    Publisher: Association for Computational Linguistics

    DOI： 10.18653/v1/W19-8606 　

    More details Close

    The writing process consists of several stages such as drafting, revising, editing, and proofreading. Studies on writing assistance, such as grammatical error correction (GEC), have mainly focused on sentence editing and proofreading, where surface-level issues such as typographical, spelling, or grammatical errors should be corrected. We broaden this focus to include the earlier revising stage, where sentences require adjustment to the information included or major rewriting and propose Sentence-level Revision (SentRev) as a new writing assistance task. Well-performing systems in this task can help inexperienced authors by producing fluent, complete sentences given their rough, incomplete drafts. We build a new freely available crowdsourced evaluation dataset consisting of incomplete sentences authored by non-native writers paired with their final versions extracted from published academic papers for developing and evaluating SentRev models. We also establish baseline performance on SentRev using our newly built evaluation dataset.

  69. Transductive Learning of Neural Language Models for Syntactic and Semantic Analysis.

    Hiroki Ouchi, Jun Suzuki, Kentaro Inui

    Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing　3663-3669　2019

    Publisher: Association for Computational Linguistics

    DOI： 10.18653/v1/D19-1379 　

  70. An Empirical Study of Incorporating Pseudo Data into Grammatical Error Correction.

    Shun Kiyono, Jun Suzuki, Masato Mita, Tomoya Mizumoto, Kentaro Inui

    Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing　1236-1242　2019

    Publisher: Association for Computational Linguistics

    DOI： 10.18653/v1/D19-1119 　

    More details Close

    The incorporation of pseudo data in the training of grammatical error correction models has been one of the main factors in improving the performance of such models. However, consensus is lacking on experimental configurations, namely, choosing how the pseudo data should be generated or used. In this study, these choices are investigated through extensive experiments, and state-of-the-art performance is achieved on the CoNLL-2014 test set ($F_{0.5}=65.0$) and the official test set of the BEA-2019 shared task ($F_{0.5}=70.2$) without making any modifications to the model architecture.

  71. Select and Attend: Towards Controllable Content Selection in Text Generation. Peer-reviewed

    Xiaoyu Shen 0001, Jun Suzuki, Kentaro Inui, Hui Su, Dietrich Klakow, Satoshi Sekine

    Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing　579-590　2019

    Publisher: Association for Computational Linguistics

    DOI： 10.18653/v1/D19-1054 　

    More details Close

    Many text generation tasks naturally contain two steps: content selection and surface realization. Current neural encoder-decoder models conflate both steps into a black-box architecture. As a result, the content to be described in the text cannot be explicitly controlled. This paper tackles this problem by decoupling content selection from the decoder. The decoupled content selection is human interpretable, whose value can be manually manipulated to control the content of generated text. The model can be trained end-to-end without human annotations by maximizing a lower bound of the marginal likelihood. We further propose an effective way to trade-off between performance and controllability with a single adjustable hyperparameter. In both data-to-text and headline generation tasks, our model achieves promising results, paving the way for controllable content selection in text generation.

  72. TEASPN: Framework and Protocol for Integrated Writing Assistance Environments.

    Masato Hagiwara, Takumi Ito, Tatsuki Kuribayashi, Jun Suzuki, Kentaro Inui

    Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing　229-234　2019

    Publisher: Association for Computational Linguistics

    DOI： 10.18653/v1/D19-3039 　

    More details Close

    Language technologies play a key role in assisting people with their writing. Although there has been steady progress in e.g., grammatical error correction (GEC), human writers are yet to benefit from this progress due to the high development cost of integrating with writing software. We propose TEASPN, a protocol and an open-source framework for achieving integrated writing assistance environments. The protocol standardizes the way writing software communicates with servers that implement such technologies, allowing developers and researchers to integrate the latest developments in natural language processing (NLP) with low cost. As a result, users can enjoy the integrated experience in their favorite writing software. The results from experiments with human participants show that users use a wide range of technologies and rate their writing experience favorably, allowing them to write more fluent text.

  73. The AIP-Tohoku System at the BEA-2019 Shared Task.

    Hiroki Asano, Masato Mita, Tomoya Mizumoto, Jun Suzuki

    Proceedings of the Fourteenth Workshop on Innovative Use of NLP for Building Educational Applications(BEA@ACL)　176-182　2019

    Publisher: Association for Computational Linguistics

    DOI： 10.18653/v1/w19-4418 　

  74. NTT Neural Machine Translation Systems at WAT 2019.

    Makoto Morishita, Jun Suzuki, Masaaki Nagata

    Proceedings of the 6th Workshop on Asian Translation(WAT@EMNLP-IJCNLP)　99-105　2019

    Publisher: Association for Computational Linguistics

    DOI： 10.18653/v1/D19-5211 　

  75. Effective Adversarial Regularization for Neural Machine Translation.

    Motoki Sato, Jun Suzuki, Shun Kiyono

    Proceedings of the 57th Conference of the Association for Computational Linguistics　204-210　2019

    Publisher: Association for Computational Linguistics

    DOI： 10.18653/v1/p19-1020 　

  76. Character n-Gram Embeddings to Improve RNN Language Models.

    Sho Takase, Jun Suzuki, Masaaki Nagata

    The Thirty-Third AAAI Conference on Artificial Intelligence(AAAI)　5074-5082　2019

    Publisher: AAAI Press

    DOI： 10.1609/aaai.v33i01.33015074 　

    More details Close

    This paper proposes a novel Recurrent Neural Network (RNN) language model that takes advantage of character information. We focus on character n-grams based on research in the field of word embedding construction (Wieting et al. 2016). Our proposed method constructs word embeddings from character n-gram embeddings and combines them with ordinary word embeddings. We demonstrate that the proposed method achieves the best perplexities on the language modeling datasets: Penn Treebank, WikiText-2, and WikiText-103. Moreover, we conduct experiments on application tasks: machine translation and headline generation. The experimental results indicate that our proposed method also positively affects these tasks.

  77. Mixture of Expert/Imitator Networks: Scalable Semi-Supervised Learning Framework Peer-reviewed

    Shun Kiyono, Jun Suzuki, Kentaro Inui

    The Thirty-Third AAAI Conference on Artificial Intelligence(AAAI)　4073-4081　2019/01

    Publisher: AAAI Press

    DOI： 10.1609/aaai.v33i01.33014073 　

    More details Close

    The current success of deep neural networks (DNNs) in an increasingly broad range of tasks involving artificial intelligence strongly depends on the quality and quantity of labeled training data. In general, the scarcity of labeled data, which is often observed in many natural language processing tasks, is one of the most important issues to be addressed. Semi-supervised learning (SSL) is a promising approach to overcoming this issue by incorporating a large amount of unlabeled data. In this paper, we propose a novel scalable method of SSL for text classification tasks. The unique property of our method, Mixture of Expert/Imitator Networks, is that imitator networks learn to "imitate" the estimated label distribution of the expert network over the unlabeled data, which potentially contributes a set of features for the classification. Our experiments demonstrate that the proposed method consistently improves the performance of several types of baseline DNNs. We also demonstrate that our method has the more data, better performance property with promising scalability to the amount of unlabeled data.

  78. Personalized Visited-POI Assignment to Individual Raw GPS Trajectories. Peer-reviewed

    Jun Suzuki, Yoshihiko Suhara, Hiroyuki Toda, Kyosuke Nishida

    ACM Trans. Spatial Algorithms Syst.　5　(3)　16-28　2019

    DOI： 10.1145/3317667 　

  79. The Sally Smedley Hyperpartisan News Detector at SemEval-2019 Task 4. Peer-reviewed

    Kazuaki Hanawa, Shota Sasaki, Hiroki Ouchi, Jun Suzuki, Kentaro Inui

    Proceedings of the 13th International Workshop on Semantic Evaluation (SemEval)　1057-1061　2019

    Publisher: Association for Computational Linguistics

    DOI： 10.18653/v1/s19-2185 　

  80. Subword-based Compact Reconstruction of Word Embeddings. Peer-reviewed

    Shota Sasaki, Jun Suzuki, Kentaro Inui

    in Proceedings of the 17th Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT 2019)　3498-3508　2019

    Publisher: Association for Computational Linguistics

    DOI： 10.18653/v1/n19-1353 　

  81. Interpretable Adversarial Perturbation in Input Embedding Space for Text.

    Motoki Sato, Jun Suzuki, Hiroyuki Shindo, Yuji Matsumoto 0001

    Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence(IJCAI)　4323-4330　2018

    Publisher: ijcai.org

    DOI： 10.24963/ijcai.2018/601 　

    More details Close

    Following great success in the image processing field, the idea of adversarial training has been applied to tasks in the natural language processing (NLP) field. One promising approach directly applies adversarial training developed in the image processing field to the input word embedding space instead of the discrete input space of texts. However, this approach abandons such interpretability as generating adversarial texts to significantly improve the performance of NLP tasks. This paper restores interpretability to such methods by restricting the directions of perturbations toward the existing words in the input embedding space. As a result, we can straightforwardly reconstruct each input with perturbations to an actual text by considering the perturbations to be the replacement of words in the sentence while maintaining or even improving the task performance.

  82. Direct Output Connection for a High-Rank Language Model.

    Sho Takase, Jun Suzuki, Masaaki Nagata

    Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing(EMNLP)　4599-4609　2018

    Publisher: Association for Computational Linguistics

    DOI： 10.18653/v1/d18-1489 　

    More details Close

    This paper proposes a state-of-the-art recurrent neural network (RNN) language model that combines probability distributions computed not only from a final RNN layer but also from middle layers. Our proposed method raises the expressive power of a language model based on the matrix factorization interpretation of language modeling introduced by Yang et al. (2018). The proposed method improves the current state-of-the-art language model and achieves the best score on the Penn Treebank and WikiText-2, which are the standard benchmark datasets. Moreover, we indicate our proposed method contributes to two application tasks: machine translation and headline generation. Our code is publicly available at: https://github.com/nttcslab-nlp/doc_lm.

  83. Unsupervised Token-wise Alignment to Improve Interpretation of Encoder-Decoder Models. Peer-reviewed

    Shun Kiyono, Sho Takase, Jun Suzuki, Naoaki Okazaki, Kentaro Inui, Masaaki Nagata

    Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP　74-81　2018

    Publisher: Association for Computational Linguistics

    DOI： 10.18653/v1/w18-5410 　

  84. Reducing Odd Generation from Neural Headline Generation. Peer-reviewed

    Shun Kiyono, Sho Takase, Jun Suzuki, Naoaki Okazaki, Kentaro Inui, Masaaki Nagata

    Proceedings of the 32nd Pacific Asia Conference on Language, Information and Computation(PACLIC)　(to appear)　2018

    Publisher: Association for Computational Linguistics

  85. NTT's Neural Machine Translation Systems for WMT 2018. Peer-reviewed

    Makoto Morishita, Jun Suzuki, Masaaki Nagata

    Proceedings of the Third Conference on Machine Translation: Shared Task Papers, WMT 2018, Belgium, Brussels, October 31 - November 1, 2018　461-466　2018

    Publisher: Association for Computational Linguistics

    DOI： 10.18653/v1/w18-6421 　

  86. Pointwise HSIC: A Linear-Time Kernelized Co-occurrence Norm for Sparse Linguistic Expressions. Peer-reviewed

    Sho Yokoi, Sosuke Kobayashi, Kenji Fukumizu, Jun Suzuki, Kentaro Inui

    EMNLP-2018　abs/1809.00800　1763-1775　2018

    Publisher: Association for Computational Linguistics

    DOI： 10.18653/v1/d18-1203 　

    More details Close

    In this paper, we propose a new kernel-based co-occurrence measure that can be applied to sparse linguistic expressions (e.g., sentences) with a very short learning time, as an alternative to pointwise mutual information (PMI). As well as deriving PMI from mutual information, we derive this new measure from the Hilbert--Schmidt independence criterion (HSIC); thus, we call the new measure the pointwise HSIC (PHSIC). PHSIC can be interpreted as a smoothed variant of PMI that allows various similarity metrics (e.g., sentence embeddings) to be plugged in as kernels. Moreover, PHSIC can be estimated by simple and fast (linear in the size of the data) matrix calculations regardless of whether we use linear or nonlinear kernels. Empirically, in a dialogue response selection task, PHSIC is learned thousands of times faster than an RNN-based PMI while outperforming PMI in accuracy. In addition, we also demonstrate that PHSIC is beneficial as a criterion of a data selection task for machine translation owing to its ability to give high (low) scores to a consistent (inconsistent) pair with other pairs.

  87. Improving Neural Machine Translation by Incorporating Hierarchical Subword Features. Peer-reviewed

    Makoto Morishita, Jun Suzuki, Masaaki Nagata

    Proceedings of the 27th International Conference on Computational Linguistics, COLING 2018, Santa Fe, New Mexico, USA, August 20-26, 2018　32nd　618-629　2018

    Publisher: Association for Computational Linguistics

    ISSN： 1347-9881

  88. An Empirical Study of Building a Strong Baseline for Constituency Parsing. Peer-reviewed

    Jun Suzuki, Sho Takase, Hidetaka Kamigaito, Makoto Morishita, Masaaki Nagata

    Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, ACL 2018, Melbourne, Australia, July 15-20, 2018, Volume 2: Short Papers　612-618　2018

    Publisher: Association for Computational Linguistics

    DOI： 10.18653/v1/P18-2097 　

  89. Memory-efficient word embedding vectors

    Suzuki, J., Nagata, M.

    NTT Technical Review　15　(11)　2017

  90. Input-to-Output Gate to Improve RNN Language Models.

    Sho Takase, Jun Suzuki, Masaaki Nagata

    Proceedings of the Eighth International Joint Conference on Natural Language Processing(IJCNLP(2))　43-48　2017

    Publisher: Asian Federation of Natural Language Processing

    More details Close

    This paper proposes a reinforcing method that refines the output layers of existing Recurrent Neural Network (RNN) language models. We refer to our proposed method as Input-to-Output Gate (IOG). IOG has an extremely simple structure, and thus, can be easily combined with any RNN language models. Our experiments on the Penn Treebank and WikiText-2 datasets demonstrate that IOG consistently boosts the performance of several different types of current topline RNN language models.

  91. Enumeration of Extractive Oracle Summaries. Peer-reviewed

    Tsutomu Hirao, Masaaki Nishino, Jun Suzuki, Masaaki Nagata

    Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics, EACL 2017, Valencia, Spain, April 3-7, 2017, Volume 1: Long Papers　386-396　2017

    Publisher: Association for Computational Linguistics

    DOI： 10.18653/v1/e17-1037 　

    More details Close

    To analyze the limitations and the future directions of the extractive summarization paradigm, this paper proposes an Integer Linear Programming (ILP) formulation to obtain extractive oracle summaries in terms of ROUGE-N. We also propose an algorithm that enumerates all of the oracle summaries for a set of reference summaries to exploit F-measures that evaluate which system summaries contain how many sentences that are extracted as an oracle summary. Our experimental results obtained from Document Understanding Conference (DUC) corpora demonstrated the following: (1) room still exists to improve the performance of extractive summarization; (2) the F-measures derived from the enumerated oracle summaries have significantly stronger correlations with human judgment than those derived from single oracle summaries.

  92. Improving Neural Text Normalization with Data Augmentation at Character- and Morphological Levels. Peer-reviewed

    Itsumi Saito, Jun Suzuki, Kyosuke Nishida, Kugatsu Sadamitsu, Satoshi Kobashikawa, Ryo Masumura, Yuji Matsumoto 0001, Junji Tomita

    Proceedings of the Eighth International Joint Conference on Natural Language Processing, IJCNLP 2017, Taipei, Taiwan, November 27 - December 1, 2017, Volume 2: Short Papers　257-262　2017

    Publisher: Asian Federation of Natural Language Processing

  93. Deep Reinforcement Learning with Hidden Layers on Future States. Peer-reviewed

    Hirotaka Kameko, Jun Suzuki, Naoki Mizukami, Yoshimasa Tsuruoka

    Communications in Computer and Information Science　818　46-60　2017

    Publisher: Communications in Computer and Information Science

    DOI： 10.1007/978-3-319-75931-9_4 　

    ISSN： 1865-0929

  94. Cutting-off Redundant Repeating Generations for Neural Abstractive Summarization. Peer-reviewed

    Jun Suzuki, Masaaki Nagata

    Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics, EACL 2017, Valencia, Spain, April 3-7, 2017, Volume 2: Short Papers　abs/1701.00138　291-297　2017

    Publisher: Association for Computational Linguistics

    DOI： 10.18653/v1/e17-2047 　

    More details Close

    This paper tackles the reduction of redundant repeating generation that is often observed in RNN-based encoder-decoder models. Our basic idea is to jointly estimate the upper-bound frequency of each target vocabulary in the encoder and control the output words based on the estimation in the decoder. Our method shows significant improvement over a strong RNN-based encoder-decoder baseline and achieved its best results on an abstractive summarization benchmark.

  95. NTT Neural Machine Translation Systems at WAT 2017. Peer-reviewed

    Makoto Morishita, Jun Suzuki, Masaaki Nagata

    Proceedings of the 4th Workshop on Asian Translation, WAT@IJCNLP 2017, Taipei, Taiwan, November 27- December 1, 2017　89-94　2017

    Publisher: Asian Federation of Natural Language Processing

  96. Exploration Bonuses Based on Upper Confidence Bounds for Sparse Reward Games. Peer-reviewed

    Naoki Mizukami, Jun Suzuki, Hirotaka Kameko, Yoshimasa Tsuruoka

    Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)　10664 LNCS　165-175　2017

    Publisher: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

    DOI： 10.1007/978-3-319-71649-7_14 　

    ISSN： 1611-3349 0302-9743

  97. Shift-reduce Spinal TAG Parsing with Dynamic Programming

    Hayashi Katsuhiko, Suzuki Jun, Nagata Masaaki

    Information and Media Technologies　11　93-100　2016

    Publisher: Information and Media Technologies Editorial Board

    DOI： 10.11185/imt.11.93 　

    ISSN： 1881-0896

    More details Close

    The spinal tree adjoining grammar (TAG) parsing model of [Carreras 08] achieves the current state-of-the-art constituent parsing accuracy on the commonly used English Penn Treebank evaluation setting. Unfortunately, the model has the serious drawback of low parsing efficiency since its Eisner-CKY style parsing algorithm needs O(n4) computation time for input length n. This paper investigates a more practical solution and presents a beam search shift-reduce algorithm for spinal TAG parsing. Since the algorithm works in O(bn) (b is beam width), it can be expected to provide a significant improvement in parsing speed. However, to achieve faster parsing, it needs to prune a large number of candidates in an exponentially large search space and often suffers from severe search errors. In fact, our experiments show that the basic beam search shift-reduce parser does not work well for spinal TAGs. To alleviate this problem, we extend the proposed shift-reduce algorithm with two techniques: Dynamic Programming of [Huang 10a] and Supertagging. The proposed extended parsing algorithm is about 8 times faster than the Berkeley parser, which is well-known to be fast constituent parsing software, while offering state-of-the-art performance. Moreover, we conduct experiments on the Keyaki Treebank for Japanese to show that the good performance of our proposed parser is language-independent.

  98. Event report: NTT Communication Science Laboratories Open House 2016

    Suzuki, J., Kitagawa, N., Tsuchida, M., Ishiguro, K., Kuroki, S.

    NTT Technical Review　14　(11)　2016

  99. Shift-reduce spinal TAG parsing with dynamic programming Peer-reviewed

    Hayashi, K., Suzuki, J., Nagata, M.

    Transactions of the Japanese Society for Artificial Intelligence　31　(2)　2016

    Publisher: Japanese Society for Artificial Intelligence

    DOI： 10.1527/tjsai.J-F83 　

    ISSN： 1346-8030 1346-0714

  100. Sequence Alignment as a Set Partitioning Problem

    Nishino Masaaki, Suzuki Jun, Umetani Shunji, Hirao Tsutomu, Nagata Masaaki

    Journal of Natural Language Processing　23　(2)　175-194　2016

    Publisher: 一般社団法人 言語処理学会

    DOI： 10.5715/jnlp.23.175 　

    ISSN： 1340-7619

    More details Close

    Sequence alignment, which involves aligning elements of two given sequences, occurs in many natural language processing (NLP) tasks such as sentence alignment. Previous approaches for solving sequence alignment problems in NLP can be categorized into two groups. The first group assumes monotonicity of alignments; the second group does not assume monotonicity or consider the continuity of alignments. However, for example, in aligning sentences of parallel legal documents, it is desirable to use a sentence alignment method that does not assume monotonicity but can consider continuity. Herein, we present a method to align sequences where block-wise changes in the order of sequence elements exist. Our method formalizes a sequence alignment problem as a set partitioning problem, a type of combinatorial optimization problem, and solves the problem to obtain an alignment. We also propose an efficient algorithm to solve the optimization problem by applying column generation.

  101. Right-truncatable Neural Word Embeddings. Peer-reviewed

    Jun Suzuki, Masaaki Nagata

    NAACL HLT 2016, The 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, San Diego California, USA, June 12-17, 2016　1145-1151　2016

    Publisher: The Association for Computational Linguistics

    DOI： 10.18653/v1/n16-1135 　

  102. Learning Compact Neural Word Embeddings by Parameter Space Sharing. Peer-reviewed

    Jun Suzuki, Masaaki Nagata

    Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence, IJCAI 2016, New York, NY, USA, 9-15 July 2016　2046-2052　2016

    Publisher: IJCAI/AAAI Press

  103. Neural Headline Generation on Abstract Meaning Representation. Peer-reviewed

    Sho Takase, Jun Suzuki, Naoaki Okazaki, Tsutomu Hirao, Masaaki Nagata

    Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, EMNLP 2016, Austin, Texas, USA, November 1-4, 2016　1054-1059　2016

    Publisher: The Association for Computational Linguistics

    DOI： 10.18653/v1/d16-1112 　

  104. Phrase Table Pruning via Submodular Function Maximization. Peer-reviewed

    Masaaki Nishino, Jun Suzuki, Masaaki Nagata

    Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, ACL 2016, August 7-12, 2016, Berlin, Germany, Volume 2: Short Papers　2016

    Publisher: The Association for Computer Linguistics

    DOI： 10.18653/v1/p16-2066 　

  105. A Unified Learning Framework of Skip-Grams and Global Vectors. Peer-reviewed

    Jun Suzuki, Masaaki Nagata

    Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing of the Asian Federation of Natural Language Processing, ACL 2015, July 26-31, 2015, Beijing,　186-191　2015

    Publisher: The Association for Computer Linguistics

    DOI： 10.3115/v1/p15-2031 　

  106. Summarizing a Document by Trimming the Discourse Tree Peer-reviewed

    Hirao, T., Nishino, M., Yoshida, Y., Suzuki, J., Yasuda, N., Nagata, M.

    IEEE/ACM Transactions on Audio Speech and Language Processing　23　(11)　2081-2092　2015

    DOI： 10.1109/TASLP.2015.2465150 　

    ISSN： 2329-9290

  107. Incremental Word Re-Ordering and Article Generation: Its Application to Japanese-to-English Machine Translation

    Hayashi Katsuhiko, Sudoh Katsuhito, Tsukada Hajime, Suzuki Jun, Nagata Masaaki

    Journal of Natural Language Processing　21　(5)　1037-1057　2014

    Publisher: 一般社団法人 言語処理学会

    DOI： 10.5715/jnlp.21.1037 　

    ISSN： 1340-7619

    More details Close

    This paper introduces a novel word re-ordering model for statistical machine translation that employs a shift-reduce parser for inversion transduction grammars. The proposed model also solves article generation problems simultaneously with word re-ordering. We applied it to the post-ordering of phrase-based machine translation (PBMT) for Japanese-to-English patent translation tasks. Our experimental results suggest that our method achieves a significant improvement of +3.15 BLEU scores against 29.99 BLEU scores of the baseline PBMT system.

  108. SCT-D3 at the NTCIR-11 MedNLP-2 Task. Peer-reviewed

    Akinori Fujino, Jun Suzuki, Tsutomu Hirao, Hisashi Kurasawa, Katsuyoshi Hayashi

    Proceedings of the 11th NTCIR Conference on Evaluation of Information Access Technologies, NTCIR-11, National Center of Sciences, Tokyo, Japan, December 9-12, 2014　2014

    Publisher: National Institute of Informatics (NII)

  109. Restructuring output layers of deep neural networks using minimum risk parameter clustering. Peer-reviewed

    Yotaro Kubo, Jun Suzuki, Takaaki Hori, Atsushi Nakamura

    15TH ANNUAL CONFERENCE OF THE INTERNATIONAL SPEECH COMMUNICATION ASSOCIATION (INTERSPEECH 2014), VOLS 1-4　1068-1072　2014

    ISSN： 2308-457X

  110. Dependency-based Discourse Parser for Single-Document Summarization. Peer-reviewed

    Yasuhisa Yoshida, Jun Suzuki, Tsutomu Hirao, Masaaki Nagata

    Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, EMNLP 2014, October 25-29, 2014, Doha, Qatar, A meeting of SIGDAT, a Special Interest Group of the ACL　1834-1839　2014

    Publisher: ACL

    DOI： 10.3115/v1/d14-1196 　

  111. Fused Feature Representation Discovery for High-Dimensional and Sparse Data. Peer-reviewed

    Jun Suzuki, Masaaki Nagata

    Proceedings of the Twenty-Eighth AAAI Conference on Artificial Intelligence, July 27 -31, 2014, Québec City, Québec, Canada.　1593-1599　2014

    Publisher: AAAI Press

  112. Spam Detection Using Online Learning From Imbalanced Data with the Focus on False Positive Rate

    数原 良彦, 鈴木 潤, 鷲崎 誠司

    情報処理学会論文誌データベース（TOD）　6　(2)　51-60　2013/03/29

    Publisher: 情報処理学会

    ISSN： 1882-7799

    More details Close

    ウェブスパム判別においては，あらかじめラベル付けされた訓練データを用いて機械学習の枠組みでスパム判別器を生成する方法が広く用いられている．本稿では，ウェブスパム判別において特に課題となる偽陽性率に着目し，偏りのある訓練データを用いた場合においても偽陽性率を抑えつつ，高精度な判別が可能となるマージン識別器のオンライン学習手法を提案する．提案手法では学習時にスパムと非スパム側に異なるマージンサイズを設定することで偽陽性率を抑え，クラスを確率的に選択したうえで当該クラスにおいて最大損失を与える事例を更新に用いることで，訓練データの偏りの影響を排除しつつ高精度な学習を可能とする．本稿ではスパムブログデータセットを用いて訓練データの事例数に偏りがある場合においても提案手法によって偽陽性率を抑えた高精度なスパム判別が可能であることを示す．Web spam detection systems often use supervised learning algorithms; the classifier is created from a labeled training dataset. This paper focuses on minimizing the false positive rate, a key goal in web spam detection. We propose an online learning algorithm for margin classifiers that can suppress false positives. Our method prepares different margin sizes for spam and non-spam instances to suppress the false positive rate; it stochastically selects the class to choose the instance that has maximum-loss in the selected class to eliminate the effect of imbalance in the training data and achieve high classification accuracy. We use real splog datasets to verify that our method can achieve high accuracy and low-false-positive-rate spam classification even when the training data is imbalanced.

  113. Robust online learning to rank via selective pairwise approach based on evaluation measures

    Suhara, Y., Suzuki, J., Kataoka, R.

    Transactions of the Japanese Society for Artificial Intelligence　28　(1)　22-33　2013

    Publisher: The Japanese Society for Artificial Intelligence

    DOI： 10.1527/tjsai.28.22 　

    ISSN： 1346-0714

  114. Recent innovations in NTT's statistical machine translation

    Nagata, M., Sudoh, K., Suzuki, J., Akiba, Y., Hirao, T., Tsukada, H.

    NTT Technical Review　11　(12)　2013

    ISSN： 1348-3447

  115. Lagrangian relaxation for scalable text summarization while maximizing multipleobjectives Peer-reviewed

    Nishino, M., Yasuda, N., Tsutomu, H., Suzuki, J., Nagata, M.

    Transactions of the Japanese Society for Artificial Intelligence　28　(5)　433-441　2013

    DOI： 10.1527/tjsai.28.433 　

    ISSN： 1346-8030 1346-0714

  116. NTT-NII Statistical Machine Translation for NTCIR-10 PatentMT. Peer-reviewed

    Katsuhito Sudoh, Jun Suzuki, Hajime Tsukada, Masaaki Nagata, Sho Hoshino, Yusuke Miyao

    Proceedings of the 10th NTCIR Conference on Evaluation of Information Access Technologies, NTCIR-10, National Center of Sciences, Tokyo, Japan, June 18-21, 2013　2013

    Publisher: National Institute of Informatics (NII)

  117. Shift-Reduce Word Reordering for Machine Translation. Peer-reviewed

    Katsuhiko Hayashi, Katsuhito Sudoh, Hajime Tsukada, Jun Suzuki, Masaaki Nagata

    Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, EMNLP 2013, 18-21 October 2013, Grand Hyatt Seattle, Seattle, Washington, USA, A meeting of SIGDAT, a Special Interest Group of the ACL　1382-1386　2013

    Publisher: ACL

  118. Text summarization while maximizing multiple objectives with Lagrangian relaxation Peer-reviewed

    Nishino, M., Yasuda, N., Hirao, T., Suzuki, J., Nagata, M.

    Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)　7814 LNCS　772-775　2013

    Publisher: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

    DOI： 10.1007/978-3-642-36973-5_81 　

    ISSN： 0302-9743 1611-3349

  119. Supervised Model Learning with Feature Grouping based on a Discrete Constraint. Peer-reviewed

    Jun Suzuki, Masaaki Nagata

    Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics, ACL 2013, 4-9 August 2013, Sofia, Bulgaria, Volume 2: Short Papers　18-23　2013

    Publisher: The Association for Computer Linguistics

  120. TripleEye: Mining closed itemsets with minimum length thresholds based on ordered inclusion tree Peer-reviewed

    Shindo, H., Hirao, T., Suzuki, J., Fujino, A., Nagata, M.

    IPSJ Online Transactions　5　(2012)　192-202　2012

    Publisher: Information Processing Society of Japan

    DOI： 10.2197/ipsjtrans.5.192 　

    ISSN： 1882-6660

  121. A Query-Focused Summarization Method that Guarantees the Inclusion of Query Words. Peer-reviewed

    Norihito Yasuda, Masaaki Nishino, Tsutomu Hirao, Jun Suzuki, Ryoji Kataoka

    2012 23RD INTERNATIONAL WORKSHOP ON DATABASE AND EXPERT SYSTEMS APPLICATIONS (DEXA)　126-130　2012

    DOI： 10.1109/DEXA.2012.59 　

    ISSN： 1529-4188

  122. Learning High-performance Dependency Parsing Models by Large-scale Semi-supervised Learning

    鈴木 潤, 磯崎 秀樹, 永田 昌明

    情報処理学会論文誌　52　(11)　3038-3051　2011/11/15

    Publisher: 情報処理学会

    ISSN： 1882-7764

    More details Close

    係り受け解析では，正解係り受け構造が付与されたデータを用いた教師あり学習により解析器を学習するのが現在最も一般的な方法であり，データ量が十分あれば非常に高い解析精度が得られることが実証されている．しかし，さらなる解析精度向上のため，正解データを増やし続けるのは作成に要する費用や時間の観点で現実的な方策ではない．そこで本論文では，正解係り受け構造が付与されていないデータも利用して解析精度を向上させる，いわゆる半教師あり学習に基づく係り受け解析モデルとその学習法を提案する．実験では，係り受け解析の標準評価データとして広く利用されている，係り受け構造が交差するチェコ語，交差しない英語の2言語の係り受け解析データを用いて，提案法の有効性を定性的，定量的に検証する，提案法は，従来の教師あり学習で得た係り受け解析器を大幅に上回る解析精度を達成することを示す．Intensive work have recently been undertaken to develop dependency parsing. Most of the recent developed dependency parsers are obtained by using supervised learning with labeled data. In contrast, this paper introduces a high-performance dependency parser trained by semi-supervised learning, which is able to effectively incorporate unlabeled data as an additional training data. We demonstrate the effectiveness of our proposed method on dependency parsing experiments using two widely used test collections: the Penn Treebank for English as a projective dependency parsing, and the Prague Dependency Treebank for Czech as a non-projective dependency parsing. Our results in the above datasets significantly outperform those obtained from conventional supervised learning approach.

  123. Training Conditional Random Fields Based on Segment-Wise Maximum Figure-of-Merit Functions Peer-reviewed

    SUZUKI Jun, ISOZAKI Hideki

    The IEICE transactions on information and systems　94　(5)　908-918　2011/05/01

    Publisher: 一般社団法人電子情報通信学会

    ISSN： 1880-4535

    More details Close

    サンプル間に依存関係があるデータに対して大域的な最適化による識別学習を行うモデルとして,条件付確率場が提案され多くの実タスクで良好な性能を示している.条件付確率場のパラメータ推定(学習)は,確率場全体のゆう度,あるいは,事後確率に基づく目的関数を最大化する方法が一般的である.しかし,実タスクを評価する際に用いる評価指標は,ゆう度や事後確率でなく,タスクの目的に合わせてF値等の様々な評価関数が用いられる.そのために,タスクの評価指標と学習時の目的関数間にはしばしば不整合が起きることがある.しかし,この不整合についてはこれまでほとんど考慮されてこなかった.そこで本論文では,条件付確率場の適用先として多く用いられている系列セグメンテーションタスクに焦点を当て,系列セグメンテーションタスクの評価に使う評価指標を直接学習時の目的関数として利用し,期待性能を向上させる枠組みを提案する.具体的には,セグメント単位の再現率,適合率,F値といったタスク評価指標を学習時の目的関数として導入する方法を提案する.実タスクでの実験として自然言語処理のチャンキング,固有表現抽出タスクを用いて提案法の性能を検証する.実際に用いるタスクの評価指標と学習時の目的関数を合わせることで,適用した評価指標での性能向上が可能であることを示す.

  124. Distributed Minimum Error Rate Training of SMT using Particle Swarm Optimization. Peer-reviewed

    Jun Suzuki, Kevin Duh, Masaaki Nagata

    Fifth International Joint Conference on Natural Language Processing, IJCNLP 2011, Chiang Mai, Thailand, November 8-13, 2011　649-657　2011

    Publisher: The Association for Computer Linguistics

  125. Learning Condensed Feature Representations from Large Unsupervised Data Sets for Supervised Learning. Peer-reviewed

    Jun Suzuki, Hideki Isozaki, Masaaki Nagata

    The 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, Proceedings of the Conference, 19-24 June, 2011, Portland, Oregon, USA - Short Papers　636-641　2011

    Publisher: The Association for Computer Linguistics

  126. A Syntax Free Approach for Sentence Compression

    平尾 努, 鈴木 潤, 磯崎 秀樹

    情報処理学会論文誌データベース（TOD）　2　(1)　1-9　2009/03/31

    Publisher: 情報処理学会

    ISSN： 1882-7799

    More details Close

    従来の文短縮手法の多くは，入力された文を構文木として表現し，その部分木を削除することで，短縮文を生成する．このようなアプローチは文法的な短縮文を生成するという観点からは理にかなっている．しかし，多くの場合，人間は構文木の刈り込みだけで短縮文を生成するわけではない．これは，構文情報に過度に依存することが，高品質な文短縮を行うための妨げとなることを示している．そこで，本稿では，構文情報を用いない文短縮手法を提案する．短縮文の言語としてのもっともらしさを構文情報を用いずに評価するため，原文と大規模コーパスから得た統計情報を組み合わせた新たな言語モデルを提案する．提案手法を文献 18) のテストセットを用いて評価したところ，自動評価指標においては，提案手法が従来法より優れていることを確認した．さらに，提案手法が日本語だけでなく英語でも有効であることも示す．Conventional sentence compression methods build a parse tree and then trim the tree. This approach is reasonable beacuse the compressed sentence keeps fluency. However, in many cases, reference compressions that were made by humans do not always retain syntactic structures of original sentences but they are acceptable. This implies that syntax is an impediment to achieving humanquality compression. Therefore, this paper propses a syntax free sentence compressor. As an alternative to syntactic information, we propose a novel language model that combines statistics from an original sentence and a general corpus. We conducted experimental evaluation on the test set used in Hirao, et al. 18). The results showed that our method outperformed the conventional method in automatic metrics. Moreover, we show the effectiveness of our method for English compression.

  127. Automatic summarization as a combinatorial optimization problem

    Hirao, T., Suzuki, J., Isozaki, H.

    Transactions of the Japanese Society for Artificial Intelligence　24　(2)　223-231　2009

    Publisher: Japanese Society for Artificial Intelligence

    DOI： 10.1527/tjsai.24.223 　

    ISSN： 1346-8030 1346-0714

    eISSN： 1346-8030

  128. An Empirical Study of Semi-supervised Structured Conditional Models for Dependency Parsing. Peer-reviewed

    Jun Suzuki, Hideki Isozaki, Xavier Carreras, Michael Collins 0001

    Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing, EMNLP 2009, 6-7 August 2009, Singapore, A meeting of SIGDAT, a Special Interest Group of the ACL　551-560　2009

    Publisher: ACL

  129. A Syntax-Free Approach to Japanese Sentence Compression. Peer-reviewed

    Tsutomu Hirao, Jun Suzuki, Hideki Isozaki

    ACL 2009, Proceedings of the 47th Annual Meeting of the Association for Computational Linguistics and the 4th International Joint Conference on Natural Language Processing of the AFNLP, 2-7 August 2009, Singapore　826-833　2009

    Publisher: The Association for Computer Linguistics

  130. NTT statistical machine translation system for IWSLT 2008. Peer-reviewed

    Katsuhito Sudoh, Taro Watanabe, Jun Suzuki, Hajime Tsukada, Hideki Isozaki

    2008 International Workshop on Spoken Language Translation, IWSLT 2008, Honolulu, Hawaii, USA, October 20-21, 2008　92-97　2008

    Publisher: ISCA

  131. Multi-label Text Categorization with Model Combination based on F1-score Maximization. Peer-reviewed

    Akinori Fujino, Hideki Isozaki, Jun Suzuki

    Third International Joint Conference on Natural Language Processing, IJCNLP 2008, Hyderabad, India, January 7-12, 2008　823-828　2008

    Publisher: The Association for Computer Linguistics

  132. Semi-Supervised Sequential Labeling and Segmentation Using Giga-Word Scale Unlabeled Data. Peer-reviewed

    Jun Suzuki, Hideki Isozaki

    ACL 2008, Proceedings of the 46th Annual Meeting of the Association for Computational Linguistics, June 15-20, 2008, Columbus, Ohio, USA　665-673　2008

    Publisher: The Association for Computer Linguistics

  133. A discriminative sentence compression method as combinatorial optimization problem

    Hirao, T., Suzuki, J., Isczaki, H.

    Transactions of the Japanese Society for Artificial Intelligence　22　(6)　574-584　2007

    Publisher: 一般社団法人 人工知能学会

    DOI： 10.1527/tjsai.22.574 　

    ISSN： 1346-0714

    eISSN： 1346-8030

  134. Larger feature set approach for machine translation in IWSLT 2007. Peer-reviewed

    Taro Watanabe, Jun Suzuki, Katsuhito Sudoh, Hajime Tsukada, Hideki Isozaki

    2007 International Workshop on Spoken Language Translation, IWSLT 2007, Trento, Italy, October 15-16, 2007　111-118　2007

    Publisher: ISCA

  135. Semi-Supervised Structured Output Learning Based on a Hybrid Generative and Discriminative Approach. Peer-reviewed

    Jun Suzuki, Akinori Fujino, Hideki Isozaki

    EMNLP-CoNLL 2007, Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, June 28-30, 2007, Prague, Czech Republic　791-800　2007

    Publisher: ACL

  136. Online Large-Margin Training for Statistical Machine Translation. Peer-reviewed

    Taro Watanabe, Jun Suzuki, Hajime Tsukada, Hideki Isozaki

    EMNLP-CoNLL 2007, Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, June 28-30, 2007, Prague, Czech Republic　764-773　2007

    Publisher: ACL

  137. NTT statistical machine translation for IWSLT 2006. Peer-reviewed

    Taro Watanabe, Jun Suzuki, Hajime Tsukada, Hideki Isozaki

    2006 International Workshop on Spoken Language Translation, IWSLT 2006, Keihanna Science City, Kyoto, Japan, November 27-28, 2006　95-102　2006

    Publisher: ISCA

  138. Training Conditional Random Fields with Multivariate Evaluation Measures. Peer-reviewed

    Jun Suzuki, Erik McDermott, Hideki Isozaki

    COLING/ACL 2006, VOLS 1 AND 2, PROCEEDINGS OF THE CONFERENCE　217-224　2006

    DOI： 10.3115/1220175.1220203 　

  139. Identifying Bloggers' Residential Areas. Peer-reviewed

    Norihito Yasuda, Tsutomu Hirao, Jun Suzuki, Hideki Isozaki

    Computational Approaches to Analyzing Weblogs, Papers from the 2006 AAAI Spring Symposium, Technical Report SS-06-03, Stanford, California, USA, March 27-29, 2006　231-236　2006

    Publisher: AAAI

  140. Hierarchical directed acyclic graph kernel Peer-reviewed

    Suzuki, J., Sasaki, Y., Maeda, E.

    Systems and Computers in Japan　37　(10)　58-68　2006

    DOI： 10.1002/scj.20485 　

    ISSN： 0882-1666 1520-684X

  141. Automatic Sentence Alignment for Monolingual Corpora Peer-reviewed

    HIRAO TSUTOMU, SUZUKI JUN, ISOZAKI HIDEKI, MAEDA EISAKU

    IPSJ journal　46　(10)　2533-2545　2005/10/15

    Publisher: 一般社団法人情報処理学会

    ISSN： 1882-7764

    More details Close

    Monolingual aligned corpora are valuable for natural language processing. In order to generate text, we can learn various kinds of knowledge from such corpora. For instance, summary sentences aligned with sentences from original documents are useful for the study of automatic summarization. However, conventional methods are not suitable for one-to-many or many-to-one correspondence. Moreover, the similarity measure for alignment is not optimal. In this paper, we propose an automatic alignment method for these monolingual corpora. First, we transform a sentence into the set of paths in its dependency structure. Next, we calculate similarity between the paths based on ESK (Extended String Subsequence Kernel) which consider both sequential patterns and semantic labels. By using these procedures, we can derive a one-to-many or many-to-one correspondence among sentences. Experimental results using TSC (Text Summarization Challenge) corpora, which align summary sentences with original sentences, showed that our method obtained 0.95-0.97 F-measure for single document summarization data and 0.72-0.83 F-measure for multiple document summarization data.

  142. Hierarchical Directed Acyclic Graph Kernel Peer-reviewed

    SUZUKI Jun, SASAKI Yutaka, MAEDA Eisaku

    The IEICE transactions on information and systems Pt. 2　88　(2)　230-240　2005/02/01

    Publisher: 一般社団法人電子情報通信学会

    ISSN： 0915-1923

    More details Close

    本論文では, テキスト処理に適したカーネル「階層非循環有向グラフカーネル」(HDAGカーネル)を提案する.テキスト処理のあらゆるタスクにおいて, テキストのもつ情報をどのように表現し, どのように演算にのせるかということは, 常に我々が直面する重要な課題である.これまで, 出現単語の頻度や系列などに着目したカーネル関数が提案されているが, テキストの表層的な情報しか扱っていない, 表現形式の有効性が十分に検証されていない, などの問題点があった.そこで, 階層非循環有向グラフ, すなわち, グラフ内のノードがサブグラフにより表現されるような構造をもつ有向グラフをテキストの表現形式として利用し, 階層的に構造化される様々な言語的特徴を統一的に表現することを可能にした.また, このグラフを入力とするカーネル関数を定義することにより, 実用的な計算量で言語的特徴を考慮したテキスト間の演算を実現した.テキスト処理の実タスクの一つである質問分類タスクを取り上げ, Sequence Kernel, Tree Kernel, Bag-of-words Kernelなどの従来手法との性能比較行い, 提案手法の有効性を実証した.

  143. Sequence and Tree Kernels with Statistical Feature Mining. Peer-reviewed

    Jun Suzuki, Hideki Isozaki

    Advances in Neural Information Processing Systems 18 [Neural Information Processing Systems, NIPS 2005, December 5-8, 2005, Vancouver, British Columbia, Canada]　1321-1328　2005

  144. The NTT statistical machine translation system for IWSLT2005. Peer-reviewed

    Hajime Tsukada, Taro Watanabe, Jun Suzuki, Hideto Kazawa, Hideki Isozaki

    2005 International Workshop on Spoken Language Translation, IWSLT 2005, Pittsburgh, PA, USA, October 24-25, 2005　112-117　2005

    Publisher: ISCA

  145. Boosting-based Parse Reranking with Subtree Features. Peer-reviewed

    Taku Kudo, Jun Suzuki, Hideki Isozaki

    ACL 2005, 43rd Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference, 25-30 June 2005, University of Michigan, USA　189-196　2005

    Publisher: The Association for Computer Linguistics

    DOI： 10.3115/1219840.1219864 　

  146. SAIQA-II: A Trainable Japanese QA System with SVM Peer-reviewed

    SASAKI YUTAKA, ISOZAKI HIDEKI, SUZUKI JUN, KOKURYOU KOJI, HIRAO TSUTOMU, KAZAWA HIDETO, MAEDA EISAKU

    Transactions of Information Processing Society of Japan　45　(2)　635-646　2004/02/15

    Publisher: 一般社団法人情報処理学会

    ISSN： 1882-7764

    More details Close

    This paper describes a Japanese Question-Answering (QA) System, SAIQA-II. These years, researchers have been attracted to the study of developing Open-Domain QA systems that find answers to a natural language question given by a user. Most of conventional QA systems take an approach to manually constructing rules and evaluation functions to find answers to a question. This paper regards the specifications of main components of a QA system, question analysis and answer extraction, as 2-class classification problems. The question analysis determines the question type of a given question and the answer extraction selects answer candidates that match the question types. To confirm the feasibility of our approach, SAIQA-II was implemented using Support Vector Machines (SVMs). We conducted experiments on a QA test collection with 2,000 question-answer pairs based on 5-fold cross validation. Experimental results showed that the trained system achieved about 0.4 in MRR and about 55% in TOP5 accuracy.

  147. Dependency-based Sentence Alignment for Multiple Document Summarization. Peer-reviewed

    Tsutomu Hirao, Jun Suzuki, Hideki Isozaki, Eisaku Maeda

    COLING 2004, 20th International Conference on Computational Linguistics, Proceedings of the Conference, 23-27 August 2004, Geneva, Switzerland　2004

  148. Convolution Kernels with Feature Selection for Natural Language Processing Tasks. Peer-reviewed

    Jun Suzuki, Hideki Isozaki, Eisaku Maeda

    Proceedings of the 42nd Annual Meeting of the Association for Computational Linguistics, 21-26 July, 2004, Barcelona, Spain.　119-126　2004

    Publisher: ACL

    DOI： 10.3115/1218955.1218971 　

  149. Question Type Classification Using Word Attribute N-gram and Statistical Machine Learning Peer-reviewed

    SUZUKI JUN, SASAKI YUTAKA, MAEDA EISAKU

    Transactions of Information Processing Society of Japan　44　(11)　2839-2853　2003/11/15

    Publisher: 一般社団法人情報処理学会

    ISSN： 1882-7764

    More details Close

    Question type classification attempts to identify the intention of a given question. The approach to high performance question classification typically yields an extremely large number of features because question types are well featured by the structures of the word attributes inside the questions. We propose a technique for finding &quot;word attribute AT-gram&quot; to extract effective features for question type classification, and then, we use these features with machine learning technique, Support Vector Machines (SVM), to create a high performance question type classifier. Results of question type classification experiments using 10,000 question samples showed much higher performance than the other conventional methods. Moreover, we clarify the performance of a feature extraction method and the effective features of each question type.

  150. 質問応答システム: SAIQA―何でも答える物知り博士 Peer-reviewed

    前田英作, 磯崎秀樹, 佐々木裕, 賀沢秀人, 平尾努, 鈴木潤

    NTT R & D　52　(2)　122-133　2003/02

    Publisher: 電気通信協会

    ISSN： 0915-2326

  151. Question answering system: SAIQA - A "learned computer" that answers any questions

    Maeda, E., Isozaki, H., Sasaki, Y., Kazawa, H., Hirao, T., Suzuki, J.

    NTT R and D　52　(2)　2003

  152. Kernels for Structured Natural Language Data. Peer-reviewed

    Jun Suzuki, Yutaka Sasaki, Eisaku Maeda

    Advances in Neural Information Processing Systems 16 [Neural Information Processing Systems, NIPS 2003, December 8-13, 2003, Vancouver and Whistler, British Columbia, Canada]　643-650　2003

    Publisher: MIT Press

  153. Hierarchical Directed Acyclic Graph Kernel: Methods for Structured Natural Language Data. Peer-reviewed

    Jun Suzuki, Tsutomu Hirao, Yutaka Sasaki, Eisaku Maeda

    41ST ANNUAL MEETING OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS, PROCEEDINGS OF THE CONFERENCE　32-39　2003

    DOI： 10.3115/1075096.1075101 　

  154. SVM Answer Selection for Open-Domain Question Answering. Peer-reviewed

    Jun Suzuki, Yutaka Sasaki, Eisaku Maeda

    19th International Conference on Computational Linguistics, COLING 2002, Howard International House and Academia Sinica, Taipei, Taiwan, August 24 - September 1, 2002　2002

Show all ︎Show first 5

Misc. 185

  1. Sales Management System Utilizing Text Analysis of Daily Reports

    飯塚新司, 塚原朋哉, 星魁人, 鈴木潤

    情報処理学会全国大会講演論文集　86th　(2)　2024

  2. 大規模言語モデル事前学習の安定化

    高瀬翔, 高瀬翔, 清野舜, 清野舜, 小林颯介, 鈴木潤

    言語処理学会年次大会発表論文集(Web)　30th　2024

    ISSN： 2188-4420

  3. 異言語間対話支援における誤訳警告メッセージの有効性調査

    LI Yunmeng, 鈴木潤, 鈴木潤, 森下睦, 阿部香央莉, 阿部香央莉, 乾健太郎, 乾健太郎, 乾健太郎

    言語処理学会年次大会発表論文集(Web)　30th　2024

    ISSN： 2188-4420

  4. Integrated Gradientsにおける理想の積分ステップ数はインスタンス毎に異なる

    牧野雅紘, 浅妻佑弥, 浅妻佑弥, 佐々木翔大, 佐々木翔大, 鈴木潤, 鈴木潤

    言語処理学会年次大会発表論文集(Web)　30th　2024

    ISSN： 2188-4420

  5. 事実正誤判定が不要な生成応答の検出に向けたデータセットの収集と分析

    亀井遼平, 塩野大輝, 赤間怜奈, 赤間怜奈, 鈴木潤, 鈴木潤

    言語処理学会年次大会発表論文集(Web)　30th　2024

    ISSN： 2188-4420

  6. InstructDoc:自然言語指示に基づく視覚的文書理解

    田中涼太, 田中涼太, 壱岐太一, 西田京介, 齋藤邦子, 鈴木潤

    言語処理学会年次大会発表論文集(Web)　30th　2024

    ISSN： 2188-4420

  7. 日本語日常対話コーパスへの基礎解析アノテーション

    赤間怜奈, 赤間怜奈, 浅原正幸, 若狭絢, 大村舞, 鈴木潤, 鈴木潤

    言語処理学会年次大会発表論文集(Web)　30th　2024

    ISSN： 2188-4420

  8. 雑談応答生成モデルによる矛盾応答の大規模収集

    佐藤志貴, 赤間怜奈, 赤間怜奈, 鈴木潤, 鈴木潤, 乾健太郎, 乾健太郎, 乾健太郎

    言語処理学会年次大会発表論文集(Web)　30th　2024

    ISSN： 2188-4420

  9. 英語中心の大規模言語モデルの言語横断汎化能力

    謝素春, 佐々木翔大, LI Yunmeng, 坂田将樹, 坂田将樹, 赤間怜奈, 赤間怜奈, 鈴木潤, 鈴木潤

    言語処理学会年次大会発表論文集(Web)　30th　2024

    ISSN： 2188-4420

  10. 文脈内学習における文脈内事例の寄与度推定

    葉夢宇, 栗林樹生, 小林悟郎, 小林悟郎, 鈴木潤, 鈴木潤

    言語処理学会年次大会発表論文集(Web)　30th　2024

    ISSN： 2188-4420

  11. 大規模視覚言語モデルに関する指示追従能力の検証

    塩野大輝, 宮脇峻平, 田中涼太, 田中涼太, 鈴木潤, 鈴木潤

    言語処理学会年次大会発表論文集(Web)　30th　2024

    ISSN： 2188-4420

  12. 対話モデルに対する敵対的プロンプトの効率的な最適化

    矢野一樹, 綿岡晃輝, TRAN Thien Q., 高橋翼, LIEW Seng Pei, 鈴木潤, 鈴木潤

    言語処理学会年次大会発表論文集(Web)　30th　2024

    ISSN： 2188-4420

  13. 指示文からの画像生成における配置指定能力の調査

    木村昴, 工藤慧音, 赤間怜奈, 赤間怜奈, 鈴木潤, 鈴木潤

    言語処理学会年次大会発表論文集(Web)　29th　2023

    ISSN： 2188-4420

  14. 計算資源が限られた複数組織での出力選択による協働の検討

    伊藤郁海, 伊藤拓海, 伊藤拓海, 鈴木潤, 鈴木潤, 乾健太郎, 乾健太郎

    言語処理学会年次大会発表論文集(Web)　29th　2023

    ISSN： 2188-4420

  15. 文単位のNbest候補制約に基づく文書翻訳

    駒田啓伍, 森下睦, 鈴木潤

    言語処理学会年次大会発表論文集(Web)　29th　2023

    ISSN： 2188-4420

  16. 自然言語生成タスクの自動評価指標のためのドメイン外検出

    伊藤拓海, 森下睦, 鈴木潤

    言語処理学会年次大会発表論文集(Web)　29th　2023

    ISSN： 2188-4420

  17. Follow-up質問による矛盾応答収集の提案

    佐藤志貴, 赤間怜奈, 赤間怜奈, 鈴木潤, 鈴木潤, 乾健太郎, 乾健太郎

    言語処理学会年次大会発表論文集(Web)　29th　2023

    ISSN： 2188-4420

  18. 訓練データ中の頻度バイアスを解消する前処理の提案

    神戸隆志, 鈴木潤, 鈴木潤, 乾健太郎, 乾健太郎

    言語処理学会年次大会発表論文集(Web)　29th　2023

    ISSN： 2188-4420

  19. 画像キャプションを利用したIconQAタスクへのアプローチ

    塩野大輝, 宮脇峻平, 宮脇峻平, 長澤春希, 鈴木潤, 鈴木潤

    言語処理学会年次大会発表論文集(Web)　29th　2023

    ISSN： 2188-4420

  20. XAIにおける忠実性評価手法の考察

    牧野雅紘, 浅妻佑弥, 浅妻佑弥, 佐々木翔大, 佐々木翔大, 鈴木潤, 鈴木潤

    言語処理学会年次大会発表論文集(Web)　29th　2023

    ISSN： 2188-4420

  21. 人間同士の雑談における話題遷移モデリング

    岸波洋介, 赤間怜奈, 赤間怜奈, 佐藤志貴, 徳久良子, 鈴木潤, 鈴木潤, 乾健太郎, 乾健太郎

    言語処理学会年次大会発表論文集(Web)　29th　2023

    ISSN： 2188-4420

  22. 異言語間対話を支援する日英雑談対話誤訳検出

    李云蒙, 鈴木潤, 鈴木潤, 森下睦, 阿部香央莉, 徳久良子, ブラサール アナ, ブラサール アナ, 乾健太郎, 乾健太郎

    言語処理学会年次大会発表論文集(Web)　29th　2023

    ISSN： 2188-4420

  23. 日本語日常対話コーパスの構築

    赤間怜奈, 赤間怜奈, 磯部順子, 鈴木潤, 鈴木潤, 乾健太郎, 乾健太郎

    言語処理学会年次大会発表論文集(Web)　29th　2023

    ISSN： 2188-4420

  24. 思考連鎖指示における大規模言語モデルの否定表現理解

    葉夢宇, 栗林樹生, 栗林樹生, 舟山弘晃, 舟山弘晃, 鈴木潤, 鈴木潤

    言語処理学会年次大会発表論文集(Web)　29th　2023

    ISSN： 2188-4420

  25. 事前学習モデルを活用したEnd-to-end型動画キーフレーム物語生成法

    仲村祐希, 工藤慧音, 鈴木潤, 清水伸幸

    言語処理学会年次大会発表論文集(Web)　29th　2023

    ISSN： 2188-4420

  26. 白色化が単語埋め込みに及ぼす効果の検証

    佐々木翔大, 佐々木翔大, HEINZERLING Benjamin, HEINZERLING Benjamin, 鈴木潤, 鈴木潤, 乾健太郎, 乾健太郎

    言語処理学会年次大会発表論文集(Web)　29th　2023

    ISSN： 2188-4420

  27. Proposal for a method to predict growth of classification performance in response to increasing the amount of data for fine-tuning of Pre-Trained language models

    倉元俊輝, 鈴木潤

    人工知能学会全国大会論文集(Web)　37th　2023

    ISSN： 2758-7347

  28. Refactoring Programs Using Large Language Models with Few-Shot Examples.

    Atsushi Shirafuji, Yusuke Oda, Jun Suzuki, Makoto Morishita, Yutaka Watanobe

    CoRR　abs/2311.11690　2023

    DOI： 10.48550/arXiv.2311.11690 　

  29. Exploring the Robustness of Large Language Models for Solving Programming Problems.

    Atsushi Shirafuji, Yutaka Watanobe, Takumi Ito, Makoto Morishita, Yuki Nakamura, Yusuke Oda, Jun Suzuki

    CoRR　abs/2306.14583　2023

    DOI： 10.48550/arXiv.2306.14583 　

  30. N-best Response-based Analysis of Contradiction-awareness in Neural Response Generation Models.

    Shiki Sato, Reina Akama, Hiroki Ouchi, Ryoko Tokuhisa, Jun Suzuki, Kentaro Inui

    CoRR　abs/2208.02578　2022/08

    DOI： 10.48550/arXiv.2208.02578 　

    More details Close

    Avoiding the generation of responses that contradict the preceding context is a significant challenge in dialogue response generation. One feasible method is post-processing, such as filtering out contradicting responses from a resulting n-best response list. In this scenario, the quality of the n-best list considerably affects the occurrence of contradictions because the final response is chosen from this n-best list. This study quantitatively analyzes the contextual contradiction-awareness of neural response generation models using the consistency of the n-best lists. Particularly, we used polar questions as stimulus inputs for concise and quantitative analyses. Our tests illustrate the contradiction-awareness of recent neural response generation models and methodologies, followed by a discussion of their properties and limitations.

  31. Balancing Cost and Quality: An Exploration of Human-in-the-loop Frameworks for Automated Short Answer Scoring.

    Hiroaki Funayama, Tasuku Sato, Yuichiroh Matsubayashi, Tomoya Mizumoto, Jun Suzuki, Kentaro Inui

    CoRR　abs/2206.08288　2022/06

    DOI： 10.48550/arXiv.2206.08288 　

    More details Close

    Short answer scoring (SAS) is the task of grading short text written by a learner. In recent years, deep-learning-based approaches have substantially improved the performance of SAS models, but how to guarantee high-quality predictions still remains a critical issue when applying such models to the education field. Towards guaranteeing high-quality predictions, we present the first study of exploring the use of human-in-the-loop framework for minimizing the grading cost while guaranteeing the grading quality by allowing a SAS model to share the grading task with a human grader. Specifically, by introducing a confidence estimation method for indicating the reliability of the model predictions, one can guarantee the scoring quality by utilizing only predictions with high reliability for the scoring results and casting predictions with low reliability to human graders. In our experiments, we investigate the feasibility of the proposed framework using multiple confidence estimation methods and multiple SAS datasets. We find that our human-in-the-loop framework allows automatic scoring models and human graders to achieve the target scoring quality.

  32. On Layer Normalizations and Residual Connections in Transformers.

    Sho Takase, Shun Kiyono, Sosuke Kobayashi, Jun Suzuki

    CoRR　abs/2206.00330　2022/06

    DOI： 10.48550/arXiv.2206.00330 　

    More details Close

    In the perspective of a layer normalization (LN) position, the architecture of Transformers can be categorized into two types: Post-LN and Pre-LN. Recent Transformers prefer to select Pre-LN because the training in Post-LN with deep Transformers, e.g., ten or more layers, often becomes unstable, resulting in useless models. However, in contrast, Post-LN has also consistently achieved better performance than Pre-LN in relatively shallow Transformers, e.g., six or fewer layers. This study first investigates the reason for these discrepant observations empirically and theoretically and discovers 1, the LN in Post-LN is the source of the vanishing gradient problem that mainly leads the unstable training whereas Pre-LN prevents it, and 2, Post-LN tends to preserve larger gradient norms in higher layers during the back-propagation that may lead an effective training. Exploiting the new findings, we propose a method that can equip both higher stability and effective training by a simple modification from Post-LN. We conduct experiments on a wide range of text generation tasks and demonstrate that our method outperforms Pre-LN, and stable training regardless of the shallow or deep layer settings.

  33. Diverse Lottery Tickets Boost Ensemble from a Single Pretrained Model.

    Sosuke Kobayashi, Shun Kiyono, Jun Suzuki, Kentaro Inui

    CoRR　abs/2205.11833　2022/05

    DOI： 10.48550/arXiv.2205.11833 　

    More details Close

    Ensembling is a popular method used to improve performance as a last resort. However, ensembling multiple models finetuned from a single pretrained model has been not very effective; this could be due to the lack of diversity among ensemble members. This paper proposes Multi-Ticket Ensemble, which finetunes different subnetworks of a single pretrained model and ensembles them. We empirically demonstrated that winning-ticket subnetworks produced more diverse predictions than dense networks, and their ensemble outperformed the standard ensemble on some tasks.

  34. Towards Automated Document Revision: Grammatical Error Correction, Fluency Edits, and Beyond.

    Masato Mita, Keisuke Sakaguchi, Masato Hagiwara, Tomoya Mizumoto, Jun Suzuki, Kentaro Inui

    CoRR　abs/2205.11484　2022/05

    DOI： 10.48550/arXiv.2205.11484 　

    More details Close

    Natural language processing technology has rapidly improved automated grammatical error correction tasks, and the community begins to explore document-level revision as one of the next challenges. To go beyond sentence-level automated grammatical error correction to NLP-based document-level revision assistant, there are two major obstacles: (1) there are few public corpora with document-level revisions being annotated by professional editors, and (2) it is not feasible to elicit all possible references and evaluate the quality of revision with such references because there are infinite possibilities of revision. This paper tackles these challenges. First, we introduce a new document-revision corpus, TETRA, where professional editors revised academic papers sampled from the ACL anthology which contain few trivial grammatical errors that enable us to focus more on document- and paragraph-level edits such as coherence and consistency. Second, we explore reference-less and interpretable methods for meta-evaluation that can detect quality improvements by document revision. We show the uniqueness of TETRA compared with existing document revision corpora and demonstrate that a fine-tuned pre-trained language model can discriminate the quality of documents after revision even when the difference is subtle. This promising result will encourage the community to further explore automated document revision models and metrics in future.

  35. JParaCrawl v3.0:大規模日英対訳コーパス

    森下睦, 森下睦, 帖佐克己, 鈴木潤, 永田昌明

    言語処理学会年次大会発表論文集(Web)　28th　2022/03

    ISSN： 2188-4420

  36. 文字情報を考慮したシーン画像検索

    宮脇峻平, 長谷川拓, 西田京介, 加藤拓真, 鈴木潤, 鈴木潤

    言語処理学会年次大会発表論文集(Web)　28th　2022/03

    ISSN： 2188-4420

  37. 多様な話者との自動対話に基づく雑談システムの自動評価

    佐藤志貴, 岸波洋介, 杉山弘晃, 赤間怜奈, 赤間怜奈, 徳久良子, 鈴木潤, 鈴木潤

    言語処理学会年次大会発表論文集(Web)　28th　2022/03

    ISSN： 2188-4420

  38. シフト付き絶対位置埋め込み

    清野舜, 清野舜, 小林颯介, 小林颯介, 鈴木潤, 鈴木潤, 乾健太郎, 乾健太郎

    言語処理学会年次大会発表論文集(Web)　28th　2022/03

    ISSN： 2188-4420

  39. 動画キーフレーム物語生成手法の提案

    佐藤俊, 佐藤汰亮, 鈴木潤, 清水伸幸

    言語処理学会年次大会発表論文集(Web)　28th　2022/03

    ISSN： 2188-4420

  40. 記述式答案自動採点における確信度推定とその役割

    舟山弘晃, 舟山弘晃, 佐藤汰亮, 佐藤汰亮, 松林優一郎, 松林優一郎, 水本智也, 水本智也, 鈴木潤, 鈴木潤, 乾健太郎, 乾健太郎

    言語処理学会年次大会発表論文集(Web)　28th　2022/03

    ISSN： 2188-4420

  41. 規範的な日本語日常対話コーパスの設計

    赤間怜奈, 赤間怜奈, 磯部順子, 磯部順子, 鈴木潤, 鈴木潤, 乾健太郎, 乾健太郎

    言語処理学会年次大会発表論文集(Web)　28th　2022/03

    ISSN： 2188-4420

  42. Transformerを多層にする際の勾配消失問題と解決法について

    高瀬翔, 清野舜, 清野舜, 小林颯介, 小林颯介, 鈴木潤, 鈴木潤

    言語処理学会年次大会発表論文集(Web)　28th　2022/03

    ISSN： 2188-4420

  43. 論述リビジョンのためのメタ評価基盤

    三田雅人, 坂口慶祐, 萩原正人, 萩原正人, 水本智也, 水本智也, 鈴木潤, 鈴木潤, 乾健太郎, 乾健太郎

    言語処理学会年次大会発表論文集(Web)　28th　2022/03

    ISSN： 2188-4420

  44. ニューラル言語モデルの効率的な学習に向けた代表データ集合の獲得

    鈴木潤, 鈴木潤, 全炳河, 賀沢秀人

    言語処理学会年次大会発表論文集(Web)　28th　2022/03

    ISSN： 2188-4420

  45. JParaCrawl v3.0: A Large-scale English-Japanese Parallel Corpus.

    Makoto Morishita, Katsuki Chousa, Jun Suzuki, Masaaki Nagata

    CoRR　abs/2202.12607　2022/02

    More details Close

    Most current machine translation models are mainly trained with parallel corpora, and their translation accuracy largely depends on the quality and quantity of the corpora. Although there are billions of parallel sentences for a few language pairs, effectively dealing with most language pairs is difficult due to a lack of publicly available parallel corpora. This paper creates a large parallel corpus for English-Japanese, a language pair for which only limited resources are available, compared to such resource-rich languages as English-German. It introduces a new web-based English-Japanese parallel corpus named JParaCrawl v3.0. Our new corpus contains more than 21 million unique parallel sentence pairs, which is more than twice as many as the previous JParaCrawl v2.0 corpus. Through experiments, we empirically show how our new corpus boosts the accuracy of machine translation models on various domains. The JParaCrawl v3.0 corpus will eventually be publicly available online for research purposes.

  46. Bipartite-play Dialogue Collection for Practical Automatic Evaluation of Dialogue Systems.

    Shiki Sato, Yosuke Kishinami, Hiroaki Sugiyama, Reina Akama, Ryoko Tokuhisa, Jun Suzuki

    CoRR　abs/2211.10596　2022

    DOI： 10.48550/arXiv.2211.10596 　

  47. Target-Guided Open-Domain Conversation Planning.

    Yosuke Kishinami, Reina Akama, Shiki Sato, Ryoko Tokuhisa, Jun Suzuki, Kentaro Inui

    CoRR　abs/2209.09746　2022

    DOI： 10.48550/arXiv.2209.09746 　

  48. Subgoal-Guided Dialogue Self-Play

    岸波洋介, 赤間怜奈, 赤間怜奈, 佐藤志貴, 徳久良子, 鈴木潤, 鈴木潤, 乾健太郎, 乾健太郎

    人工知能学会言語・音声理解と対話処理研究会資料　93　70-73　2021/11/20

    Publisher: 一般社団法人 人工知能学会

    DOI： 10.11517/jsaislud.93.0_70 　

    ISSN： 0918-5682

    eISSN： 2436-4576

  49. Live Competition: AI King—Quiz AI Japan Championship— Invited

    Suzuki Jun, Matshuda Koji, Suzuki Masatoshi, Kato Takuma, Miyawaki Shumpei, Nishida Kyosuke

    Journal of Natural Language Processing　28　(3)　888-894　2021/09

    Publisher: The Association for Natural Language Processing

    DOI： 10.5715/jnlp.28.888 　

    ISSN： 1340-7619

    eISSN： 2185-8314

  50. 単語埋め込みの決定的縮約

    仲村祐希, 鈴木潤, 高橋諒, 乾健太郎

    言語処理学会年次大会発表論文集(Web)　27th　2021/03

    ISSN： 2188-4420

  51. オープンドメインQAにおけるDPRの有効性検証

    加藤拓真, 宮脇峻平, 西田京介, 鈴木潤

    言語処理学会年次大会発表論文集(Web)　27th　2021/03

    ISSN： 2188-4420

  52. 単一事例エキスパートの統合によるドメイン適応

    清野舜, 清野舜, 小林颯介, 鈴木潤, 乾健太郎

    言語処理学会年次大会発表論文集(Web)　27th　2021/03

    ISSN： 2188-4420

  53. オープンドメイン質問応答における解答可能性判別の役割

    鈴木正敏, 松田耕史, 大内啓樹, 鈴木潤, 乾健太郎

    言語処理学会年次大会発表論文集(Web)　27th　2021/03

    ISSN： 2188-4420

  54. 事例ベース依存構造解析のための依存関係表現学習

    大内啓樹, 鈴木潤, 小林颯介, 横井祥, 栗林樹生, 吉川将司, 乾健太郎

    言語処理学会年次大会発表論文集(Web)　27th　2021/03

    ISSN： 2188-4420

  55. 対話システムの先読み能力を分析可能なタスクの検討

    岸波洋介, 赤間怜奈, 佐藤志貴, 鈴木潤, 乾健太郎

    言語処理学会年次大会発表論文集(Web)　27th　2021/03

    ISSN： 2188-4420

  56. クラウドソーシングによるWebサイトマイニングを用いた翻訳モデルの即時領域適応

    森下睦, 鈴木潤, 永田昌明

    言語処理学会年次大会発表論文集(Web)　27th　2021/03

    ISSN： 2188-4420

  57. 対話システムの矛盾応答の生成に対する脆弱性の分析

    佐藤志貴, 赤間怜奈, 大内啓樹, 鈴木潤, 乾健太郎

    言語処理学会年次大会発表論文集(Web)　27th　2021/03

    ISSN： 2188-4420

  58. Langsmith:人とシステムの協働による論文執筆

    伊藤拓海, 栗林樹生, 日高雅俊, 鈴木潤, 乾健太郎

    言語処理学会年次大会発表論文集(Web)　27th　2021/03

    ISSN： 2188-4420

  59. An Investigation Between Schema Linking and Text-to-SQL Performance

    Yasufumi Taniguchi, Hiroki Nakayama, Kubo Takahiro, Jun Suzuki

    2021/02/03

    More details Close

    Text-to-SQL is a crucial task toward developing methods for understanding natural language by computers. Recent neural approaches deliver excellent performance; however, models that are difficult to interpret inhibit future developments. Hence, this study aims to provide a better approach toward the interpretation of neural models. We hypothesize that the internal behavior of models at hand becomes much easier to analyze if we identify the detailed performance of schema linking simultaneously as the additional information of the text-to-SQL performance. We provide the ground-truth annotation of schema linking information onto the Spider dataset. We demonstrate the usefulness of the annotated data and how to analyze the current state-of-the-art neural models.

  60. Data-oriented Approach for Lookahead Response Generation

    KISHINAMI Yosuke, AKAMA Reina, SATO Shiki, SUZUKI Jun, TOKUHISA Ryoko, INUI Kentaro

    Proceedings of the Annual Conference of JSAI　JSAI2021　3J2GS6b02-3J2GS6b02　2021

    Publisher: The Japanese Society for Artificial Intelligence

    DOI： 10.11517/pjsai.jsai2021.0_3j2gs6b02 　

    ISSN： 2758-7347

    More details Close

    In human-human conversation, the current utterance in a dialog is often influenced by previous and future contexts. Among these, looking ahead over future context is one of the most critical factors for active conversation. In this paper, we propose a novel training strategy to help neural response generation models generate responses that take into account information from the future context. Our training strategy considers a sequence consisting of the response and its future context as an output sequence, and the model learns to generate the output sequence from an input sequence, i.e., past utterances. In our experiments, we investigate the effect of the proposed strategy on the look-ahead ability of a dialog system via the "Lookahead Chit Chat Task."

  61. A Proposal of Video Key-frame Captioning Task and its Dataset Construction

    KITAYAMA Kotaro, SUZUKI Jun, SHIMIZU Nobuyuki

    Proceedings of the Annual Conference of JSAI　JSAI2021　4I4GS7e03-4I4GS7e03　2021

    Publisher: The Japanese Society for Artificial Intelligence

    DOI： 10.11517/pjsai.jsai2021.0_4i4gs7e03 　

    ISSN： 2758-7347

    More details Close

    Automatic video summarization is one of the crucial technologies to alleviate the cost of developers and end-usersto check the contents of videos. Moreover, it can also work as clues of video retrieval to only obtain required videosfrom extremely many consumer-generated videos. This paper specifically focuses on a video summarization task,which we callvideo key-frame captioning. This task requires systems to extract a predefined number of key-framesand simultaneously generate a description of the series of extracted key-frames that summarize the given video well.We introduce a formal task definition of our new task and discuss procedures for creating a dataset for evaluationof key-frame captioning tasks.

  62. NeurIPS 2020 EfficientQA Competition: Systems, Analyses and Lessons Learned

    Sewon Min, Jordan Boyd-Graber, Chris Alberti, Danqi Chen, Eunsol Choi, Michael Collins, Kelvin Guu, Hannaneh Hajishirzi, Kenton Lee, Jennimaria Palomaki, Colin Raffel, Adam Roberts, Tom Kwiatkowski, Patrick Lewis, Yuxiang Wu, Heinrich Küttler, Linqing Liu, Pasquale Minervini, Pontus Stenetorp, Sebastian Riedel, Sohee Yang, Minjoon Seo, Gautier Izacard, Fabio Petroni, Lucas Hosseini, Nicola De Cao, Edouard Grave, Ikuya Yamada, Sonse Shimaoka, Masatoshi Suzuki, Shumpei Miyawaki, Shun Sato, Ryo Takahashi, Jun Suzuki, Martin Fajcik, Martin Docekal, Karel Ondrej, Pavel Smrz, Hao Cheng, Yelong Shen, Xiaodong Liu, Pengcheng He, Weizhu Chen, Jianfeng Gao, Barlas Oguz, Xilun Chen, Vladimir Karpukhin, Stan Peshterliev, Dmytro Okhonko, Michael Schlichtkrull, Sonal Gupta, Yashar Mehdad, Wen-tau Yih

    2021/01/01

    More details Close

    We review the EfficientQA competition from NeurIPS 2020. The competition focused on open-domain question answering (QA), where systems take natural language questions as input and return natural language answers. The aim of the competition was to build systems that can predict correct answers while also satisfying strict on-disk memory budgets. These memory budgets were designed to encourage contestants to explore the trade-off between storing retrieval corpora or the parameters of learned models. In this report, we describe the motivation and organization of the competition, review the best submissions, and analyze system predictions to inform a discussion of evaluation for open-domain QA.

  63. Efficient Estimation of Influence of a Training Instance

    Sosuke Kobayashi, Sho Yokoi, Jun Suzuki, Kentaro Inui

    自然言語処理(Web)　28　(2)　2020/12/08

    ISSN： 2185-8314

  64. ILYS aoba bot: A Chatbot Combining Rules and Large-Scale Neural Response Generation

    藤原吏生, 岸波洋介, 今野颯人, 佐藤志貴, 佐藤汰亮, 宮脇峻平, 加藤拓真, 鈴木潤, 鈴木潤, 乾健太郎, 乾健太郎

    人工知能学会言語・音声理解と対話処理研究会資料　90th　2020/12

    ISSN： 0918-5682

  65. PheMT: A Phenomenon-wise Dataset for Machine Translation Robustness on User-Generated Contents

    Ryo Fujii, Masato Mita, Kaori Abe, Kazuaki Hanawa, Makoto Morishita, Jun Suzuki, Kentaro Inui

    2020/11/04

  66. Langsmith: An Interactive Academic Text Revision System

    Takumi Ito, Tatsuki Kuribayashi, Masatoshi Hidaka, Jun Suzuki, Kentaro Inui

    2020/10/09

  67. A Self-Refinement Strategy for Noise Reduction in Grammatical Error Correction

    Masato Mita, Shun Kiyono, Masahiro Kaneko, Jun Suzuki, Kentaro Inui

    2020/10/07

  68. Embeddings of Label Components for Sequence Labeling: A Case Study of Fine-grained Named Entity Recognition

    Takuma Kato, Kaori Abe, Hiroki Ouchi, Shumpei Miyawaki, Jun Suzuki, Kentaro Inui

    2020/06/02

  69. Encoder-Decoder Models Can Benefit from Pre-trained Masked Language Models in Grammatical Error Correction

    Masahiro Kaneko, Masato Mita, Shun Kiyono, Jun Suzuki, Kentaro Inui

    2020/05/03

  70. Language Models as an Alternative Evaluator of Word Order Hypotheses: A Case Study in Japanese

    Tatsuki Kuribayashi, Takumi Ito, Jun Suzuki, Kentaro Inui

    2020/05/02

  71. Single Model Ensemble using Pseudo-Tags and Distinct Vectors

    Ryosuke Kuwabara, Jun Suzuki, Hideki Nakayama

    2020/05/02

  72. Language-independent Dialogue Data Filtering for Neural Dialogue Response Generation

    AKAMA Reina, YOKOI Sho, SUZUKI Jun, KENTARO Inui

    人工知能学会全国大会(Web)　34th　4Q2GS902-4Q2GS902　2020/05

    Publisher: The Japanese Society for Artificial Intelligence

    DOI： 10.11517/pjsai.jsai2020.0_4q2gs902 　

    More details Close

    In the area of sentence generation using deep neural network technology, e.g., machine translation, automatic summarization, and dialog response generation, approaches to increase the performance of models by improving the quality of training data have been spotlighted. In this paper, we propose a scoring function that detects low-quality utterance-response pairs in training data to improve the performance of a neural dialogue response generation model. Specifically, our function combines two viewpoints, "typical phrase interconnection" and "topic consistency", to rate the plausibility of two consecutive utterances as dialogue. In our experiments, we apply the proposed method to conversation data in multiple languages and demonstrate that the proposed score is correlated with human subjective ratings. Moreover, we demonstrate that training data filtering with our score is effective for improving the performance of response generation models using automatic evaluation and manual evaluation.

  73. Systematic Analysis of Linguistic Phenomena for Better Understanding Translation on User-Generated Contents

    FUJII Ryo, MITA Masato, ABE Kaori, HANAWA Kazuaki, MORISHITA Makoto, SUZUKI Jun, INUI Kentaro

    人工知能学会全国大会(Web)　34th　3Rin426-3Rin426　2020/05

    Publisher: The Japanese Society for Artificial Intelligence

    DOI： 10.11517/pjsai.jsai2020.0_3rin426 　

    ISSN： 2758-7347

    More details Close

    Neural Machine Translation (NMT) has shown drastic improvement on its quality when translating clean input. However, it still struggles with some kind of input with plentiful of noises, like User-Generated Contents (UGC) on the Internet. In order to make NMT systems indeed useful in promoting cross-cultural communication, one of the most promising direction we have to follow is to correctly handle with these input. Though necessary, it is still an open question that what brings the great gap of performance between translation of clean input and UGC. In this paper, we conducted systematic analysis on current dataset focusing on UGC and made it clear which linguistic phenomena greatly affected the translation performance.

  74. Optimal Transport Cost between Texts via Norm-Direction Decomposition

    YOKOI Sho, TAKAHASHI Ryo, AKAMA Reina, SUZUKI Jun, INUI Kentaro

    人工知能学会全国大会(Web)　34th　3Q5GS903-3Q5GS903　2020/05

    Publisher: The Japanese Society for Artificial Intelligence

    DOI： 10.11517/pjsai.jsai2020.0_3q5gs903 　

    ISSN： 2758-7347

    More details Close

    One key principle for assessing semantic textual similarity is to measure the degree of semantic overlap of two texts by considering word-by-word alignment; however, such methods are empirically inferior to methods based on generic sentence encoders. We hypothesize that the reason for the inferiority of alignment-based methods is due to the fact that they do not distinguish word importance and word meaning. To solve this, we propose to separate word importance and word meaning by decomposing word vectors into their norm and direction, then compute word-by-word alignment based similarity using optimal transport. We call the method word rotator's distance (WRD) because direction vectors are aligned by rotation on the unit hypersphere. In addition, to incorporate the advance of cutting edge additive sentence encoders, we propose to re-decompose such sentence vectors into word vectors and use them as inputs to WRD. Empirically, WRD outperforms current methods considering the word-by-word alignment including word mover's distance with a big difference; moreover, our method outperforms state-of-the-art additive sentence encoders on the most competitive dataset, STS-benchmark.

  75. Word Rotator's Distance

    Sho Yokoi, Ryo Takahashi, Reina Akama, Jun Suzuki, Kentaro Inui

    2020/04/30

  76. Instance-Based Learning of Span Representations: A Case Study through Named Entity Recognition

    Hiroki Ouchi, Jun Suzuki, Sosuke Kobayashi, Sho Yokoi, Tatsuki Kuribayashi, Ryuto Konno, Kentaro Inui

    2020/04/29

  77. Filtering Noisy Dialogue Corpora by Connectivity and Content Relatedness

    Reina Akama, Sho Yokoi, Jun Suzuki, Kentaro Inui

    2020/04/29

  78. Evaluating Dialogue Generation Systems via Response Selection

    Shiki Sato, Reina Akama, Hiroki Ouchi, Jun Suzuki, Kentaro Inui

    自然言語処理(Web)　29　(1)　2020/04/29

    ISSN： 2185-8314

  79. JParaCrawl:大規模Webベース日英対訳コーパス

    森下睦, 鈴木潤, 永田昌明

    言語処理学会年次大会発表論文集(Web)　26th　2020/03

    ISSN： 2188-4420

  80. 日本語語順分析に言語モデルを用いることの妥当性について

    栗林樹生, 伊藤拓海, 鈴木潤, 乾健太郎

    言語処理学会年次大会発表論文集(Web)　26th　2020/03

    ISSN： 2188-4420

  81. テキストを通して世界を見る:機械読解における常識的推論のための画像説明文の評価

    GALVAN-SOSA Diana, 西田京介, 松田耕史, 鈴木潤, 乾健太郎

    言語処理学会年次大会発表論文集(Web)　26th　2020/03

    ISSN： 2188-4420

  82. JAQKET:クイズを題材にした日本語QAデータセットの構築

    鈴木正敏, 鈴木潤, 松田耕史, 西田京介, 井之上直也

    言語処理学会年次大会発表論文集(Web)　26th　2020/03

    ISSN： 2188-4420

  83. 文法誤り訂正のための自己改良戦略に基づくノイズ除去

    三田雅人, 清野舜, 金子正弘, 鈴木潤, 乾健太郎

    言語処理学会年次大会発表論文集(Web)　26th　2020/03

    ISSN： 2188-4420

  84. 擬似タグと線形移動ベクトルを用いた単一モデルによる擬似モデルアンサンブル

    桑原亮介, 鈴木潤, 中山英樹

    言語処理学会年次大会発表論文集(Web)　26th　2020/03

    ISSN： 2188-4420

  85. スパン間の類似性に基づく事例ベース構造予測

    大内啓樹, 鈴木潤, 小林颯介, 横井祥, 栗林樹生, 乾健太郎

    言語処理学会年次大会発表論文集(Web)　26th　2020/03

    ISSN： 2188-4420

  86. 評価データのクラスタリングを用いた記述式答案自動採点のためのトランズダクティブ学習

    佐藤俊, 佐々木翔大, 大内啓樹, 鈴木潤, 乾健太郎

    言語処理学会年次大会発表論文集(Web)　26th　2020/03

    ISSN： 2188-4420

  87. 大規模疑似データを用いた高性能文法誤り訂正モデルの構築

    清野舜, 鈴木潤, 三田雅人, 水本智也, 乾健太郎

    言語処理学会年次大会発表論文集(Web)　26th　2020/03

    ISSN： 2188-4420

  88. 単一評価サンプルのためのトランズダクティブ学習

    佐々木翔大, 大内啓樹, 鈴木潤, BRASSARD Ana, 乾健太郎

    言語処理学会年次大会発表論文集(Web)　26th　2020/03

    ISSN： 2188-4420

  89. 超球面上での最適輸送に基づく文類似性尺度

    横井祥, 高橋諒, 赤間怜奈, 鈴木潤, 乾健太郎

    言語処理学会年次大会発表論文集(Web)　26th　2020/03

    ISSN： 2188-4420

  90. ベクトル長に基づく自己注意機構の解析

    小林悟郎, 栗林樹生, 横井祥, 鈴木潤, 乾健太郎

    言語処理学会年次大会発表論文集(Web)　26th　2020/03

    ISSN： 2188-4420

  91. Text-to-SQLにおけるSQL構文に着目したデータ拡張手法

    谷口泰史, 中山光樹, 久保隆宏, 鈴木潤

    言語処理学会年次大会発表論文集(Web)　26th　2020/03

    ISSN： 2188-4420

  92. 記述式答案自動採点のための確信度推定手法の検討

    舟山弘晃, 佐々木翔大, 水本智也, 三田雅人, 鈴木潤, 松林優一郎, 乾健太郎

    言語処理学会年次大会発表論文集(Web)　26th　2020/03

    ISSN： 2188-4420

  93. 対話応答選択による対話応答生成モデルの評価

    佐藤志貴, 赤間怜奈, 大内啓樹, 鈴木潤, 乾健太郎

    言語処理学会年次大会発表論文集(Web)　26th　2020/03

    ISSN： 2188-4420

  94. 句の呼応と話題の一貫性に着目した低品質対話データの教師なしフィルタリング

    赤間怜奈, 鈴木潤, 横井祥, 乾健太郎, 赤間怜奈, 鈴木潤, 横井祥, 乾健太郎

    言語処理学会年次大会発表論文集(Web)　26th　2020

    ISSN： 2188-4420

  95. JParaCrawl: A Large Scale Web-Based English-Japanese Parallel Corpus

    Makoto Morishita, Jun Suzuki, Masaaki Nagata

    LREC　6704-6710　2019/11/25

    More details Close

    Recent machine translation algorithms mainly rely on parallel corpora. However, since the availability of parallel corpora remains limited, only some resource-rich language pairs can benefit from them. We constructed a parallel corpus for English-Japanese, for which the amount of publicly available parallel corpora is still limited. We constructed the parallel corpus by broadly crawling the web and automatically aligning parallel sentences. Our collected corpus, called JParaCrawl, amassed over 8.7 million sentence pairs. We show how it includes a broader range of domains and how a neural machine translation model trained with it works as a good pre-trained model for fine-tuning specific domains. The pre-training and fine-tuning approaches achieved or surpassed performance comparable to model training from the initial state and reduced the training time. Additionally, we trained the model with an in-domain dataset and JParaCrawl to show how we achieved the best performance with them. JParaCrawl and the pre-trained models are freely available online for research purposes.

  96. Diamonds in the Rough: Generating Fluent Sentences from Early-Stage Drafts for Academic Writing Assistance

    Takumi Ito, Tatsuki Kuribayashi, Hayato Kobayashi, Ana Brassard, Masato Hagiwara, Jun Suzuki, Kentaro Inui

    2019/10/21

    More details Close

    The writing process consists of several stages such as drafting, revising, editing, and proofreading. Studies on writing assistance, such as grammatical error correction (GEC), have mainly focused on sentence editing and proofreading, where surface-level issues such as typographical, spelling, or grammatical errors should be corrected. We broaden this focus to include the earlier revising stage, where sentences require adjustment to the information included or major rewriting and propose Sentence-level Revision (SentRev) as a new writing assistance task. Well-performing systems in this task can help inexperienced authors by producing fluent, complete sentences given their rough, incomplete drafts. We build a new freely available crowdsourced evaluation dataset consisting of incomplete sentences authored by non-native writers paired with their final versions extracted from published academic papers for developing and evaluating SentRev models. We also establish baseline performance on SentRev using our newly built evaluation dataset.

  97. Select and Attend: Towards Controllable Content Selection in Text Generation

    Xiaoyu Shen, Jun Suzuki, Kentaro Inui, Hui Su, Dietrich Klakow, Satoshi Sekine

    2019/09/10

    More details Close

    Many text generation tasks naturally contain two steps: content selection and surface realization. Current neural encoder-decoder models conflate both steps into a black-box architecture. As a result, the content to be described in the text cannot be explicitly controlled. This paper tackles this problem by decoupling content selection from the decoder. The decoupled content selection is human interpretable, whose value can be manually manipulated to control the content of generated text. The model can be trained end-to-end without human annotations by maximizing a lower bound of the marginal likelihood. We further propose an effective way to trade-off between performance and controllability with a single adjustable hyperparameter. In both data-to-text and headline generation tasks, our model achieves promising results, paving the way for controllable content selection in text generation.

  98. TEASPN: Framework and Protocol for Integrated Writing Assistance Environments

    Masato Hagiwara, Takumi Ito, Tatsuki Kuribayashi, Jun Suzuki, Kentaro Inui

    2019/09/05

    More details Close

    Language technologies play a key role in assisting people with their writing. Although there has been steady progress in e.g., grammatical error correction (GEC), human writers are yet to benefit from this progress due to the high development cost of integrating with writing software. We propose TEASPN, a protocol and an open-source framework for achieving integrated writing assistance environments. The protocol standardizes the way writing software communicates with servers that implement such technologies, allowing developers and researchers to integrate the latest developments in natural language processing (NLP) with low cost. As a result, users can enjoy the integrated experience in their favorite writing software. The results from experiments with human participants show that users use a wide range of technologies and rate their writing experience favorably, allowing them to write more fluent text.

  99. An Empirical Study of Incorporating Pseudo Data into Grammatical Error Correction

    Shun Kiyono, Jun Suzuki, Masato Mita, Tomoya Mizumoto, Kentaro Inui

    2019/09/02

    More details Close

    The incorporation of pseudo data in the training of grammatical error correction models has been one of the main factors in improving the performance of such models. However, consensus is lacking on experimental configurations, namely, choosing how the pseudo data should be generated or used. In this study, these choices are investigated through extensive experiments, and state-of-the-art performance is achieved on the CoNLL-2014 test set ($F_{0.5}=65.0$) and the official test set of the BEA-2019 shared task ($F_{0.5}=70.2$) without making any modifications to the model architecture.

  100. Character n-gram Embeddings to Improve RNN Language Models

    Sho Takase, Jun Suzuki, Masaaki Nagata

    2019/06/13

    More details Close

    This paper proposes a novel Recurrent Neural Network (RNN) language model that takes advantage of character information. We focus on character n-grams based on research in the field of word embedding construction (Wieting et al. 2016). Our proposed method constructs word embeddings from character n-gram embeddings and combines them with ordinary word embeddings. We demonstrate that the proposed method achieves the best perplexities on the language modeling datasets: Penn Treebank, WikiText-2, and WikiText-103. Moreover, we conduct experiments on application tasks: machine translation and headline generation. The experimental results indicate that our proposed method also positively affects these tasks.

  101. Personalized Visited-POI Assignment to Individual Raw GPS Trajectories

    Jun Suzuki, Yoshihiko Suhara, Hiroyuki Toda, Kyosuke Nishida

    2019/01/11

    DOI： 10.1145/3317667 　

    More details Close

    Knowledge discovery from GPS trajectory data is an important topic in several scientific areas, including data mining, human behavior analysis, and user modeling. This paper proposes a task that assigns personalized visited-POIs. Its goal is to estimate fine-grained and pre-defined locations (i.e., points of interest (POI)) that are actually visited by users and assign visited-location information to the corresponding span of their (personal) GPS trajectories. We also introduce a novel algorithm to solve this assignment task. First, we exhaustively extract stay-points as candidates for significant locations using a variant of a conventional stay-point extraction method. Then we select significant locations and simultaneously assign visited-POIs to them by considering various aspects, which we formulate in integer linear programming. Experimental results conducted on an actual user dataset show that our method achieves higher accuracy in the visited-POI assignment task than the various cascaded procedures of conventional methods.

  102. フレーズ単位の発話応答ペアを用いた対話応答生成の多様化

    佐藤志貴, 大内啓樹, 大内啓樹, 井之上直也, 井之上直也, 鈴木潤, 鈴木潤, 乾健太郎, 乾健太郎

    言語処理学会年次大会発表論文集(Web)　25th　2019

    ISSN： 2188-4420

  103. 強化学習によるプログラム生成のためのプログラム系列分析

    佐藤拓海, 大内啓樹, 大内啓樹, 松田耕史, 鈴木正敏, 鈴木潤, 鈴木潤, 乾健太郎, 乾健太郎

    言語処理学会年次大会発表論文集(Web)　25th　2019

    ISSN： 2188-4420

  104. 独立発話の繋ぎ合わせによる発話-応答ペアの獲得

    赤間怜奈, 武藤由依, 鈴木潤, 鈴木潤, 乾健太郎, 乾健太郎

    言語処理学会年次大会発表論文集(Web)　25th　2019

    ISSN： 2188-4420

  105. 意味役割付与におけるトランズダクティブ分野適応

    大内啓樹, 大内啓樹, 鈴木潤, 鈴木潤, 乾健太郎, 乾健太郎

    言語処理学会年次大会発表論文集(Web)　25th　2019

    ISSN： 2188-4420

  106. ライティング支援を想定した情報補完型生成

    伊藤拓海, 伊藤拓海, 栗林樹生, 栗林樹生, 小林隼人, 小林隼人, 鈴木潤, 鈴木潤, 乾健太郎, 乾健太郎

    言語処理学会年次大会発表論文集(Web)　25th　2019

    ISSN： 2188-4420

  107. サブワードに基づく単語分散表現の縮約モデリング

    佐々木翔大, 鈴木潤, 鈴木潤, 乾健太郎, 乾健太郎

    言語処理学会年次大会発表論文集(Web)　25th　2019

    ISSN： 2188-4420

  108. ニューラル機械翻訳における文脈情報の選択的利用

    藤井諒, 清野舜, 清野舜, 鈴木潤, 鈴木潤, 乾健太郎, 乾健太郎

    言語処理学会年次大会発表論文集(Web)　25th　2019

    ISSN： 2188-4420

  109. ExpertとImitatorの混合ネットワークによる大規模半教師あり学習

    清野舜, 清野舜, 鈴木潤, 鈴木潤, 乾健太郎, 乾健太郎

    言語処理学会年次大会発表論文集(Web)　25th　2019

    ISSN： 2188-4420

  110. 双方向学習と再現学習を統合したニューラル機械翻訳

    森下睦, 鈴木潤, 永田昌明

    言語処理学会年次大会発表論文集(Web)　25th　2019

    ISSN： 2188-4420

  111. 画像/言語同時埋め込みベクトル空間の構築に向けた埋め込み粒度の比較検討

    北山晃太郎, 清野舜, 清野舜, 鈴木潤, 鈴木潤, 乾健太郎, 乾健太郎

    言語処理学会年次大会発表論文集(Web)　25th　2019

    ISSN： 2188-4420

  112. 言語モデルを用いた日本語の語順評価と基本語順の分析

    栗林樹生, 栗林樹生, 伊藤拓海, 伊藤拓海, 内山香, 鈴木潤, 鈴木潤, 乾健太郎, 乾健太郎

    言語処理学会年次大会発表論文集(Web)　25th　2019

    ISSN： 2188-4420

  113. 複数の言語単位に対するスパン表現を用いた論述構造解析

    栗林樹生, 大内啓樹, 大内啓樹, 井之上直也, 井之上直也, REISERT Paul, 三好利昇, 三好利昇, 鈴木潤, 鈴木潤, 乾健太郎, 乾健太郎

    言語処理学会年次大会発表論文集(Web)　25th　2019

    ISSN： 2188-4420

  114. 文法誤り訂正における反復訂正の効果検証

    浅野広樹, 浅野広樹, 鈴木潤, 鈴木潤, 水本智也, 乾健太郎, 乾健太郎

    言語処理学会年次大会発表論文集(Web)　25th　2019

    ISSN： 2188-4420

  115. 計算機科学論文における手法の利点・欠点に着目したデータの構築と分析

    白井穂乃, 井之上直也, 井之上直也, 鈴木潤, 鈴木潤, 乾健太郎, 乾健太郎

    言語処理学会年次大会発表論文集(Web)　25th　2019

    ISSN： 2188-4420

  116. クイズ解答タスクにおける大規模ラベルなしコーパスの利用:言語モデルとデータ拡張

    鈴木正敏, 松田耕史, 松田耕史, 大内啓樹, 大内啓樹, 鈴木潤, 鈴木潤, 乾健太郎, 乾健太郎

    情報処理学会研究報告(Web)　2019　(NL-241)　2019

  117. Construction of Coherent Translation Evaluation Dataset in Machine Translation

    阿部香央莉, 鈴木潤, 鈴木潤, 鈴木潤, 永田昌明, 乾健太郎, 乾健太郎

    人工知能学会全国大会(Web)　33rd　2019

    ISSN： 2758-7347

  118. The first step to create a better response selection test set with carefully chosen false candidates

    佐藤志貴, 赤間怜奈, 赤間怜奈, 大内啓樹, 大内啓樹, 鈴木潤, 鈴木潤, 乾健太郎, 乾健太郎

    人工知能学会言語・音声理解と対話処理研究会資料　87th　24　2019

    Publisher: The Japanese Society for Artificial Intelligence

    DOI： 10.11517/jsaislud.87.0_24 　

    ISSN： 0918-5682

    eISSN： 2436-4576

  119. Mixture of Expert/Imitator Networks: Scalable Semi-supervised Learning Framework

    Shun Kiyono, Jun Suzuki, Kentaro Inui

    2018/10/13

    More details Close

    The current success of deep neural networks (DNNs) in an increasingly broad range of tasks involving artificial intelligence strongly depends on the quality and quantity of labeled training data. In general, the scarcity of labeled data, which is often observed in many natural language processing tasks, is one of the most important issues to be addressed. Semi-supervised learning (SSL) is a promising approach to overcoming this issue by incorporating a large amount of unlabeled data. In this paper, we propose a novel scalable method of SSL for text classification tasks. The unique property of our method, Mixture of Expert/Imitator Networks, is that imitator networks learn to "imitate" the estimated label distribution of the expert network over the unlabeled data, which potentially contributes a set of features for the classification. Our experiments demonstrate that the proposed method consistently improves the performance of several types of baseline DNNs. We also demonstrate that our method has the more data, better performance property with promising scalability to the amount of unlabeled data.

  120. Pointwise HSIC: A Linear-Time Kernelized Co-occurrence Norm for Sparse Linguistic Expressions

    Sho Yokoi, Sosuke Kobayashi, Kenji Fukumizu, Jun Suzuki, Kentaro Inui

    2018/09/04

    DOI： 10.18653/v1/D18-1203 　

    More details Close

    In this paper, we propose a new kernel-based co-occurrence measure that can be applied to sparse linguistic expressions (e.g., sentences) with a very short learning time, as an alternative to pointwise mutual information (PMI). As well as deriving PMI from mutual information, we derive this new measure from the Hilbert--Schmidt independence criterion (HSIC); thus, we call the new measure the pointwise HSIC (PHSIC). PHSIC can be interpreted as a smoothed variant of PMI that allows various similarity metrics (e.g., sentence embeddings) to be plugged in as kernels. Moreover, PHSIC can be estimated by simple and fast (linear in the size of the data) matrix calculations regardless of whether we use linear or nonlinear kernels. Empirically, in a dialogue response selection task, PHSIC is learned thousands of times faster than an RNN-based PMI while outperforming PMI in accuracy. In addition, we also demonstrate that PHSIC is beneficial as a criterion of a data selection task for machine translation owing to its ability to give high (low) scores to a consistent (inconsistent) pair with other pairs.

  121. Direct Output Connection for a High-Rank Language Model

    Sho Takase, Jun Suzuki, Masaaki Nagata

    2018/08/30

    More details Close

    This paper proposes a state-of-the-art recurrent neural network (RNN) language model that combines probability distributions computed not only from a final RNN layer but also from middle layers. Our proposed method raises the expressive power of a language model based on the matrix factorization interpretation of language modeling introduced by Yang et al. (2018). The proposed method improves the current state-of-the-art language model and achieves the best score on the Penn Treebank and WikiText-2, which are the standard benchmark datasets. Moreover, we indicate our proposed method contributes to two application tasks: machine translation and headline generation. Our code is publicly available at: https://github.com/nttcslab-nlp/doc_lm.

  122. Interpretable Adversarial Perturbation in Input Embedding Space for Text

    Motoki Sato, Jun Suzuki, Hiroyuki Shindo, Yuji Matsumoto

    2018/05/08

    More details Close

    Following great success in the image processing field, the idea of adversarial training has been applied to tasks in the natural language processing (NLP) field. One promising approach directly applies adversarial training developed in the image processing field to the input word embedding space instead of the discrete input space of texts. However, this approach abandons such interpretability as generating adversarial texts to significantly improve the performance of NLP tasks. This paper restores interpretability to such methods by restricting the directions of perturbations toward the existing words in the input embedding space. As a result, we can straightforwardly reconstruct each input with perturbations to an actual text by considering the perturbations to be the replacement of words in the sentence while maintaining or even improving the task performance.

  123. 自然言語処理における解釈可能な敵対的摂動の学習

    佐藤元紀, 鈴木潤, 鈴木潤, 進藤裕之, 進藤裕之, 松本裕治, 松本裕治

    言語処理学会年次大会発表論文集(Web)　24th　2018

    ISSN： 2188-4420

  124. 中間層の利用によるRNN言語モデルの表現力向上

    高瀬翔, 鈴木潤, 永田昌明

    言語処理学会年次大会発表論文集(Web)　24th　2018

    ISSN： 2188-4420

  125. 非即時的なタスク設定における固有表現抽出の改善

    澤山熱気, 鈴木潤, 鈴木潤, 進藤裕之, 進藤裕之, 松本裕治, 松本裕治

    言語処理学会年次大会発表論文集(Web)　24th　2018

    ISSN： 2188-4420

  126. Zunkobot: A Multiple Knowledge Bases Integrated Chatbot System

    阿部香央莉, 佐藤志貴, 佐藤拓海, 藤井諒, 松田耕史, 鈴木正敏, 山口健史, 赤間怜奈, 赤間怜奈, 大内啓樹, 大内啓樹, 鈴木潤, 鈴木潤, 乾健太郎, 乾健太郎

    人工知能学会言語・音声理解と対話処理研究会資料　84th　2018

    ISSN： 0918-5682

  127. ニューラルヘッドライン生成における誤生成問題の改善

    清野舜, 高瀬翔, 鈴木潤, 岡崎直観, 乾健太郎, 乾健太郎, 永田昌明

    言語処理学会年次大会発表論文集(Web)　24th　2018

    ISSN： 2188-4420

  128. Excluding the Data with Exploration from Supervised Learning Improves Neural Fictitious Self-Play

    KAWAMURA Keigo, SUZUKI Jun, TSURUOKA Yoshimasa

    Proceedings of the Annual Conference of JSAI　2018　(0)　1N301-1N301　2018

    Publisher: The Japanese Society for Artificial Intelligence

    ISSN： 1347-9881

    More details Close

    &lt;p&gt;Neural fictitious self-play (NFSP) is a method for solving imperfect information games. While methods developed in recent years such as counterfactual regret minimization or DeepStack require the state transition rules of the games, NFSP works without them. In this paper, we propose to exclude the exploration data from the supervised learning component in NFSP and keep the probability of exploration, in order to explore without breaking the average strategy. We show that this change significantly improves the performance of NFSP in a simplified poker game, Leduc Hold&#039;em, and compare the results for different exploration plobabilities.&lt;/p&gt;

  129. Source-side Prediction for Neural Headline Generation

    Shun Kiyono, Sho Takase, Jun Suzuki, Naoaki Okazaki, Kentaro Inui, Masaaki Nagata

    2017/12/22

    More details Close

    The encoder-decoder model is widely used in natural language generation tasks. However, the model sometimes suffers from repeated redundant generation, misses important phrases, and includes irrelevant entities. Toward solving these problems we propose a novel source-side token prediction module. Our method jointly estimates the probability distributions over source and target vocabularies to capture a correspondence between source and target tokens. The experiments show that the proposed model outperforms the current state-of-the-art method in the headline generation task. Additionally, we show that our method has an ability to learn a reasonable token-wise correspondence without knowing any true alignments.

  130. 自然言語解析 : 整数計画問題としての定式化と解法 (特集 自然言語処理と数理モデル)

    鈴木 潤

    オペレーションズ・リサーチ = Communications of the Operations Research Society of Japan : 経営の科学　62　(11)　703-710　2017/11

    Publisher: 日本オペレーションズ・リサーチ学会 ; 1956-

    ISSN： 0030-3674

  131. Input-to-Output Gate to Improve RNN Language Models

    Sho Takase, Jun Suzuki, Masaaki Nagata

    2017/09/26

    More details Close

    This paper proposes a reinforcing method that refines the output layers of existing Recurrent Neural Network (RNN) language models. We refer to our proposed method as Input-to-Output Gate (IOG). IOG has an extremely simple structure, and thus, can be easily combined with any RNN language models. Our experiments on the Penn Treebank and WikiText-2 datasets demonstrate that IOG consistently boosts the performance of several different types of current topline RNN language models.

  132. 単語埋め込みベクトルの圧縮法 (特集 人に迫るAI,人に寄り添うAI : corevoを支えるコミュニケーション科学)

    鈴木 潤, 永田 昌明

    NTT技術ジャーナル　29　(9)　17-20　2017/09

    Publisher: 電気通信協会

    ISSN： 0915-2318

  133. Enumeration of Extractive Oracle Summaries

    Tsutomu Hirao, Masaaki Nishino, Jun Suzuki, Masaaki Nagata

    2017/01/06

    More details Close

    To analyze the limitations and the future directions of the extractive summarization paradigm, this paper proposes an Integer Linear Programming (ILP) formulation to obtain extractive oracle summaries in terms of ROUGE-N. We also propose an algorithm that enumerates all of the oracle summaries for a set of reference summaries to exploit F-measures that evaluate which system summaries contain how many sentences that are extracted as an oracle summary. Our experimental results obtained from Document Understanding Conference (DUC) corpora demonstrated the following: (1) room still exists to improve the performance of extractive summarization; (2) the F-measures derived from the enumerated oracle summaries have significantly stronger correlations with human judgment than those derived from single oracle summaries.

  134. 単語出現頻度予測機能付きRNNエンコーダデコーダモデル

    鈴木潤, 永田昌明

    言語処理学会年次大会発表論文集(Web)　23rd　2017

    ISSN： 2188-4420

  135. 擬似データの事前学習に基づくencoder-decoder型日本語崩れ表記正規化

    斉藤いつみ, 鈴木潤, 貞光九月, 西田京介, 齋藤邦子, 松尾義博

    言語処理学会年次大会発表論文集(Web)　23rd　2017

    ISSN： 2188-4420

  136. Cutting-off Redundant Repeating Generations for Neural Abstractive Summarization

    Jun Suzuki, Masaaki Nagata

    2016/12/31

    More details Close

    This paper tackles the reduction of redundant repeating generation that is often observed in RNN-based encoder-decoder models. Our basic idea is to jointly estimate the upper-bound frequency of each target vocabulary in the encoder and control the output words based on the estimation in the decoder. Our method shows significant improvement over a strong RNN-based encoder-decoder baseline and achieved its best results on an abstractive summarization benchmark.

  137. 「NTTコミュニケーション科学基礎研究所 オープンハウス2016」開催報告

    鈴木 潤, 北川 智利, 土田 勝, 石黒 勝彦, 黒木 忍

    NTT技術ジャーナル　28　(9)　52-55　2016/09

    Publisher: 電気通信協会

    ISSN： 0915-2318

  138. 劣モジュラ関数最大化によるフレーズテーブル削減

    西野正彬, 鈴木潤, 永田昌明

    言語処理学会年次大会発表論文集(Web)　22nd　2016

    ISSN： 2188-4420

  139. 単語分散表現獲得法の縮約モデリング

    鈴木潤, 永田昌明

    言語処理学会年次大会発表論文集(Web)　22nd　2016

    ISSN： 2188-4420

  140. 逐次最適解更新による頑健な単語分散表現の学習方式

    鈴木 潤, 永田 昌明

    情報処理学会研究報告. SLP, 音声言語情報処理　2015　(16)　1-9　2015/05/18

    Publisher: 一般社団法人情報処理学会

    More details Close

    SkipGram, GloVe といった対数双線形言語モデルに属する単語分散表現のモデルは,これまで確率的勾配法 (SGD) やその拡張である AdaGrad といった勾配に基づくオンライン学習アルゴリズムを用いてパラメタ推定を行ってきた.しかし,対数双線形言語モデルと勾配に基づくパラメタ推定法の組み合わせは,解の収束性や再現性といった観点で,必ずしも適切な選択とは言えない.本稿では,より信頼性の高い単語分散表現を獲得する枠組みを構築することを目的として,対数双線形言語モデルが持つ性質に対応したパラメタ推定法を提案する.

  141. オラクル要約の列挙

    平尾努, 西野正彬, 鈴木潤, 永田昌明

    言語処理学会年次大会発表論文集(Web)　20th　2014

    ISSN： 2188-4420

  142. 列生成法を用いた高速なアラインメント

    西野正彬, 鈴木潤, 梅谷俊治, 平尾努, 永田昌明

    言語処理学会年次大会発表論文集(Web)　20th　2014

    ISSN： 2188-4420

  143. 英中韓から日本語への特許文向け統計翻訳システム

    須藤克仁, 鈴木潤, 秋葉泰弘, 塚田元, 永田昌明

    言語処理学会年次大会発表論文集(Web)　20th　2014

    ISSN： 2188-4420

  144. 係り受け木に基づく談話構造の提案

    吉田康久, 鈴木潤, 平尾努, 永田昌明

    言語処理学会年次大会発表論文集(Web)　20th　2014

    ISSN： 2188-4420

  145. 大規模素性集合に対する教師あり縮約モデリング

    鈴木潤, 永田昌明

    言語処理学会年次大会発表論文集(Web)　20th　2014

    ISSN： 2188-4420

  146. Robust Wi-Fi Indoor Positioning Method using Boosting

    Daisuke Taniuchi, Takuya Maekawa, Jun Suzuki, Yasue Kishino

    研究報告ユビキタスコンピューティングシステム（UBI）　2013　(5)　1-7　2013/10/29

    Publisher: Information Processing Society of Japan (IPSJ)

    ISSN： 0919-6072

    More details Close

    Recently, many indoor positioning techniques based on Wi-Fi signals have been studied. Wi-Fi fingerprinting technique, which is one of the most popular and practical method, makes use of the Wi-Fi received signal strength (RSS) information collected at several indoor places in advance to construct an indoor positioning model. However, changing environmental dynamics, i.e., layout changes and moving or removal of WiFi access points, may cause the instability of Wi-Fi based positioning methods. In this work, we try to cope with the instability with a boosted positioning estimator consists of several weak estimators. Each weak estimator uses only the signals from some randomly selected APs. Even when signal strength from a specific AP may change, some weak estimators that do not employ the AP are not affected by the change. In our proposed method, we track a user&#039;s coordinates with the particle filter and we evaluate each weak estimator&#039;s prediction by using the particle filter outputs. That is, we find weak estimators that are not affected by the AP by comparing the predictions and coordinates estimated by the particle filter based on the past coordinate history. Our boosted estimator computes final estimation based on the trustworthy weak estimators.

  147. Robust Wi-Fi Indoor Positioning Method using Boosting

    Daisuke Taniuchi, Takuya Maekawa, Jun Suzuki, Yasue Kishino

    IPSJ SIG technical reports　2013　(5)　1-7　2013/10/29

    Publisher: Information Processing Society of Japan (IPSJ)

    ISSN： 0919-6072

    More details Close

    Recently, many indoor positioning techniques based on Wi-Fi signals have been studied. Wi-Fi fingerprinting technique, which is one of the most popular and practical method, makes use of the Wi-Fi received signal strength (RSS) information collected at several indoor places in advance to construct an indoor positioning model. However, changing environmental dynamics, i.e., layout changes and moving or removal of WiFi access points, may cause the instability of Wi-Fi based positioning methods. In this work, we try to cope with the instability with a boosted positioning estimator consists of several weak estimators. Each weak estimator uses only the signals from some randomly selected APs. Even when signal strength from a specific AP may change, some weak estimators that do not employ the AP are not affected by the change. In our proposed method, we track a user&#039;s coordinates with the particle filter and we evaluate each weak estimator&#039;s prediction by using the particle filter outputs. That is, we find weak estimators that are not affected by the AP by comparing the predictions and coordinates estimated by the particle filter based on the past coordinate history. Our boosted estimator computes final estimation based on the trustworthy weak estimators.

  148. 革新的発展期を迎えた統計翻訳 (特集 こころまで伝わるコミュニケーションを支える音声言語と聴覚研究の最前線)

    永田 昌明, 須藤 克仁, 鈴木 潤

    NTT技術ジャーナル　25　(9)　14-17　2013/09

    Publisher: 電気通信協会

    ISSN： 0915-2318

  149. 動的変化する文章を対象とした自然言語解析に適した解析アルゴリズムの考案

    鈴木潤, 永田昌明

    言語処理学会年次大会発表論文集(Web)　19th　2013

    ISSN： 2188-4420

  150. 構造学習を用いたテキストからの地域イベント情報抽出

    数原 良彦, 鈴木 潤, 鷲崎 誠司

    人工知能学会全国大会論文集　2013　(0)　1F31in-1F31in　2013

    Publisher: 一般社団法人 人工知能学会

    ISSN： 1347-9881

    More details Close

    &lt;p&gt;テキストからイベント名称，開催場所，開催日時という項目からなるイベント情報を抽出する際に，固有表現抽出を用いて各項目の候補を取り出すことができる．しかし，一般的に各項目の候補が複数存在するため，全ての組み合わせの中から正しい組み合わせを選択する必要がある．本稿では，正しい組み合わ せを選択するモデルを構造学習の枠組みで生成し，テキストからの地域イベント情報抽出を実現する手法を提案する．&lt;/p&gt;

  151. Local-area Event Extraction from Texts Based on Structured Output Learning

    数原 良彦, 鈴木 潤, 鷲崎 誠司

    人工知能学会全国大会論文集　27　1-4　2013

    Publisher: 人工知能学会

    ISSN： 1347-9881

  152. Statistical Patent Translation Focusing on Word Reordering

    須藤 克仁, 鈴木 潤, 塚田 元

    Japio year book　292-296　2013

    Publisher: 日本特許情報機構

  153. D-007 Spam Detection Using Online Learning Focused on False Positive Rate

    Suhara Yoshihiko, Suzuki Jun, Kataoka Ryoji

    情報科学技術フォーラム講演論文集　11　(2)　91-92　2012/09/04

    Publisher: Forum on Information Technology

  154. ナップサック問題と劣モジュラ関数最大化問題の合意解形成による要約

    安田宜仁, 西野正彬, 平尾努, 鈴木潤

    言語処理学会年次大会発表論文集　18th (CD-ROM)　2012

  155. クエリ中の語を含むことを保証するクエリフォーカス要約

    西野正彬, 安田宜仁, 平尾努, 鈴木潤

    言語処理学会年次大会発表論文集　18th (CD-ROM)　2012

  156. 拡張ラグランジュ緩和を用いた同時自然言語解析法

    鈴木潤, DUH Kevin, 永田昌明

    言語処理学会年次大会発表論文集　18th (CD-ROM)　2012

  157. 条件付確率場とベイズ階層言語モデルの統合による半教師あり形態素解析

    持橋大地, 鈴木潤, 藤野昭典

    言語処理学会年次大会発表論文集　17th (CD-ROM)　2011

  158. 評価指標をマージンに反映したオンラインランキング学習

    数原良彦, 鈴木潤, 安田宜仁, 小池義昌, 片岡良治

    言語処理学会年次大会発表論文集　17th (CD-ROM)　2011

  159. L1正則化特徴選択に基づく大規模データ・特徴集合に適した半教師あり学習

    鈴木潤, 磯崎秀樹, 永田昌明

    言語処理学会年次大会発表論文集　17th (CD-ROM)　2011

  160. 半教師あり系列ラベリングによるアブストラクトのセクション分割

    平尾努, 鈴木潤, 磯崎秀樹, 永田昌明

    言語処理学会年次大会発表論文集　16th　2010

  161. 大規模ラベルなしデータを利用した係り受け解析の性能検証

    鈴木潤, 磯崎秀樹

    言語処理学会年次大会発表論文集　16th　2010

  162. 自然言語処理による医療情報の読解支援

    平尾努, 磯崎秀樹, 須藤克仁, 鈴木潤, 塚田元, 藤野昭典, 永田昌明

    平成21年度情報処理学会関西支部支部大会講演論文集　2009　2009/09/29

    ISSN： 1884-197X

  163. 軽量な文短縮手法

    平尾努, 鈴木潤, 磯崎秀樹

    言語処理学会年次大会発表論文集　14th　2008

  164. 大規模ラベルなしデータを利用した言語解析器の性能検証

    鈴木潤, 磯崎秀樹

    言語処理学会年次大会発表論文集　14th　2008

  165. Semi-supervised Conditional Random Fields for Extremely Large and Sparse Feature Spaces

    SUZUKI Jun, FUJINO Akinori, ISOZAKI Hideki

    IPSJ SIG Notes　2007　(94)　21-28　2007/09/25

    Publisher: Information Processing Society of Japan (IPSJ)

    ISSN： 0919-6072

    More details Close

    This paper proposes a novel semi-supervised conditional random field which provides good characteristics with respect to handling the large and sparse feature spaces. Experiments on two real NLP tasks with extremely large feature spaces, such as named entity recognition and syntactic chunking, show that our proposed method significantly improves the state-of-the-art performance obtained from supervised CRFs [2], and semi-supervised CRFs employing the entropy regularization approach [3]. Moreover, this paper reveals that, theoretically and experimentally, semi-supervised CRFs based on the entropy regularization approach [3] cannot work well for improving the performance of tasks with large and sparse feature spaces.

  166. Multi-labeling of Documents based on F-score Maximization

    FUJINO Akinori, ISOZAKI Hideki, SUZUKI Jun

    IPSJ SIG Notes　2007　(94)　29-34　2007/09/25

    Publisher: Information Processing Society of Japan (IPSJ)

    ISSN： 0919-6072

    More details Close

    Text categorization is generally defined as a multi-labeling problem, where multiple category labels are assigned to each text document. We focus on machine learning approaches to multi-labeling problems and present a classifier design method based on model combination and F_1-score maximization. In our formulation, we first design multiple models for binary classification per category label, which determine whether a category label is assigned or not to each data sample. Then, we combine these models to maximize the F_1-score of a training dataset. Using three real text datasets, we confirmed experimentally that our proposed method was useful especially for the datasets where many category labels were assigned to each data sample and which consisted of many combinations of category labels.

  167. 統計的機械翻訳 (特集 コミュニケーション環境の未来に向けた研究最前線)

    塚田 元, 渡辺 太郎, 鈴木 潤

    NTT技術ジャ-ナル　19　(6)　23-25　2007/06

    Publisher: 電気通信協会

    ISSN： 0915-2318

  168. CRFを用いたブログからの固有表現抽出

    齋藤邦子, 鈴木潤, 今村賢治

    言語処理学会年次大会発表論文集　13th　2007

  169. データの分布特性を利用した半教師あり系列構造学習:言語解析への適用

    鈴木潤, 藤野昭典, 磯崎秀樹

    言語処理学会年次大会発表論文集　13th　2007

  170. ブログ作者の居住域の推定

    安田宜仁, 平尾努, 鈴木潤, 磯崎秀樹

    言語処理学会年次大会発表論文集　12th　2006

  171. 学習誤り最小化に基づく条件付き確率場の学習:言語解析への適用

    鈴木潤, 磯崎秀樹

    言語処理学会年次大会発表論文集　12th　2006

  172. Multiple Document Summarization using Sequential Pattern Mining

    HIRAO Tsutomu, SUZUKI Jun, ISOZAKI Hideki, MAEDA Eisaku

    IPSJ SIG Notes　2003　(108)　31-38　2003/11/06

    Publisher: Information Processing Society of Japan (IPSJ)

    ISSN： 0919-6072

    More details Close

    In this paper, we propose a multiple document summarization method using a sequential pattern mining algorithm. We extract important sentences in the following way ; First, extracting term patterns from target docment set by using PrefixSpan. Second, identifying significant patterns based on X^2 statistics, Third, determining a sentence score using the patterns weighting based on TF・IDF. Moreover, we propose a kernel-based MMR (Maximal Marginal Relavance) for minimizing reduandant sentences. This method employs a similarity measure based on Extended String Subsequence kernel instead of cosine similarity. In addition, we define an evaluation measure for deta set includes redundant sentences, i. e., there are many sentences whose meaning are the same. The evaluation results show that our extraction method is better than conventional methods and the lernel-based MMR out performs conventional MMR.

  173. On Selection Criteria of Combinatorial Features for Machine Learning

    ISOZAKI Hideki, HIRAO Tsutomu, SUZUKI Jun

    IPSJ SIG Notes　2003　(108)　63-68　2003/11/06

    Publisher: Information Processing Society of Japan (IPSJ)

    ISSN： 0919-6072

    More details Close

    Machine Learning is used for various tasks of Natural Language processing such as Named Entity Rcognition, Important Sentence Extraction, and Dependency Analysis. Features for Machine Learning are found by trial and error. However, it is possible to find useful features by using statistical measures. For example, PrefixSpan finds frequent word patterns and TidalSMP finds useful feature combinations. Such combinatiorial features are often redundant and are not optimized for Machine Learning. Here, we show that a simple reranking method improves the performance of Machine Learning in two tasks : Important Sentence Extraction and English Dependency Analysis.

  174. Multiple Document Summarization using Sequential Pattern Mining(Natural Language Understanding and Models of Communication)

    Hirao Tsutomu, Suzuki Jun, Isozaki Hideki, Maeda Eisaku

    IEICE technical report. Natural language understanding and models of communication　103　(407)　31-38　2003/11/06

    Publisher: The Institute of Electronics, Information and Communication Engineers

    ISSN： 0913-5685

    More details Close

    In this paper, we propose a multiple document summarization method using a sequential pattern mining algorithm. We extract important sentences in the following way; First, extracting term patterns from target docment set by using PrefixS- pan. Second, identifying significant patterns based on x^2 statistics, Third, determining a sentence score using the patterns weighting based on TF・IDF. Moreover, we propose a kernel-based MMR (Maximal Marginal Relevance) for minimizing reduandant sentences. This method employs a similarity measure based on Extended String Subsequence kernel instead of cosine similarity. In addition, we define an evaluation measure for deta set includes redundant sentences, i.e., there are many sentences whose meaning are the same. The evaluation results show that our extraction method is better than conventional methods and the kernel-based MMR outperforms conventional MMR.

  175. On Selection Criteria of Combinatorial Features for Machine Learning(Natural Language Understanding and Models of Communication)

    ISOZAKI Hideki, HIRAO Tsutomu, SUZUKI Jun

    IEICE technical report. Natural language understanding and models of communication　103　(407)　63-68　2003/11/06

    Publisher: The Institute of Electronics, Information and Communication Engineers

    ISSN： 0913-5685

    More details Close

    Machine Learning is used for various tasks of Natural Language Processing such as Named Entity Recognition, Important Sentence Extraction, and Dependency Analysis. Features for Machine Learning are found by trial and error. However, it is possible to find useful features by using statistical measures. For example, PrefixSpan finds frequent word patterns and TidalSMP finds useful feature combinations. Such combinatiorial features are often redundant and are not optimized for Machine Learning. Here, we show that a simple reranking method improves the performance of Machine Learning in two tasks: Important Sentence Extraction and English Dependency Analysis.

  176. Multiple Document Summarization using Sequential Pattern Mining

    HIRAO Tsutomu, SUZUKI Jun, ISOZAKI Hideki, MAEDA Eisaku

    電子情報通信学会技術研究報告. NLC, 言語理解とコミュニケーション　103　(407)　31-38　2003/10/30

  177. On Selection Criteria of Combinatorial Features for Machine Learning

    ISOZAKI Hideki, HIRAO Tsutomu, SUZUKI Jun

    電子情報通信学会技術研究報告. NLC, 言語理解とコミュニケーション　103　(407)　63-68　2003/10/30

  178. String Kernel with Feature Selection Function

    鈴木 潤, 平尾 努, 磯崎 秀樹, 前田 英作

    情報処理学会研究報告自然言語処理（NL）　2003　(98)　41-48　2003/09/29

    ISSN： 0919-6072

    More details Close

    本稿では，String Kernelに対する素性選択手法について議論する．StringKernelを含むConvolution Kernelsの枠組では，全ての部分構造間の部分カーネルの総和を全体のカーネルの値と定義している．しかし，可能な全ての部分構造を素性として使用すると，素性空間の次元数が高くなり，データスパースネスの問題が起こることが実験的に示されている．このためString Kernelでは，カーネル計算に使用する部分記号列を部分記号列のサイズに応じて選択する枠組が導入されている．本稿では，この従来手法による素性選択基準の問題点について議論し，その問題点を改善した素性選択手法を提案する．さらに，質問分類と文のモダリティ判定の二つのタスクを用いて，本稿で取り上げた従来手法の問題点の正当性を検証するとともに，従来手法との比較実験によって提案手法の有効性を示す．This paper discusses feature selection methods about String Kernel. The kernel value of Convolution Kernels is defined as a sum of all sub-kernels between all sub-structures of input objects. However, it is known experimentally that the data-sparseness problem arises if treating all sub-structures to calculate the kernel value. For this reason, the framework of a feature selection method based on the size of sub-sequences is introduced in String Kernel. In this paper, first, we discuss an issue about this kind of conventional feature selection method, and then we propose a new feature selection method. After that, we confirm the issue of conventional method experimentally and compare the performance between a conventional method and the proposed method by using using question classification task and sentence modality identification task.

  179. String Kernel with Feature Selection Function

    SUZUKI Jun, HIRAO Tsutomu, ISOZAKI Hideki, MAEDA Eisaku

    IPSJ SIG Notes　2003　(98)　41-48　2003/09/29

    Publisher: Information Processing Society of Japan (IPSJ)

    ISSN： 0919-6072

    More details Close

    This paper discusses feature selection methods about String Kernel. The kernel value of Convolution Kernels is defined as a sum of all sub-kernels between all sub-structures of input objects. However, it is known experimentally that the datasparseness problem arises if treating all sub-structures to calculate the kernel value. For this reason, the framework of a feature selection method based on the size of sub-sequences is introduced in String Kernel. In this paper, first, we discuss an issue about this kind of conventional feature selection method, and then we propose a new feature selection method. After that, we confirm the issue of conventional method experimentally and compare the performance between a conventional method and the proposed method by using using question classification task and sentence modality identification task

  180. Efficient Calculation for Similarity between Texts using Hierarchical Structure

    SUZUKI Jun, HIRAO Tsutomu, SASAKI Yutaka, MAEDA Eisaku

    IPSJ SIG Notes　2003　(23)　101-108　2003/03/06

    Publisher: Information Processing Society of Japan (IPSJ)

    ISSN： 0919-6072

    More details Close

    This paper proposes an efficient calculation method of the similarity between two texts which reflects various structures inside the texts. We realize this similarity by expressing the structures inside a text by hierarchical directed acyclic graph (HDAG). Specifically, we define the similarity between HDAGs as the matching of all alignment sequences extracted from all sub-pathes inside HDAGs. In this care, the number of the sub-pathes inside graphs usually yeald extremely large numbers. However, we can calculate the similarity efficiently without matching of all alignment sequences.

  181. テキストの構造を考慮したテキスト間類似度の効率的計算法

    鈴木潤, 平尾努, 佐々木裕, 前田英作

    言語処理学会年次大会発表論文集　9th　2003

  182. LE-10 Question Type Classification using Statistical Machine Learning

    Suzuki Jun, Sasaki Yutaka, Maeda Eisaku

    情報技術レターズ　1　89-90　2002/09/13

    Publisher: Forum on Information Technology

  183. SAIQA : A Japanese QA System Based on a Large - Scale Corpus

    SASAKI Y, ISOZAKI H, TAIRA H, HIRAO T, KAZAWA H, SUZUKI J, KOKURYO K, MAEDA E

    IPSJ SIG Notes　2001　(86)　77-82　2001/09/10

    Publisher: Information Processing Society of Japan (IPSJ)

    ISSN： 0919-6072

    More details Close

    Conventional TREC-style Question-Answering (QA) involves extracting passages (250 bytes or 50 bytes) that contain answers to a question. The most attractive feature of QA is that it can provide exact answers to the question, rather than a list of ranked passages, paragraphs, or documants. This paper describes a Japanese QA system SAIQA which finds exact answers to a question from a large-scale Japanese text corpus and an experimental evaluation on exact answer extraction. The experiments include evaluations of answers to 2000 questions with justification by article summarization. The results show that around 50% of correct answers can be found from the top of five answers and that over 87% of short summaries can be used for answer justification (or evidence) instead of full texts.

  184. SAIQA : A Japanese QA System Based on a Large - Scale Corpus

    Sasaki Y, Isozaki H, Taira H, Hirao T, Kazawa H, Suzuki J, Kokuryo K, Maeda E

    IPSJ SIG Notes　2001　(86)　77-82　2001/09/10

    Publisher: Information Processing Society of Japan (IPSJ)

    ISSN： 0919-6072

    More details Close

    Conventional TREC-style Question-Answering (QA) involves extracting passages (250 bytes or 50 bytes) that contain answers to a question. The most attractive feature of QA is that it can provide exact answers to the question, rather than a list of ranked passages, paragraphs, or documants. This paper describes a Japanese QA system SAIQA which finds exact answers to a question from a large-scale Japanese text corpus and an experimental evaluation on exact answer extraction. The experiments include evaluations of answers to 2000 questions with justification by article summarization. The results show that around 50% of correct answers can be found from the top of five answers and that over 87% of short summaries can be used for answer justification (or evidence) instead of full texts.

  185. GDAタグを利用した回答抽出システムの提案

    鈴木潤, 橋田浩一

    言語処理学会年次大会発表論文集　7th　2001

Show all ︎Show first 5

Books and Other Publications 3

  1. IT Text 自然言語処理の基礎

    岡﨑 直観, 荒瀬 由紀, 鈴木 潤, 鶴岡 慶雅, 宮尾 祐介

    オーム社　2022/08/24

    ISBN: 4274229009

  2. 自然言語処理技術 ～目的に応じた手法選択／精度向上手法／業務活用への提言

    鈴木 潤, 土屋 誠司, 本橋 和貴, 高橋 寛治, 田村 晃裕, 山田 育矢, 荒木 健治, 森 信介, 渡邉 信一, 原 紳, 水本 智也, 清水 武

    2020/03

  3. Natural language processing by deep learning

    2017/05

    ISBN: 9784061529243

Presentations 16

  1. WMT-2020ニュース翻訳タスクに参加して：Team Tohoku-AIP-NTT at WMT-2020 Invited

    鈴木潤

    AAMT 2021, Online ～機械翻訳最前線～ 2021年AAMT長尾賞受賞者講演　2021/12/08

  2. AI系トップカンファレンスへの論文採択に向けた試験対策 Invited

    鈴木 潤

    2020年度 人工知能学会全国大会（第34回）チュートリアル講演　2020/06/10

  3. トップカンファレンスへの論文採択に向けて（AI研究分野版） Invited

    鈴木 潤

    人工知能学会合同研究会2019 IJCAI2020に向けた論文書き方セミナー　2019/11/23

  4. 国際会議への論文の通し方（一般論） Invited

    鈴木 潤

    第4回 統計・機械学習若手シンポジウム　2019/11/15

  5. 単語埋め込みベクトルの利便性向上 Invited

    鈴木 潤

    東北大学 情報科学研究科 第76回 情報科学談話　2019/09/19

  6. トップカンファレンスへの論文採択に向けて（NLP研究分野版） Invited

    鈴木 潤

    NLP若手の会 (YANS) 第14回シンポジウム（YANS-2019）　2019/08/26

  7. Usability Enhancements to Neural Word Embeddings Invited

    Jun Suzuki

    Third International Workshop on Symbolic-Neural Learning (SNL-2019)　2019/07/11

  8. 深層学習による自然言語処理の解釈性／説明性 Invited

    鈴木 潤

    情報処理学会 連続セミナーAIと歩む未来(1)：自然言語処理の最新動向　2019/06/26

  9. NAACL 2016 ～会議概要と研究傾向分析～ Invited

    鈴木 潤

    第9回テキストマイニング・シンポジウム 国際会議報告　2016/09/09

  10. "Learning Compact Neural Word Embeddings by Parameter Space Sharing＋IJCAI-2016参加レポート" Invited

    鈴木 潤

    河原林ERATO グラフマイニング&WEB&AIセミナー　2016/07/25

  11. 単語埋め込みベクトル獲得法の現状といくつかの考察 Invited

    鈴木 潤

    産総研人工知能セミナー　2016/06/21

  12. Supervised Model Learning with Feature Grouping based on a Discrete Constraint

    鈴木 潤

    NII Shonan Meeting, Discrete Algorithms Meet Machine Learning　2013/08/13

  13. 大規模教師なしデータからの縮約素性表現学習 Invited

    鈴木 潤

    名古屋地区NLPセミナー 第９回　2011/08/03

  14. Semi-supervised Structured Output Learning using Giga-scale Unlabeled Data Invited

    鈴木 潤

    生命情報工学研究センター セミナー講演　2008/06/27

  15. 半教師あり学習による分類法: -現状と自然言語処理への適用- Invited

    鈴木 潤

    言語処理学会第14回年次大会(NLP2008) チュートリアル　2008/03/17

  16. Semi-supervised Structured Output Learning Invited

    Jun Suzuki

    Tokyo Forum on Advanced NLP and TM　2008/02/13

Show all Show first 5

Industrial Property Rights 59

  1. 自然言語処理のための装置、方法及びプログラム

    鈴木 潤, 高瀬 翔, 乾 健太郎, 岡▲崎▼ 直観, 清野 舜

    特許第7072178号

    Property Type: Patent

  2. 情報処理装置、情報学習装置、情報処理方法、情報学習方法及びプログラム

    森下 睦, 鈴木 潤, 高瀬 翔, 上垣外 英剛, 永田 昌明

    特許第6772393号

    Property Type: Patent

  3. 情報学習装置、情報処理装置、情報学習方法、情報処理方法及びプログラム

    森下 睦, 鈴木 潤, 高瀬 翔, 上垣外 英剛, 永田 昌明

    特許第6772394号

    Property Type: Patent

  4. 集合分割問題求解装置、方法、及びプログラム

    西野 正彬, 鈴木 潤, 梅谷 俊治

    特許第6725920号

    Property Type: Patent

  5. 文生成装置、文生成学習装置、文生成方法、及びプログラム

    高瀬 翔, 鈴木 潤, 永田 昌明

    特許第6712973号

    Property Type: Patent

  6. 分類器学習装置、クラス決定装置、方法、及びプログラム

    鈴木 潤

    特許第6706214号

    Property Type: Patent

  7. 文字列変換装置、モデル学習装置、方法、及びプログラム

    斉藤 いつみ, 鈴木 潤, 浅野 久子, 齋藤 邦子, 松尾 義博

    特許第6684693号

    Property Type: Patent

  8. 要約生成装置、テキスト変換装置、方法、及びプログラム

    鈴木 潤, 平尾 努, 岡崎 直観, 高瀬 翔

    特許第6635307号

    Property Type: Patent

  9. 符号器学習装置、変換装置、方法、及びプログラム

    鈴木 潤

    特許第6633999号

    Property Type: Patent

  10. 最適化問題解決装置、方法、及びプログラム

    鈴木 潤, 西野 正彬, 梅谷 俊治

    特許第6628041号

    Property Type: Patent

  11. 単語ベクトル学習装置、自然言語処理装置、方法、及びプログラム

    鈴木 潤

    特許第6586026号

    Property Type: Patent

  12. 行動決定装置、未来予測モデル学習装置、ネットワーク学習装置、方法、及びプログラム

    鈴木 潤, 鶴岡 慶雅

    特許第6550678号

    Property Type: Patent

  13. 符号化装置、復号化装置、離散系列変換装置、方法、及びプログラム

    鈴木 潤, 岡崎 直観, 高瀬 翔

    特許第6550677号

    Property Type: Patent

  14. 秘匿計算システム、第一秘匿計算装置、第二秘匿計算装置、秘匿回路生成方法、秘匿回路評価方法、プログラム

    鈴木 幸太郎, 富田 潤一

    特許第6532843号

    Property Type: Patent

  15. 単語ベクトル学習装置、自然言語処理装置、方法、及びプログラム

    鈴木 潤

    特許第6517537号

    Property Type: Patent

  16. 情報処理方法、装置、及びプログラム

    西野 正彬, 鈴木 潤, 永田 昌明

    特許第6498135号

    Property Type: Patent

  17. 用語意味コード判定装置、用語意味コード判定モデル学習装置、方法、及びプログラム

    藤野 昭典, 鈴木 潤, 平尾 努

    特許第6495124号

    Property Type: Patent

  18. 情報処理方法、装置、及びプログラム

    西野 正彬, 鈴木 潤, 平尾 努, 梅谷 俊治

    特許第6482073号

    Property Type: Patent

  19. パラメータ学習方法、装置、及びプログラム

    吉田 康久, 鈴木 潤, 平尾 努, 林 克彦, 永田 昌明

    特許第6291440号

    Property Type: Patent

  20. 位置推定装置、方法、及びプログラム

    鈴木 潤, 岸野 泰恵, 前川 卓也

    特許第6284151号

    Property Type: Patent

  21. 縮約素性生成装置、情報処理装置、方法、及びプログラム

    鈴木 潤

    特許第6230501号

    Property Type: Patent

  22. 用語抽出装置、方法、及びプログラム

    鈴木 潤, 藤野 昭典, 平尾 努

    特許第6220767号

    Property Type: Patent

  23. システムパラメタ学習装置、情報処理装置、方法、及びプログラム

    鈴木 潤

    特許第6101650号

    Property Type: Patent

  24. 係り受け関係解析パラメータ学習装置、係り受け関係解析装置、方法、及びプログラム

    吉田 康久, 平尾 努, 鈴木 潤, 永田 昌明

    特許第6062829号

    Property Type: Patent

  25. 語順並び替え装置、翻訳装置、方法、及びプログラム

    林 克彦, 須藤 克仁, 塚田 元, 鈴木 潤, 永田 昌明

    特許第6058513号

    Property Type: Patent

  26. 文対応付け決定装置、方法、及びプログラム

    西野 正彬, 鈴木 潤, 梅谷 俊治

    特許第6019538号

    Property Type: Patent

  27. システムパラメータ最適化装置、方法、及びプログラム

    平尾 努, 鈴木 潤, 永田 昌明

    特許第5985344号

    Property Type: Patent

  28. 翻訳システム、方法、及びプログラム

    吉田 仙, 引地 孝文, 片山 太一, 鈴木 潤, 須藤 克仁

    特許第5973986号

    Property Type: Patent

  29. オラクル要約探索装置、方法、及びプログラム

    平尾 努, 鈴木 潤, 永田 昌明

    特許第5964791号

    Property Type: Patent

  30. 近似オラクル文選択装置、方法、及びプログラム

    安田 宜仁, 平尾 努, 鈴木 潤, 永田 昌明

    特許第5889225号

    Property Type: Patent

  31. 自然言語解析処理装置、方法、及びプログラム

    鈴木 潤

    特許第5886220号

    Property Type: Patent

  32. 情報抽出装置、情報抽出方法及び情報抽出プログラム

    数原 良彦, 鈴木 潤, 鷲崎 誠司

    特許第5863193号

    Property Type: Patent

  33. スパム分類モデル生成装置及び方法及びプログラム

    数原 良彦, 鈴木 潤, 片岡 良治

    特許第5815468号

    Property Type: Patent

  34. システムパラメータ学習装置、情報処理装置、方法、及びプログラム

    鈴木 潤

    特許第5766753号

    Property Type: Patent

  35. アイテムパターン抽出装置、方法、及びプログラム

    進藤 裕之, 平尾 努, 鈴木 潤, 永田 昌明

    特許第5734932号

    Property Type: Patent

  36. ランキング関数学習装置、方法、及びプログラム

    ケヴィン ドゥ, 鈴木 潤

    特許第5734820号

    Property Type: Patent

  37. システムパラメータ最適化装置、方法、及びプログラム

    鈴木 潤

    特許第5728357号

    Property Type: Patent

  38. 文書要約装置及び方法及びプログラム

    安田 宜仁, 西野 正彬, 平尾 努, 鈴木 潤, 片岡 良治

    特許第5702744号

    Property Type: Patent

  39. サポートベクタ選択装置、方法、及びプログラム

    数原 良彦, 鈴木 潤, 安田 宜仁, 小池 義昌, 片岡 良治

    特許第5684077号

    Property Type: Patent

  40. 文書要約装置及び方法及びプログラム

    西野 正彬, 安田 宜仁, 平尾 努, 鈴木 潤, 片岡 良治

    特許第5670939号

    Property Type: Patent

  41. 縮約素性生成装置、方法、プログラム、モデル構築装置及び方法

    鈴木 潤

    特許第5623344号

    Property Type: Patent

  42. 自然言語解析処理装置、方法、及びプログラム

    鈴木 潤

    特許第5530469号

    Property Type: Patent

  43. ID付与装置、方法、及びプログラム

    鈴木 潤

    特許第5521064号

    Property Type: Patent

  44. 準頻出構造パターンマイニング装置と頻出構造パターンマイニング装置とそれらの方法、及びプログラム

    鈴木 潤

    特許第5506629号

    Property Type: Patent

  45. 文書検索装置、文書検索方法および文書検索プログラム

    数原 良彦, 鈴木 潤, 安田 宜仁, 小池 義昌, 片岡 良治

    特許第5475704号

    Property Type: Patent

  46. 言語モデル学習装置、言語モデル学習方法、言語解析装置、及びプログラム

    持橋 大地, 鈴木 潤, 藤野 昭典

    特許第5441937号

    Property Type: Patent

  47. ランキングモデル選択機能を有する文書検索装置、ランキングモデル選択機能を有する文書検索方法およびランキングモデル選択機能を有する文書検索プログラム

    数原 良彦, 鈴木 潤, 安田 宜仁, 小池 義昌, 片岡 良治

    特許第5432936号

    Property Type: Patent

  48. 予測器選択装置、予測器選択方法、予測器選択プログラム

    数原 良彦, 鈴木 潤, 安田 宜仁, 小池 義昌, 片岡 良治

    特許第5432935号

    Property Type: Patent

  49. パターン抽出装置、パターン抽出方法及びプログラム

    平尾 努, 鈴木 潤, 磯崎 秀樹, 永田 昌明

    特許第5325131号

    Property Type: Patent

  50. 構造予測モデル学習装置、方法、プログラム、及び記録媒体

    鈴木 潤, マイケル ジェイ. コリンズ

    特許第5250076号

    Property Type: Patent

  51. 機械翻訳装置、機械翻訳方法、およびそのプログラムならびに記録媒体

    塚田 元, 渡辺 太郎, 鈴木 潤, 磯崎 秀樹

    特許第5180522号

    Property Type: Patent

  52. 言語解析モデル学習装置、言語解析モデル学習方法、言語解析モデル学習プログラムならびにその記録媒体

    鈴木 潤

    特許第5139701号

    Property Type: Patent

  53. 文抽出および文短縮を組合せた文書要約方法、文書要約装置、文書要約プログラムおよびそのプログラムを記録した記録媒体

    平尾 努, 鈴木 潤, 磯崎 秀樹

    特許第5111300号

    Property Type: Patent

  54. 言語解析モデル学習装置、言語解析モデル学習方法、言語解析モデル学習プログラム及びその記録媒体

    鈴木 潤

    特許第4328362号

    Property Type: Patent

  55. 素性選択機能付き離散カーネル関数の計算装置、プログラムならびに該プログラムを格納した記録媒体

    鈴木 潤

    特許第4250098号

    Property Type: Patent

  56. 類似度計算方法、装置、プログラムおよび該プログラムを格納した記録媒体

    鈴木 潤, 前田 英作

    特許第4073015号

    Property Type: Patent

  57. 質問タイプ学習装置、質問タイプ学習プログラム、同プログラムを記録した記録媒体、学習サンプルが記録されている記録媒体、質問タイプ同定装置、質問タイプ同定プログラム、同プログラムを記録した記録媒体

    鈴木 潤, 佐々木 裕, 前田 英作

    特許第4008313号

    Property Type: Patent

  58. 応答対話文生成方法、応答対話文作成装置、応答対話文作成プログラム、このプログラムを記録した記録媒体

    堀 智織, 鈴木 潤, 堀 貴明, 磯崎 秀樹

    特許第3946115号

    Property Type: Patent

  59. 文変換装置、文変換方法及びプログラム

    永田 昌明, 森下 睦, 鈴木 潤

    Property Type: Patent

Show all Show first 5

Research Projects 8

  1. Evolution of Large Language Models into Large Knowledge Models through Integration with Human Knowledge

    Offer Organization: Japan Society for the Promotion of Science

    System: Grants-in-Aid for Scientific Research

    Category: Grant-in-Aid for Scientific Research (A)

    Institution: Waseda University

    2024/04/01 - 2028/03/31

  2. コードレビューAIによるプログラミング学習支援を目指すデータエコシステム基盤

    渡部 有隆, 森下 睦, 小田 悠介, 鈴木 潤

    Offer Organization: 日本学術振興会

    System: 科学研究費助成事業

    Category: 基盤研究(B)

    Institution: 会津大学

    2023/04/01 - 2027/03/31

    More details Close

    オンラインジャッジシステムにレビュー機能を実装し，研究基盤を構築した．まず，学習者が提出したソースコードに対してレビューをリクエストし，それを他者や生成AIがレビューし，そのフィードバッグに対して学習者が評価する仕組みを確立した．本基盤は，研究代表者が開発・運用するAizu Online Judgeを拡張することで実現した．具体的には，レビュー活動を行うユーザインタフェース，レビュー管理システム，データベース，それらを連携させるAPI を実装した．本基盤によって，学習者，レビューア，生成AIの活動から有益なデータを取得できるようになった．また，生成AIの性能を調査するために，様々なプロンプト手法・形態に対する言語モデルの堅牢性を調査した．具体的には，既存の代表的な大規模言語モデルを対象とし，問題文の様々な変更パタンに対するコード生成の精度を検証した．さらに，コードレビュー活動において重要な活動であるリファクタリングを生成AIで実施するためのプロンプト手法を開発した．具体的には，複雑さを軽減するリファクタリングにおいて，大規模言語モデルのIn-Context Learningに使われる例題の生成法の有効性を検証した．

  3. 自然言語処理の研究

    Offer Organization: JST

    System: ムーンショット型研究開発制度

    2020/12 - 2025/11

  4. 深奥質感のマルチモーダル深層モデルの確立

    岡谷 貴之, 菅沼 雅徳, 劉 星, 鈴木 潤

    Offer Organization: 日本学術振興会

    System: 科学研究費助成事業 学術変革領域研究(A)

    Category: 学術変革領域研究(A)

    Institution: 東北大学

    2020/11/19 - 2025/03/31

    More details Close

    まず，DNNによる画像理解の研究を行い，複数の成果を得た．第一に，自然言語の指示によって作業を行うAIエージェントを開発した．エージェントが空間を見る視野をなるべく広く取るとともに，作業の指示を与える言語情報を2回解釈することに特徴があり，これによって高い作業成功率の達成が可能となった．手法は，国際的なベンチマークテストAlfredにおいて論文発表時点で全世界トップの性能を達成するとともに，成果の一端を国際会議IJACI2021にて発表した． 第二に，画像理解タスクの性能向上を目指して，画像記述の研究を行った．まず，これまでの手法が「比喩的表現」を基本的に行えないことに着目し，データセットとベンチマークテストを設計するとともに，評価方法を検討した．成果の一部は国際会議に投稿中である．第三に，画像記述のためのより高性能かつ高効率なDNNの設計を行った．画像特徴を取り出すのに，従来手法のように物体検出器に由来するCNNを使うのではなく，トランスフォーマーを用いることで高速化を達成した．同時に，物体領域から抽出した特徴と，画像を格子状に分割した領域から取り出した特徴の双方を効果的に統合することで，記述の精度を大幅に向上させている．国際的なベンチマークテストCOCO captionsにおいて，世界トップクラスの記述精度を従来手法の少なくとも数分の1の計算量で達成可能なDNNとなっている．本成果は国際会議に投稿中である．さらに，主に自然言語で表現された外部知識を，画像理解に導入する手法の検討を行っており，継続中である． また，以上とは独立に，自己教師学習の方法について研究を行った．特に近年活発に研究されている負例を用いない画像特徴の自己教師学習手法について，その有効性がどこから由来するものかを理論的に検討した．成果は国際会議に投稿中である．

  5. Building General Language Understanding Infrastructure by Fusing Computational and Human Intelligence

    Offer Organization: Japan Society for the Promotion of Science

    System: Grants-in-Aid for Scientific Research Grant-in-Aid for Scientific Research (A)

    Category: Grant-in-Aid for Scientific Research (A)

    Institution: Waseda University

    2021/04 - 2025/03

  6. Sentence translation mechanism equipped with an explainable process based on real-world and linguistic knowledge

    Suzuki Jun

    Offer Organization: Japan Society for the Promotion of Science

    System: Grants-in-Aid for Scientific Research Grant-in-Aid for Scientific Research (B)

    Category: Grant-in-Aid for Scientific Research (B)

    Institution: Tohoku University

    2019/04/01 - 2022/03/31

    More details Close

    Realizing a text generation mechanism at the human level is one of the most critical and challenging unsolved problems in artificial intelligence and natural language processing research. While current methods based on deep neural networks can generate fluent sentences, a new problem, the erroneous generation problem, has been pointed out. In this study, to solve the erroneous generation problem and reveal the mechanism of its occurrence, we have developed several methods, such as an example-based method to detect reasons for erroneous generation, and an improved natural language analysis system for analyzing erroneous generations.

  7. The computational modeling of deep, robust discourse analysis by integrating abductive reasoning, machine learning, and physical computing

    Inui Kentaro

    Offer Organization: Japan Society for the Promotion of Science

    System: Grants-in-Aid for Scientific Research Grant-in-Aid for Scientific Research (A)

    Category: Grant-in-Aid for Scientific Research (A)

    Institution: Tohoku University

    2015/04/01 - 2019/03/31

    More details Close

    This project aimed at the computational modeling of deep, robust discourse analysis that was able to "read between the lines" by integrating abductive reasoning, machine learning, and physical computing. The main achievements are as follows. First, we significantly enhanced the computational capacity of abductive reasoning by formalizing the problem as weighted maximum satisfiability and devising efficient pruning methods. Second, we developed novel methods for large-scale knowledge acquisition from both Web and Wikipedia documents and demonstrated the impact of leveraging acquired knowledge on semantic and discourse analysis. Third, we built and empirically evaluated a computational model that innovatively integrated abductive reasoning and physical simulation to predict risks involved in given traffic scenes. The resources and tools developed in this project are made publicly available on our website.

  8. 大規模分散並列処理に基づく統計的機械翻訳 Competitive

    塚田 元, 磯崎 秀樹, 渡辺 太郎, 藤野 昭典, 鈴木 潤, 須藤 克仁

    Offer Organization: 日本学術振興会

    System: 科学研究費助成事業 特定領域研究

    Category: 特定領域研究

    Institution: 日本電信電話株式会社NTTコミュニケーション科学基礎研究所

    2007 - 2008

    More details Close

    本年度は、(1)階層的な句に基づく翻訳手法の高度化(須藤,渡辺,塚田)、および(2)半教師あり学習に基づく言語情報解析法の研究(鈴木,藤野,磯崎)の二つの課題に取り組んだ。 研究課題(1)として,本年度はNTTの研究費によって開発した高次元素性を活用した翻訳手法を,評価型ワークショップIWSLT2008のタスクで評価した.昨年度も高次元素性を活用するアプローチでIWSLT2007に参加したが,今年は文脈情報を含むより高度な素性の活用を検討した.昨年度は翻訳デコーダと密に結合した学習器によって,高次元素性を扱ったが今年より柔軟な素性の扱いを優先し,翻訳結果を再順序付けする学習器で高次元素性を扱うアプローチをとった.最終的に、中英チャレンジタスクにおいて,我々は他チームの1/10〜1/100の量の学習データ(対訳コーパス)で、11チーム中,3位相当の成績を達成することができた. 研究課題(2)としては,昨年度考案した「半教師あり学習法」による固有表現抽出器などのスケーラビリティの確認をさらなる大規模学習データで確認した.昨年度は10億語を超える学習データを利用した実験を行い,その成果は今年度ACL-2008に採録されたが,今年度はデータをさらに増やして40億語近いデータでも実験を行った.その結果,性能がさらに向上することを確認した.この他,NTTの研究費で考案した半教師あり学習に基づく文書分類法を、公開データ(報道記事,NTCIRタスク)で評価し,有効性を確認した.

Show all Show first 5

Teaching Experience 6

  1. Natural Language Processing　Tohoku University

  2. Artificial Intelligence　Tohoku University

  3. Programming Exercise A　Tohoku University

  4. 情報伝達学　東北大学

  5. 創造工学研修　東北大学

  6. 情報システム第二　慶應義塾大学

Show all Show first 5