Language pre-training

Author: Yu Shiwen / Huang Juren, Editors
Publisher:
Publish Date: 2005-09-01
Features: This book consists of six chapters written by internationally renowned scholars in the field of computational linguistics. These chapters are extended talks or comprehensive reviews presented at the 19th International Conference on Computational Linguistics (COLING 2002). They evaluate the current state of research in the field, introduce cutting-edge achievements, and outline the prospects for computational linguistics in the 21st century.
[Preface] Preface: Natural language processing techniques and computational linguistics, closely related to the process of social informatization, have made significant progress in China in recent years. The National High-Tech Research and Development Program (863), the National Key Basic Research Development Program (973), the National Natural Science Foundation, and the National Social Science Foundation have all supported relevant research topics. The biennial National Conference on Computational Linguistics was held at Harbin Institute of Technology in 2003, and the 2nd Student Conference on Computational Linguistics was held in late August 2004 at Beijing Language and Culture University. Many other academic conferences have also been held. Research, teaching, and academic exchanges in this field are highly active. Scholars in this field are well aware of the saying, "A distant land's stone can be used to polish one's own jade," and have consistently strived to learn from and introduce advanced theories and experiences from abroad, strengthening exchanges with international colleagues.
The book Computational Linguistics and Beyond (co-edited by Huang Juren and Winfried Lenders) was published in July 2004 by the Institute of Linguistics, Academia Sinica, Taiwan. This is likely the latest published work on computational linguistics. The book consists of six chapters written by internationally renowned scholars in the field. These chapters were extended talks or comprehensive reviews presented at the 19th International Conference on Computational Linguistics (COLING 2002) held in Taipei in 2002, and were compiled into written form after the conference. The 19th International Conference on Computational Linguistics, a biennial event, was successfully held in Taipei in August 2002. This conference holds two special significances:
— It is the first COLING conference of the 21st century, providing an excellent opportunity for the computational linguistics community to gather insights and set the tone for the new century.
— It is the first COLING conference to be held outside advanced countries such as Europe, America, and Japan. It not only represents the entry of the Asia-Pacific region into the ranks of technological development countries but also highlights the pivotal role of the region's languages, especially Chinese, in the international stage of information processing.
Among the nearly three hundred papers presented at the conference, each made contributions to its field. Six papers stood out as particularly noteworthy. Three of these were written by keynote speakers at the conference, whose international academic prestige needs no elaboration. The foresight of these three papers is particularly remarkable, making them not only essential reading for the current academic community but also classics for future generations. The other three papers were written by influential scholars from three different fields, providing comprehensive assessments of the current state of the field and outlining blueprints for future development. These six papers are not only classics of computational linguistics but also hold significant inspiration for language technology in the 21st century. Especially in the age of the internet, how to use language technology to solve knowledge economy problems (such as the multilingual information challenges of the 2008 Beijing Olympics) will be a major contribution of computational linguistics to society. Academically, the greatest challenge lies in addressing fundamental questions about human nature—such as "Why is intelligence?" and "Where does language come from?"—through interdisciplinary research at the intersection of neuroscience, psychology, and computer science.
This book draws on the essence of both academia and technology. If published in Chinese, it will not only benefit students but also elevate domestic academic standards and align with international high-level research. Huang Juren is also a part-time foreign professor at the Institute of Computational Linguistics, Peking University. At Huang Juren's suggestion, Yu Shiwen organized a group of doctoral students from the Institute of Computational Linguistics, Peking University, to translate these six chapters. The translation was authorized by the original authors. The Commercial Press's publication of this translation is a support for the emerging discipline of computational linguistics and an encouragement to the translators.
Everyone knows that translating high-level, forward-looking academic works is not an easy task. To ensure translation quality, the Institute of Computational Linguistics, Peking University, took the following measures:
(1) Both translators and proofreaders were listed by name to demonstrate responsibility.
(2) Translators reported on the content and insights of each original chapter in discussion sessions, with the reporting process and audience questions assessing their understanding.
(3) A editorial committee was established, including members such as Yu Shiwen, Huang Juren, Liu Ling, Liu Yang, and Wu Yunfang. The committee oversaw the overall quality and style of the book and formulated the style guide for the translation of "Computational Linguistics and Beyond." In practice, each chapter was finalized through multiple rounds of interaction between translators, proofreaders, and the editorial committee.
(4) When the original authors were fluent in Chinese, the translators submitted their preliminary drafts for confirmation.
We sincerely thank the original authors for their cooperation and guidance. Their dedication and professionalism have been a great encouragement and motivation for the translators. Special thanks are also due to Professor Wang Shiwen, who provided an updated version of the original text after the English edition was published, making the Chinese edition more comprehensive.
Academic works, whether in the original or translated form, can be difficult to understand, which is normal. For this reason, one feature of this book is a "guide" at the beginning of each chapter. There is a common saying in the translation of scientific works: "The translation is hard to understand, but the original text is even harder." This is a great failure in translation. The Chinese edition of this book strives to avoid such situations. On the other hand, translations of scientific works should not be arbitrarily creative but should reflect the content and style of the original text. The translation of this book primarily adopts a literal approach, with translator notes added when necessary.
In the sixth chapter, when introducing the development and current status of Chinese information processing technology, the author, who has been working in Hong Kong, focuses on regions outside mainland China. This is understandable and complements the relevant works, journals, and conference proceedings published in mainland China.
Each of the six translated chapters includes the following components: title, author, translator, proofreader, guide, main text, and author bio. Two appendices are included at the end: a Chinese-English glossary and an English index.
The translators and proofreaders—Liu Yang (liuyang@pku.edu.cn), Liu Yun (liuyun@pku.edu.cn), Li Yun (liyun2003@pku.edu.cn), Wu Yunfang (wuyf@pku.edu.cn), Li Suojian (lisujian@pku.edu.cn), Lü Xueqiang (lxq@pku.edu.cn), Zhan Weidong (zwd@pku.edu.cn), and Chang Baobao (chbb@pku.edu.cn)—are all doctoral students from the Institute of Computational Linguistics, Peking University. Dr. Li Zuowen is now a professor at Hebei University, Dr. Li Jinxia is an editor at the journal Language and Character Application, and Peng Guozhen is a Ph.D. student at the Department of Chinese at Peking University. Chen Yirong is a master's student at the Institute of Computational Linguistics, Peking University. In addition to those named, other colleagues from the Institute of Computational Linguistics, Peking University, also contributed to the publication of the Chinese edition. We apologize for not being able to name them all here.
We sincerely thank Mr. Zhou Hongbo of the Commercial Press for his support and Ms. Liu Ling for her wisdom and hard work in editing this book.
Before the 2nd Student Conference on Computational Linguistics, a limited number of copies were distributed to the attendees (excluding the glossary and index) to gather feedback on the translation. The goal was to eliminate flaws and errors before publication. This approach has indeed yielded results.
We hope that the Chinese edition of Computational Linguistics and Beyond will contribute to the development of computational linguistics in China, and we look forward to receiving corrections and suggestions from readers.
Yu Shiwen Huang Juren October 12, 2004