Constructing a Chinese-Japanese Parallel Corpus from Wikipedia Chenhui Chu, Toshiaki Nakazawa, Sadao Kurohashi (Graduate School of Informatics, Kyoto.

Presentation on theme: "Constructing a Chinese-Japanese Parallel Corpus from Wikipedia Chenhui Chu, Toshiaki Nakazawa, Sadao Kurohashi (Graduate School of Informatics, Kyoto."— Presentation transcript:

1 Constructing a Chinese-Japanese Parallel Corpus from Wikipedia Chenhui Chu, Toshiaki Nakazawa, Sadao Kurohashi (Graduate School of Informatics, Kyoto University) Background Features Baseline features General features: sentence length, word overlap Word alignment features Lack of Chinese-Japanese parallel corpora for SMT Corpora Language Domain Europarl European Politic NIST Zh-En, Arab-En News BTEC Asian-En Travel NTCIR Zh-En, Ja-En Patent ASPEC Ja-En, Zh-Ja Science … Chinese character features (+CC) 而被指定为政令指定都市、中核市、特例市。 別途政令指定都市、中核市、特例市に定められている。 Zh: Ja: c Non-CC word features (+Non-CC) # 日本的一级行政区划单位为都道府县，全国划分为 1都、1道、2府、43县。 都道府県（1都1道2府43県）という広域行政区画から構成される。 Zh: Ja: Chinese-Japanese Wikipedia Zh: ．．． 日本的一级行政区划单位为都道府县，全国划分为1都、1道、2府、43县。部份市因人口较多，在当地影响较大，而被指定为政令指定都市、中核市、特例市。都道府县下的行政区划为市町村，此外还有郡、支厅、区、特别区等行政单位。 Ja: ．．． 都道府県（1都1道2府43県）という広域行政区画から構成される。但し、地域区分（地方区分）には、揺れが見られる。また、一部の市は、行政上、別途政令指定都市、中核市、特例市に定められている。他にも、市町村や、町村をまとめた郡がある（全国市町村一覧参照）。 Content word features (+Con) ＹＹ/的/尸体/，/和/活着/的/黑/猩猩/相比/，/皮肤/ 的/颜色/看起来/稍微/明朗/一些/。 つぎに/，/配線/に/使用/する/パターン/幅/や/クリアランス/の/設定/の/方法/を/説明/した/。 Zh: Ja: 1 3 Parallel Sentence Extraction System Experiments p Overview Classification results with WF Classifier Parallel sentences Parallel sentence candidates Zh-Ja Wikipedia Filter Article pairs Inter-language link Common Chinese characters Seed parallel corpus (1) (2) (3) . Bilingual dictionary Common Chinese characters filtering Novel features Extraction results (#extracted sentences [unit: k]) Parallel sentence candidate filtering WF: dictionary-based word overlap (Baseline) CCF: common Chinese character (cognate) overlap WF and CCF: logical conjunction of WF and CCF WF or CCF: logical disjunction of WF and CCF Parallel sentence classifier MT results (BLEU-4) Classifier Filter Cartesian product Non-parallel sentence pairs Filtered non-parallel Positive instances Negative instances Bilingual dictionary Seed parallel corpus Common Chinese characters # The resource is freely available at: 2 4