如何从文本中去除发音准则？答案

【问题标题】：How to strip pronunciation guidelines from text?如何从文本中去除发音准则？
【发布时间】：2017-12-06 07:29:04
【问题描述】：

我正在处理来自 Wikipedia 的一堆文本，我想删除条目中包含的各种发音指南。例如，给定以下条目：

Sigmund Freud (/ˈfrɔɪd/ FROYD; German: [ˈziːkmʊnt ˈfʁɔʏt]; born Sigismund Schlomo Freud; 6 May 1856 – 23 September 1939) was an…
Plato (/ˈpleɪtoʊ/; Greek: Πλάτων Plátōn, pronounced [plá.tɔːn] in Classical Attic; 428/427 or 424/423 – 348/347 BC) was a…
Napoleon Bonaparte (/nəˈpoʊliən ˈboʊnəpɑːrt/; French: [napɔleɔ̃ bɔnapaʁt]; 15 August 1769 – 5 May 1821) was a…
Michael Faraday FRS (/ˈfæ.rəˌdeɪ/; 22 September 1791 – 25 August 1867) was an…
Martin Luther (/ˈluːθər/; German: [ˈmaɐ̯tiːn ˈlʊtɐ]; 10 November 1483 – 18 February 1546), O.S.A., was a…
Louis Pasteur (/ˈluːi pæˈstɜːr/, French: [lwi pastœʁ]; December 27, 1822 – September 28, 1895) was a…

理想情况下，我希望得到更多类似于以下内容的内容：

Sigmund Freud (born Sigismund Schlomo Freud; 6 May 1856 – 23 September 1939) was an…
Plato (428/427 or 424/423 – 348/347 BC) was a…
Napoleon Bonaparte (15 August 1769 – 5 May 1821) was a…
Michael Faraday FRS (22 September 1791 – 25 August 1867) was an…
Martin Luther (10 November 1483 – 18 February 1546), O.S.A., was a…
Louis Pasteur (December 27, 1822 – September 28, 1895) was a…

有没有一种程序化的方式来做到这一点？

【问题讨论】：

你试过什么？你如何区分发音指南和文本？.. 给定样本足够简单的正则表达式是可能的，但这不会很健壮

标签： text awk sed normalization wikipedia

【解决方案1】：

sed解决方案：

sed 's|/[^/]*/[^,;]*[,;]\(.*\[[^][]*\][^;]*;\)* *||g' file

输出：

Sigmund Freud (born Sigismund Schlomo Freud; 6 May 1856 – 23 September 1939) was an…
Plato (428/427 or 424/423 – 348/347 BC) was a…
Napoleon Bonaparte (15 August 1769 – 5 May 1821) was a…
Michael Faraday FRS (22 September 1791 – 25 August 1867) was an…
Martin Luther (10 November 1483 – 18 February 1546), O.S.A., was a…
Louis Pasteur (December 27, 1822 – September 28, 1895) was a…

/[^/]*/[^,;]*[,;] - 匹配发音部分 /.../ 与可选的后续词 [^,;]* 并以 , 或 ; 结尾
\(.*\[[^][]*\][^;]*;\)* - 匹配发音部分 [...] 被可选单词包围（由 .* 和 [^;]* 表示）并以 ; 结尾。所有这些匹配都标记为可选(....)*

【讨论】：