R .csv 无法正确读取，因为文本中有双引号

Question

我有一个包含所有文本字段的 .csv 文件。但是，某些文本字段包含未转义的双引号字符，例如：

"ID","Text","Optional text","Date" 
"1","Today is going to be a good day","","2013-02-03"
"2","And I am inspired by the quote "every dog must have it's day"","Hi","2013-01-01"
"3","Did not the bard say All the World's a stage" this quote is so true","Terrible","2013-05-05"

第 1 行和第 2 行很好，但第 3 行没有正确读入。目前，我正在手动浏览 Notepad++ 中的文件以尝试删除此类引号。理想情况下，我希望 R 能够处理这个问题，但我认为无与伦比的双引号的未转义性质使得这种期望不合理。

在 Notepad++ 中，我正在尝试构建一个正则表达式来识别没有逗号之前或之后的双引号。逻辑是有效的双引号将位于字段的开头或结尾，这由相邻的逗号表示。这可能有助于确定我的大部分案件，然后我可以处理这些案件。

只是说我有大约 340 万条记录，大约 0.1% 似乎有问题。

编辑： 已建议使用来自 data.table 的 fread 作为替代方案，但使用 fread 更不成功：

1: In fread(paste(infilename, "1", ".csv", sep = "")) :
  Stopped early on line 21. Expected 18 fields but found 9. Consider fill=TRUE and comment.char=. First discarded non-empty line

建议的选项均无效。我认为这是因为“文本”字段也可以包含 CRLF 字符。 read.csv 似乎只是忽略了这些（好），而 fread 例外。抱歉，我无法提供实际文本，但这里有一些更全面的测试数据，其中包含不匹配的双引号（read.csv 有问题）和 CRLF（fread 有问题）。

"ID","Text","Optional text","Date" 
"1","Today is going to be a good day","","2013-02-03"
"2","And I am inspired by the quote "every dog must have it's day"","Hi","2013-01-01"
"3","An issue with this line is that it contains a CRLF here 
which is not usual.","Again an unusual CRLF
is present in these data","2013-02-02"
"4","Did not the bard say All the World's a stage" this quote is so true","Terrible","2013-05-05"

帮助使用 Notepad++ 中的正则表达式会很棒。

Answer 1

也许一种选择是在 notepad++ 中使用条件替换。

您可以找到所有以双引号开头、以逗号开头或在字符串开头的字符串。

然后不匹配双引号，直到遇到下一个逗号后跟或字符串结尾的双引号。这些是白色的线是可以的，所以对于你想要捕获和替换的替换部分，匹配一个不在逗号之间的双引号。

查找内容：

(?:^|,)"[^"\n]*"(?=$|,)|(?<!,)(")(?!,)

替换为：

有条件的替换。如果是第 1 组，则替换为空，否则替换为匹配项。

(?{1}:$0)

Regex demo

说明

• (?:^|,) 匹配逗号或断言字符串的开头
• "[^"\n]*" 中间没有双引号时匹配双引号
• (?=$|,)断言右边是字符串的结尾或逗号
• | 或
• (?<!,)(")(?!,)在 group1 中捕获双引号，同时断言左侧和右侧的内容不是逗号

Answer 2

似乎与data.table::fread 配合得很好：

fread("E:/temp/test.txt")
#   ID                                                                 Text Optional text     "Date"
#1:  1                                      Today is going to be a good day               2013-02-03
#2:  2        And I am inspired by the quote "every dog must have it's day"            Hi 2013-01-01
#3:  3 Did not the bard say "All the World's a stage" this quote is so true      Terrible 2013-05-05
#Warning message:
#In fread("E:/temp/test.txt") :
#  Found and resolved improper quoting in first 100 rows. If the fields are not quoted (e.g. field separator does not appear within any field), try quote="" to avoid this warning.