Python CSV模块处理字段内引号内的逗号

Question

我正在使用 Python 的 csv 模块从我的应用程序中的 CSV 文件中解析数据。在测试应用程序时，我的同事输入了一段从随机网站复制粘贴的示例文本。

示例文本在字段内有双引号，在双引号内有一个逗号。双引号外的逗号由 csv 模块正确处理，但双引号内的逗号被拆分到下一列。我查看了 csv 规范，该字段确实符合规范，将双引号转义为另一组双引号。

我检查了 libreoffice 中的文件，并且处理正确。

这是我遇到问题的 csv 数据中的一行：

company_name,company_revenue,company_start_year,company_website,company_description,company_email
Acme Inc,80000000000000,2004,http://google.com,"The company is never clearly defined in Road Runner cartoons but appears to be a conglomerate which produces every product type imaginable, no matter how elaborate or extravagant - most of which never work as desired or expected. In the Road Runner cartoon Beep, Beep, it was referred to as ""Acme Rocket-Powered Products, Inc."" based in Fairfield, New Jersey. Many of its products appear to be produced specifically for Wile E. Coyote; for example, the Acme Giant Rubber Band, subtitled ""(For Tripping Road Runners)"".

Sometimes, Acme can also send living creatures through the mail, though that isn't done very often. Two examples of this are the Acme Wild-Cat, which had been used on Elmer Fudd and Sam Sheepdog (which doesn't maul its intended victim); and Acme Bumblebees in one-fifth bottles (which sting Wile E. Coyote). The Wild Cat was used in the shorts Don't Give Up the Sheep and A Mutt in a Rut, while the bees were used in the short Zoom and Bored.

While their products leave much to be desired, Acme delivery service is second to none; Wile E. can merely drop an order into a mailbox (or enter an order on a website, as in the Looney Tunes: Back in Action movie), and have the product in his hands within seconds.",roadrunner@acme.com

这是调试日志中的样子：

2014-08-27 21:35:53,922 - DEBUG: company_website=http://google.com
2014-08-27 21:35:53,923 - DEBUG: company_revenue=80000000000000
2014-08-27 21:35:53,923 - DEBUG: company_start_year=2004
2014-08-27 21:35:53,923 - DEBUG: account_description=The company is never clearly defined in Road Runner cartoons but appears to be a conglomerate which produces every product type imaginable, no matter how elaborate or extravagant - most of which never work as desired or expected. In the Road Runner cartoon Beep, Beep, it was referred to as "Acme Rocket-Powered Products
2014-08-27 21:35:53,924 - DEBUG: company_name=Acme Inc
2014-08-27 21:35:53,925 - DEBUG: company_email=Inc."" based in Fairfield

处理csv解析的相关代码：

with open(csvfile, 'rU') as contactsfile:
    # sniff for dialect of csvfile so we can automatically determine
    # what delimiters to use
    try:
        dialect = csv.Sniffer().sniff(contactsfile.read(2048))
    except:
        dialect = 'excel'
    get_total_jobs(contactsfile, dialect)
    contacts = csv.DictReader(contactsfile, dialect=dialect, skipinitialspace=True, quoting=csv.QUOTE_MINIMAL)
    # Start reading the rows
    for row in contacts:
        process_job()
        for key, value in row.iteritems():
            logging.debug("{}={}".format(key,value))

我知道这只是垃圾数据，我们可能永远不会遇到这样的数据，但我们收到的 csv 文件不在我们的控制范围内，我们可能会遇到这样的边缘情况。由于它是一个有效的 csv 文件，由 libreoffice 正确处理，因此我也可以正确处理它。

我搜索了有关 csv 处理的其他问题，其中人们在处理字段中的引号或逗号时遇到问题。我有这两个工作正常，我的问题是逗号嵌套在字段内的引号内。有一个同样问题的问题确实解决了Comma in DoubleDouble Quotes in CSV File 的问题，但这是一种骇人听闻的方式，我没有保留提供给我的内容，这是根据 RFC4180 的有效方式。

Answer 1

Dialect.doublequote attribute

控制 quotechar 实例出现在字段中的方式 自己被引用。为 True 时，字符翻倍。假时， escapechar 用作quotechar 的前缀。它默认为 没错。

嗅探器将双引号属性设置为 False，但您发布的 CSV 应使用 doublequote = True 解析：

import csv
with open(csvfile, 'rb') as contactsfile:
    # sniff for dialect of csvfile so we can automatically determine
    # what delimiters to use
    try:
        dialect = csv.Sniffer().sniff(contactsfile.read(2048))
    except:
        dialect = 'excel'
    # get_total_jobs(contactsfile, dialect)
    contactsfile.seek(0)
    contacts = csv.DictReader(contactsfile, dialect=dialect, skipinitialspace=True,
                              quoting=csv.QUOTE_MINIMAL, doublequote=True)
    # Start reading the rows
    for row in contacts:
        for key, value in row.iteritems():
            print("{}={}".format(key,value))

产量

company_description=The company is never clearly defined in Road Runner cartoons but appears to be a conglomerate which produces every product type imaginable, no matter how elaborate or extravagant - most of which never work as desired or expected. In the Road Runner cartoon Beep, Beep, it was referred to as "Acme Rocket-Powered Products, Inc." based in Fairfield, New Jersey. Many of its products appear to be produced specifically for Wile E. Coyote; for example, the Acme Giant Rubber Band, subtitled "(For Tripping Road Runners)".

Sometimes, Acme can also send living creatures through the mail, though that isn't done very often. Two examples of this are the Acme Wild-Cat, which had been used on Elmer Fudd and Sam Sheepdog (which doesn't maul its intended victim); and Acme Bumblebees in one-fifth bottles (which sting Wile E. Coyote). The Wild Cat was used in the shorts Don't Give Up the Sheep and A Mutt in a Rut, while the bees were used in the short Zoom and Bored.

While their products leave much to be desired, Acme delivery service is second to none; Wile E. can merely drop an order into a mailbox (or enter an order on a website, as in the Looney Tunes: Back in Action movie), and have the product in his hands within seconds.
company_website=http://google.com
company_start_year=2004
company_name=Acme Inc
company_revenue=80000000000000
company_email=roadrunner@acme.com

---

另外，per the docs，在 Python2 中，文件句柄应该以 'rb' 模式打开，而不是 'rU' 模式：

如果 csvfile 是一个文件对象，它必须以‘b’标志打开 不同的平台。