awk 获取包含逗号和换行符的 .csv 列[重复]答案

【问题标题】：Awk to get .csv column containing commas and newlines [duplicate]awk 获取包含逗号和换行符的 .csv 列[重复]
【发布时间】：2016-12-08 11:35:09
【问题描述】：

我的 .csv 列中的数据有时包含逗号和换行符。如果我的数据中有逗号，我会用双引号将整个字符串括起来。在考虑换行符和逗号的情况下，我将如何将该列的输出解析为 .txt 文件。

不适用于我的命令的示例数据：

,"This is some text with a , in it.", #data with commas are enclosed in double quotes

,line 1 of data
line 2 of data, #data with a couple of newlines

,"Data that may a have , in it and
also be on a newline as well.",

这是我目前所拥有的：

awk -F "\"*,\"*" '{print $4}' file.csv > column_output.txt

【问题讨论】：

您能否在双引号分隔的字段中转义双引号，如果是，它们是如何转义的，例如"foo\"bar" 或 "foo""bar" 或其他？

标签： linux bash awk

【解决方案1】：

$ cat decsv.awk
BEGIN { FPAT = "([^,]*)|(\"[^\"]+\")"; OFS="," }
{
    # create strings that cannot exist in the input to map escaped quotes to
    gsub(/a/,"aA")
    gsub(/\\"/,"aB")
    gsub(/""/,"aC")

    # prepend previous incomplete record segment if any
    $0 = prev $0
    numq = gsub(/"/,"&")
    if ( numq % 2 ) {
        # this is inside double quotes so incomplete record
        prev = $0 RT
        next
    }
    prev = ""

    for (i=1;i<=NF;i++) {
        # map the replacement strings back to their original values
        gsub(/aC/,"\"\"",$i)
        gsub(/aB/,"\\\"",$i)
        gsub(/aA/,"a",$i)
    }

    printf "Record %d:\n", ++recNr
    for (i=0;i<=NF;i++) {
        printf "\t$%d=<%s>\n", i, $i
    }
    print "#######"

$ awk -f decsv.awk file
Record 1:
        $0=<,"This is some text with a , in it.", #data with commas are enclosed in double quotes>
        $1=<>
        $2=<"This is some text with a , in it.">
        $3=< #data with commas are enclosed in double quotes>
#######
Record 2:
        $0=<,"line 1 of data
line 2 of data", #data with a couple of newlines>
        $1=<>
        $2=<"line 1 of data
line 2 of data">
        $3=< #data with a couple of newlines>
#######
Record 3:
        $0=<,"Data that may a have , in it and
also be on a newline as well.",>
        $1=<>
        $2=<"Data that may a have , in it and
also be on a newline as well.">
        $3=<>
#######
Record 4:
        $0=<,"Data that \"may\" a have ""quote"" in it and
also be on a newline as well.",>
        $1=<>
        $2=<"Data that \"may\" a have ""quote"" in it and
also be on a newline as well.">
        $3=<>
#######

上面使用 GNU awk 进行 FPAT 和 RT。我不知道有任何 CSV 格式可以让你在一个没有用引号括起来的字段中间有一个换行符（如果有的话，你永远不知道任何记录在哪里结束）所以脚本不允许那。以上是在这个输入文件上运行的：

$ cat file
,"This is some text with a , in it.", #data with commas are enclosed in double quotes
,"line 1 of data
line 2 of data", #data with a couple of newlines
,"Data that may a have , in it and
also be on a newline as well.",
,"Data that \"may\" a have ""quote"" in it and
also be on a newline as well.",

【讨论】：