如何匹配列字段并将它们的值组合在一起？

Question

我正在对我使用 pdfgrep 创建的一些文件进行排序，以列出我拥有的某些 PDF 的页码。它产生了以下输出：

./Buddhism in the Shadow of Brahmanism.pdf:111:      Then, rising from his seat, covering one shoulder with his robe, the king
./Buddhism in the Shadow of Brahmanism.pdf:182:branch who has adopted the yellow robes of Buddhism; he is sur-
./Buddhism in the Shadow of Brahmanism.pdf:229:       resolve that his body, his bowl, and his monastic robe (which had been
./Buddhism in the Shadow of Brahmanism.pdf:230:robe. In this way, Mahākāśyapa (or at least his body) is to act as a sort
./Buddhism in the Shadow of Brahmanism.pdf:230:corpse to his disciples and displays to them the Buddha’s robe, and they
./Buddhism in the Shadow of Brahmanism.pdf:230:offer him the robe that the Buddha had confided to him. Only then will
./Introduction to the History of Indian Buddhism.pdf:31:the robes of a Buddhist monk in an effort to convert them, he was Sciequia. For
./Introduction to the History of Indian Buddhism.pdf:54:monks, and in particular on retreat, robes, and chastity, p. 308.—On the life of
./Introduction to the History of Indian Buddhism.pdf:97:are the Kat.hināvadāna, which deals with the bowl, the staff, and the robes of
./Introduction to the History of Indian Buddhism.pdf:111:of a sort of robe.
./Introduction to the History of Indian Buddhism.pdf:112:cover his nakedness, and who rejects all other robes as superfluous.
./Introduction to the History of Indian Buddhism.pdf:127:noon, after having taken his robe and his bowl,
./Introduction to the History of Indian Buddhism.pdf:127:bowl and his robe, he went to the place where the Cāpāla caitya6 was located,

我想要做的是将第二列上与文件名匹配的页码组合在一起，我希望输出看起来像：

./Buddhism in the Shadow of Brahmanism.pdf:111, 182, 229, 230
./Introduction to the History of Indian Buddhism.pdf:31, 54, 97, 111, 112, 127

我尝试使用 awk 解析第一个值，然后在同一个文件上使用这些结果仅打印页码，这样我就可以 grep 结果并稍后附加到文件名之后，如下所示：

awk -F : '{print $1}' parsing_file | uniq | while read line; do awk -v number="$line" -F : '$1 == "$number" { print $2 }' parsing_file; done 

但这并没有通过，我猜 uniq 和 while read 可能会被删除，也许只使用带有 awk 的数组？

我在这里看到过类似的事情：

https://unix.stackexchange.com/questions/167280/awk-group-by-and-sum-column-values

但我不想总结列上的值，而只是想将它们组合在一起。

谢谢

Answer 1

使用您展示的示例，请尝试以下操作。用 GNU awk 编写和测试。

awk  -v OFS=":" '
match($0,/^\.\/.*\.pdf:[0-9]+/){
  value=substr($0,RSTART,RLENGTH)
  split(value,arr,":")
  if(!seen[arr[1],arr[2]]++){
    name[arr[1]]=(name[arr[1]]?name[arr[1]]", ":"")arr[2]
  }
}
END{
  for(key in name){
    print key,name[key]
  }
}
'  Input_file

您显示的示例输出如下：

./Buddhism in the Shadow of Brahmanism.pdf:111, 182, 229, 230
./Introduction to the History of Indian Buddhism.pdf:31, 54, 97, 111, 112, 127

说明：为上述添加详细说明。

awk  -v OFS=":" '                   ##Starting awk program from here.
match($0,/^\.\/.*\.pdf:[0-9]+/){    ##Using match function to match from starting ./ till .pdf : digits as per shown samples.
  value=substr($0,RSTART,RLENGTH)   ##Creating value with matched sub string here.
  split(value,arr,":")              ##Splitting value into array arr with : delimiter.
  if(!seen[arr[1],arr[2]]++){
     name[arr[1]]=(name[arr[1]]?name[arr[1]]", ":"")arr[2]  ##Creating name array with index of book name and its value it digits as per needed output.
  }
}
END{                                ##Starting END block of this program from here.
  for(key in name){                 ##Traversing through name here.
    print key,name[key]             ##Printing key and array value here.
  }
}
' Input_file                        ##Mentioning Input_file name here.

注意：之前的上述解决方案没有处理同一段落的重复数字，因此我编辑了解决方案以在 Ed 回答后处理这种情况。

Answer 2

$ cat tst.awk
BEGIN { FS=OFS=":" }
$1 != title {
    if ( title != "" ) {
        print title, pages
    }
    title = $1
    pages = $2
    delete seen
    next
}
!seen[$2]++ {
    pages = pages ", " $2
}
END {
    print title, pages
}

$ awk -f tst.awk file
./Buddhism in the Shadow of Brahmanism.pdf:111, 182, 229, 230
./Introduction to the History of Indian Buddhism.pdf:31, 54, 97, 111, 112, 127

在我写这篇文章时（在那个答案被接受之后），上述内容和@Ravinder's answer 之间的区别是：

1. 这取决于您的输入按标题和页码排序，如您问题中的示例所示，而 Ravinders 没有。如果您的真实数据未排序，请运行sort -t':' -k1,1 -k2,2n file | awk ...'。
2. 这不会将所有标题和页码读入内存，它只会一次读取 1 个标题的页码，因此无论您的输入文件有多大，它都能正常工作。
3. 这将按照它们在输入中出现的顺序而不是随机顺序输出标题。
4. 这会通过仅输出每个标题的唯一页码而不是相同的页码来产生预期的输出 每次出现在输入中时都会输出页码（例如 111, 182, 229, 230 与 111, 182, 229, 230, 230, 230。
5. 这会产生预期的输出，方法是在标题和第一个页码之间放置 : 而不是空白。