Perl 文本文件之间的无序差异

Question

我基本上想在两个文本文件（CSV 样式）之间进行无序差异，我比较前两列中的字段（我不关心第三列的值）。然后我打印出 file1.txt 具有但在 file2.txt 中不存在的值，反之亦然，file2.txt 与 file1.txt 相比。

file1.txt：

cat,val 1,43432
cat,val 2,4342
dog,value,23
cat2,value,2222
hedgehog,input,233

file2.txt：

cat2,value,312
cat,val 2,11
cat,val 3,22
dog,value,23
hedgehog,input,2145
bird,output,9999

输出会是这样的：

file1.txt:
cat,val 1,43432

file2.txt:
cat,val 3,22
bird,output,9999

我是 Perl 的新手，所以目前我不了解一些更好、更不丑陋的方法来实现这一点。感谢您的帮助。

当前代码：

#!/usr/bin/perl -w

use Cwd;
use strict;
use Data::Dumper;
use Getopt::Long;

my $myName = 'MyDiff.pl';
my $usage = "$myName is blah blah blah";

#retreive the command line options, set up the environment
 use vars qw($file1 $file2);

#grab the specified values or exit program
GetOptions("file1=s" => \$file1,
        "file2=s" => \$file2) 
        or die $usage;
 ( $file1 and $file2 ) or die $usage;

 open (FH, "< $file1") or die "Can't open $file1 for read: $!";
 my @array1 = <FH>;
 close FH or die "Cannot close $file1: $!"; 
 open (FH, "< $file2") or die "Can't open $file2 for read: $!";
 my @array2 = <FH>;
 close FH or die "Cannot close $file2: $!"; 

 #...do a sort and match

Answer 1

为此使用哈希，前 2 列作为键。 一旦你有了这两个哈希，你就可以迭代和删除公共条目， 各个哈希中剩余的内容就是您要查找的内容。

初始化，

my %hash1 = ();
my %hash2 = ();

读入第一个文件，连接前两列以形成密钥并将其保存在哈希中。这假定字段是逗号分隔的。您也可以使用 CSV 模块。

open( my $fh1, "<", $file1 ) || die "Can't open $file1: $!";
while(my $line = <$fh1>) {
    chomp $line;

    # join first two columns for key
    my $key = join ",", (split ",", $line)[0,1];

    # create hash entry for file1
    $hash1{$key} = $line;
}

对 file2 执行相同操作并创建 %hash2

open( my $fh2, "<", $file2 ) || die "Can't open $file2: $!";
while(my $line = <$fh2>) {
    chomp $line;

    # join first two columns for key
    my $key = join ",", (split ",", $line)[0,1];

    # create hash entry for file2
    $hash2{$key} = $line;
}

现在检查条目并删除常见的条目，

foreach my $key (keys %hash1) {
    if (exists $hash2{$key}) {
        # common entry, delete from both hashes
        delete $hash1{$key};
        delete $hash2{$key};
    }
}

%hash1 现在将包含仅在 file1 中的行。

您可以将它们打印为，

foreach my $key (keys %hash1) {
    print "$hash1{$key}\n";
}

foreach my $key (keys %hash2) {
    print "$hash2{$key}\n";
}

Answer 2

我认为上述问题可以通过上述任一算法解决

a) 我们可以使用上面提到的哈希

b) 1. 使用 Key1 和 Key2 对文件进行排序（使用排序乐趣）

遍历 FILE1

  Match the key1 and key2 entry of FILE1 with FILE2
      If yes then
        take action by printing common lines it to desired file as required
        Move to next row in File1 (continue with the loop )
      If No then
        Iterate through File2 startign from the POS-FILE2 until match is found
            Match the key1 and key2 entry of FILE1 with FILE2
            If yes then
              take action by printing common lines it to desired file as required
              setting FILE2-END as true
              exit from the loop noting the position of FILE2
            If no then
              take action by printing unmatched lines to desired file as req.
              Move to next row in File2
  If FILE2-END is true
     Rest of Lines in FILE1 doesnt exist in FILE2

Answer 3

也许以下内容会有所帮助：

use strict;
use warnings;

my @files = @ARGV;
pop;
my %file1 = map { chomp; /(.+),/; $1 => $_ } <>;

push @ARGV, $files[1];
my %file2 = map { chomp; /(.+),/; $1 => $_ } <>;

print "$files[0]:\n";
print $file1{$_}, "\n" for grep !exists $file2{$_}, keys %file1;

print "\n$files[1]:\n";
print $file2{$_}, "\n" for grep !exists $file1{$_}, keys %file2;

用法：perl script.pl file1.txt file2.txt

数据集的输出：

file1.txt:
cat,val 1,43432

file2.txt:
cat,val 3,22
bird,output,9999

这将为每个文件构建一个哈希。键是前两列，关联的值是整行。 grep 用于过滤共享密钥。

编辑：在相对较小的文件上，使用上面的map 处理文件的行会正常工作。但是，首先创建文件所有行的列表，然后将其传递给map。在较大的文件上，最好使用while (<>) { ... 构造，一次读取一行。下面的代码执行此操作——生成与上面相同的输出——并使用哈希哈希 (HoH)。因为它使用 HoH，所以您会注意到一些取消引用：

use strict;
use warnings;

my %hash;
my @files = @ARGV;

while (<>) {
    chomp;
    $hash{$ARGV}{$1} = $_ if /(.+),/;
}

print "$files[0]:\n";
print $hash{ $files[0] }{$_}, "\n"
  for grep !exists $hash{ $files[1] }{$_}, keys %{ $hash{ $files[0] } };

print "\n$files[1]:\n";
print $hash{ $files[1] }{$_}, "\n"
  for grep !exists $hash{ $files[0] }{$_}, keys %{ $hash{ $files[1] } };