WWW::Mechanize 不处理撇号或破折号

Question

我一直致力于从 Metacritic 中提取信息，但我现在遇到了无法干净地提取带有撇号或破折号的文本的问题。

以下代码说明了这个问题：

use WWW::Mechanize;
 $reviewspage = 'http://www.metacritic.com/movie/a-band-called-death/critic-reviews';
 $Review = 'In the end Death triumphs, but its allure and obsession remain a mystery.';
 $l = WWW::Mechanize->new();
    $l->get($reviewspage);
    $k = $l->content;
    @Review = $k =~ m{$Review.*?<div class="review_body">(.*?)</div>}s;
    print "@Review\n";

输出：

                                Too much of the doc takes our taste for granted; Alice Cooper, Henry Rollins and others won’t persuade you that Death could have been huge, nor does a clichéd last-act reunion show. But the film’s alternating inquiry — into family love, slow compromise and, yes, death — resonates strongly.

即使网站上的编码是：

<div class="review_body">
                                Too much of the doc takes our taste for granted; Alice Cooper, Henry Rollins and others won’t persuade you that Death could have been huge, nor does a clichéd last-act reunion show. But the film’s alternating inquiry — into family love, slow compromise and, yes, death — resonates strongly.
                            </div>

我之前使用 WWW::Mechanize 创建了类似的脚本，但没有一个替换过这样的字符。

Answer 1

Metacritic 使用 utf8 字符集：

<meta http-equiv="content-type" content="text/html; charset=UTF-8">

因此，要将此内容打印到控制台，必须适应该字符集。

在我的 Windows 机器上，我必须在执行我的 perl 脚本之前在控制台中运行 chcp 65001。而且我必须指定 STDOUT character set:

use strict;
use warnings;
use utf8;

use WWW::Mechanize;

binmode STDOUT, ':utf8';   # output should be in UTF-8

my $url = 'http://www.metacritic.com/movie/a-band-called-death/critic-reviews';
my $Review = 'In the end Death triumphs, but its allure and obsession remain a mystery.';

my $lwp = WWW::Mechanize->new();
$lwp->get($url);
my $data = $lwp->content;

if ($data =~ m{$Review.*?<div class="review_body">(.*?)</div>}s) {
    print "$1\n";
} else {
    warn "Review not found";
}

输出（添加换行符）：

Too much of the doc takes our taste for granted; Alice Cooper, Henry Rollins
and others won’t persuade you that Death could have been huge, nor does a
clichéd last-act reunion show. But the film’s alternating inquiry — into
family love, slow compromise and, yes, death — resonates strongly.

Answer 2

显然这是一个 unicode 问题。

根据this answer 中的建议，我能够让你的这个版本的代码工作：

use v5.12 ;
use utf8 ;
use open qw( :encoding(UTF-8) :std ) ;

use WWW::Mechanize ;
 my $reviewspage = 'http://www.metacritic.com/movie/a-band-called-death/critic-reviews' ;
 my $Review = 'In the end Death triumphs, but its allure and obsession remain a mystery.' ;
 my $l = WWW::Mechanize->new() ;
    $l->get($reviewspage) ;   
    my $k = $l->content ;   
    my @Review = $k =~ m{$Review.*?<div class="review_body">(.*?)</div>}s ;
    print "@Review\n" ;

输出：

                                    Too much of the doc takes our taste for granted; Alice Cooper, Henry Rollins and others won’t persuade you that Death could have been huge, nor does a clichéd last-act reunion show. But the film’s alternating inquiry — into family love, slow compromise and, yes, death — resonates strongly.