如何在 IMAP 邮件主题中搜索非 ascii 文本

Question

这是 MWE：

#!/usr/bin/perl

use utf8;
use strict;
use warnings;
use Net::IMAP::Client;
use Encode qw/decode/;
use open ':std', ':encoding(UTF-8)';

my $user = 'my-user@gmail.com';
my $pwd = 'secret';

my $imap = Net::IMAP::Client->new(
    server          => 'imap.gmail.com',
    user            => $user,
    pass            => $pwd,
    ssl             => 1,      # (use SSL? default no)
    ssl_verify_peer => 1,      # (use ca to verify server, default yes)
    port            => 993
) or die "Could not connect to IMAP server: $!";

$imap->login or die('Login failed: ' . $imap->last_error);

# all the incoices from my telephone company
$imap->select('INBOX');
my $messages = $imap->search({
    from    => 'invoice@mgts.ru',
    #subject => '2020',
});

unless(defined($messages))
{
    $imap->logout();
    die "no messages";
}

foreach my $id (@$messages)
{
    my $summary = $imap->get_summaries([$id])->[0];

    my $subject = $summary->subject;
    $subject = decode('MIME-Header', $subject);
    print $subject."\n";
}

这会输出邮箱中invoice@mgts.ru的所有发票：

Счёт за услуги ПАО МГТС за Июнь 2017 г.
Счёт за услуги ПАО МГТС за Июль 2017 г.
Счёт за услуги ПАО МГТС за Август 2017 г.
Счёт за услуги ПАО МГТС за Ноябрь 2017 г.
Счёт за услуги ПАО МГТС за Декабрь 2017 г.
Счёт за услуги ПАО МГТС за Ноябрь 2018 г.
Счёт за услуги ПАО МГТС за Декабрь 2018 г.
Счёт за услуги ПАО МГТС за Декабрь 2019 г.
Счёт за услуги ПАО МГТС за Март 2020 г.
Счёт за услуги ПАО МГТС за Апрель 2020 г.

一切都是正确的。

现在我添加新条件，取消注释：

#subject => '2020',

我收到了 2020 年的所有发票：

Счёт за услуги ПАО МГТС за Март 2020 г.
Счёт за услуги ПАО МГТС за Апрель 2020 г.

但是当我在搜索中添加“Апрель”（四月）这个词时：

subject => 'Апрель 2020',

我得到“没有消息”，尽管这个子字符串存在于邮箱中 1 封电子邮件的主题中。

邮件中的主题如下：

Subject: =?utf-8?Q?=D0=A1=D1=87=D1=91=D1=82=20?=

出了什么问题以及如何解决？

Answer 1

我将开始处理 Unicode：

use strict; use warnings;
use utf8;
binmode $_, ":utf8" for qw/STDOUT STDIN STDERR/;

使用最近的 Perl：

use feature 'unicode_strings';

https://perldoc.perl.org/perlunicode.html#Important-Caveats

Answer 2

（请参阅本文底部的一个似乎可行的 Python 示例）

我尝试了另一个模块Net::IMAP::Simple::Gmail，因为它有debug 输出选项（我首先给自己发送了一封主题为Апрель 2020 的电子邮件，以便我可以轻松测试行为）：

use feature qw(say);
use strict;
use warnings;
use utf8;
use Net::IMAP::Simple::Gmail;
use Encode qw(encode_utf8);

my $server = 'imap.gmail.com';
my $imap = Net::IMAP::Simple::Gmail->new($server, debug => 1);
my $user = 'me@gmail.com';
my $pass = 'mypass';

if(!$imap->login($user,$pass)){
    die "Login failed: " . $imap->errstr . "\n";
}
my $num_messages = $imap->select('INBOX') or die $imap->errstr;
my @ids = $imap->search(encode_utf8('SUBJECT "Апрель 2020"'));
say "Found ", (scalar @ids), " messages";

输出：

[...l/5.30.0/Net/IMAP/Simple.pm line 133 in sub _connect] connecting to imap.gmail.com:993
[...l/5.30.0/Net/IMAP/Simple.pm line 133 in sub _connect] connected, returning socket
[./p.pl line 11 in sub new] waiting for socket ready
[./p.pl line 11 in sub new] looking for greeting
[./p.pl line 11 in sub new] got a greeting: * OK Gimap ready for requests from 51.174.5.83 u18mb43700719ljl\r\n
[...l/5.30.0/Net/IMAP/Simple.pm line 1252 in sub _send_cmd] 0 LOGIN me@gmail.com "mypass"\r\n
[...l/5.30.0/Net/IMAP/Simple.pm line 265 in sub _process_cmd] * CAPABILITY IMAP4rev1 UNSELECT IDLE NAMESPACE QUOTA ID XLIST CHILDREN X-GM-EXT-1 UIDPLUS COMPRESS=DEFLATE ENABLE MOVE CONDSTORE ESEARCH UTF8=ACCEPT LIST-EXTENDED LIST-STATUS LITERAL- SPECIAL-USE APPENDLIMIT=35651584\r\n
[...l/5.30.0/Net/IMAP/Simple.pm line 1277 in sub _cmd_ok] * CAPABILITY IMAP4rev1 UNSELECT IDLE NAMESPACE QUOTA ID XLIST CHILDREN X-GM-EXT-1 UIDPLUS COMPRESS=DEFLATE ENABLE MOVE CONDSTORE ESEARCH UTF8=ACCEPT LIST-EXTENDED LIST-STATUS LITERAL- SPECIAL-USE APPENDLIMIT=35651584\r\n
[...l/5.30.0/Net/IMAP/Simple.pm line 265 in sub _process_cmd] 0 OK hakon.hagland@gmail.com authenticated (Success)\r\n
[...l/5.30.0/Net/IMAP/Simple.pm line 1277 in sub _cmd_ok] 0 OK hakon.hagland@gmail.com authenticated (Success)\r\n
[...l/5.30.0/Net/IMAP/Simple.pm line 1252 in sub _send_cmd] 1 SELECT "INBOX"\r\n
[...l/5.30.0/Net/IMAP/Simple.pm line 413 in sub _process_cmd] * FLAGS (\Answered \Flagged \Draft \Deleted \Seen $MailFlagBit0 $MailFlagBit1 $NotJunk $NotPhishing $Phishing NotJunk)\r\n
[...l/5.30.0/Net/IMAP/Simple.pm line 1277 in sub _cmd_ok] * FLAGS (\Answered \Flagged \Draft \Deleted \Seen $MailFlagBit0 $MailFlagBit1 $NotJunk $NotPhishing $Phishing NotJunk)\r\n
[...l/5.30.0/Net/IMAP/Simple.pm line 413 in sub _process_cmd] * OK [PERMANENTFLAGS (\Answered \Flagged \Draft \Deleted \Seen $MailFlagBit0 $MailFlagBit1 $NotJunk $NotPhishing $Phishing NotJunk \*)] Flags permitted.\r\n
[...l/5.30.0/Net/IMAP/Simple.pm line 1277 in sub _cmd_ok] * OK [PERMANENTFLAGS (\Answered \Flagged \Draft \Deleted \Seen $MailFlagBit0 $MailFlagBit1 $NotJunk $NotPhishing $Phishing NotJunk \*)] Flags permitted.\r\n
[...l/5.30.0/Net/IMAP/Simple.pm line 413 in sub _process_cmd] * OK [UIDVALIDITY 638142060] UIDs valid.\r\n
[...l/5.30.0/Net/IMAP/Simple.pm line 1277 in sub _cmd_ok] * OK [UIDVALIDITY 638142060] UIDs valid.\r\n
[...l/5.30.0/Net/IMAP/Simple.pm line 413 in sub _process_cmd] * 27869 EXISTS\r\n
[...l/5.30.0/Net/IMAP/Simple.pm line 1277 in sub _cmd_ok] * 27869 EXISTS\r\n
[...l/5.30.0/Net/IMAP/Simple.pm line 413 in sub _process_cmd] * 0 RECENT\r\n
[...l/5.30.0/Net/IMAP/Simple.pm line 1277 in sub _cmd_ok] * 0 RECENT\r\n
[...l/5.30.0/Net/IMAP/Simple.pm line 413 in sub _process_cmd] * OK [UIDNEXT 32724] Predicted next UID.\r\n
[...l/5.30.0/Net/IMAP/Simple.pm line 1277 in sub _cmd_ok] * OK [UIDNEXT 32724] Predicted next UID.\r\n
[...l/5.30.0/Net/IMAP/Simple.pm line 413 in sub _process_cmd] * OK [HIGHESTMODSEQ 4375397]\r\n
[...l/5.30.0/Net/IMAP/Simple.pm line 1277 in sub _cmd_ok] * OK [HIGHESTMODSEQ 4375397]\r\n
[...l/5.30.0/Net/IMAP/Simple.pm line 413 in sub _process_cmd] 1 OK [READ-WRITE] INBOX selected. (Success)\r\n
[...l/5.30.0/Net/IMAP/Simple.pm line 1277 in sub _cmd_ok] 1 OK [READ-WRITE] INBOX selected. (Success)\r\n
[...l/5.30.0/Net/IMAP/Simple.pm line 1252 in sub _send_cmd] 2 SEARCH SUBJECT "Апрель 2020"\r\n
[...l/5.30.0/Net/IMAP/Simple.pm line 615 in sub _process_cmd] 2 BAD Could not parse command\r\n
[...l/5.30.0/Net/IMAP/Simple.pm line 1277 in sub _cmd_ok] 2 BAD Could not parse command\r\n
[...l/5.30.0/Net/IMAP/Simple.pm line 1192 in sub _seterrstr] Could not parse command\r
Found 0 messages

注意输出：

[...l/5.30.0/Net/IMAP/Simple.pm line 1277 in sub _cmd_ok] 2 BAD Could not parse command\r\n

这是在line 1260处返回2 BAD Could not parse command的套接字

---

同样，使用 Python IMAPClient 模块：

user = 'me@gmail.com';
passw = "mypass";
from imapclient import IMAPClient
server = IMAPClient('imap.gmail.com', use_uid=True, ssl=True)
result = server.login(user, passw)
print(result)
select_info = server.select_folder('INBOX')
print(select_info)
messages = server.search(['SUBJECT', 'Апрель 2020'.encode('utf8')])
print(messages)

给出输出：

b'me@gmail.com authenticated (Success)'
{b'PERMANENTFLAGS': (b'\\Answered', b'\\Flagged', b'\\Draft', b'\\Deleted', b'\\Seen', b'$MailFlagBit0', b'$MailFlagBit1', b'$NotJunk', b'$NotPhishing', b'$Phishing', b'NotJunk', b'\\*'), b'FLAGS': (b'\\Answered', b'\\Flagged', b'\\Draft', b'\\Deleted', b'\\Seen', b'$MailFlagBit0', b'$MailFlagBit1', b'$NotJunk', b'$NotPhishing', b'$Phishing', b'NotJunk'), b'UIDVALIDITY': 638142060, b'EXISTS': 27869, b'RECENT': 0, b'UIDNEXT': 32724, b'HIGHESTMODSEQ': 4375417, b'READ-WRITE': True}
Traceback (most recent call last):
  File "/home/hakon/.pyenv/versions/3.7.3/lib/python3.7/site-packages/imapclient/imapclient.py", line 982, in _search
    data = self._raw_command_untagged(b'SEARCH', args)
  File "/home/hakon/.pyenv/versions/3.7.3/lib/python3.7/site-packages/imapclient/imapclient.py", line 1445, in _raw_command_untagged
    typ, data = self._raw_command(command, args, uid=uid)
  File "/home/hakon/.pyenv/versions/3.7.3/lib/python3.7/site-packages/imapclient/imapclient.py", line 1507, in _raw_command
    return self._imap._command_complete(to_unicode(command), tag)
  File "/home/hakon/.pyenv/versions/3.7.3/lib/python3.7/imaplib.py", line 1027, in _command_complete
    raise self.error('%s command error: %s %s' % (name, typ, data))
imaplib.error: SEARCH command error: BAD [b'Could not parse command']

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "./t.py", line 11, in <module>
    messages = server.search(['SUBJECT', 'Апрель 2020'.encode('utf8')])
  File "/home/hakon/.pyenv/versions/3.7.3/lib/python3.7/site-packages/imapclient/imapclient.py", line 956, in search
    return self._search(criteria, charset)
  File "/home/hakon/.pyenv/versions/3.7.3/lib/python3.7/site-packages/imapclient/imapclient.py", line 995, in _search
    criteria='"%s"' % criteria if not isinstance(criteria, list) else criteria
imapclient.exceptions.InvalidCriteriaError: b'Could not parse command'

This error may have been caused by a syntax error in the criteria: ['SUBJECT', b'\xd0\x90\xd0\xbf\xd1\x80\xd0\xb5\xd0\xbb\xd1\x8c 2020']
Please refer to the documentation for more information about search criteria syntax..
https://imapclient.readthedocs.io/en/master/#imapclient.IMAPClient.search

---

最后我发现a Python example 似乎确实有效：

import imaplib
user = "me@gmail.com"
passw = "mypass";
sock = imaplib.IMAP4_SSL("imap.gmail.com", 993)
sock.login(user, passw)
sock.select()
sock.debug = 4
sock.literal = u"Апрель 2020".encode('utf8')
res = sock.uid('SEARCH', 'CHARSET', 'UTF-8', 'SUBJECT')
print(res)

输出：

  04:44.46 > b'DMOM3 UID SEARCH CHARSET UTF-8 SUBJECT {17}'
  04:44.52 < b'+ go ahead'
  04:44.52 write literal size 17
  04:44.67 < b'* SEARCH 32720'
  04:44.67 < b'DMOM3 OK SEARCH completed (Success)'
('OK', [b'32720'])

Answer 3

1. Net::IMAP::Client 似乎不支持涵盖 UTF-8 编码字符串的IMAP4rev1。

2. 根据 Gmail 文档，您可以使用X-GM-RAW 属性来获取与 Gmail 网页界面相同的结果。

如果我必须使用Net::IMAP::Client，我会添加一个新方法来执行X-GM-RAW：

#!/usr/bin/perl

use utf8;
use strict;
use warnings;
use Net::IMAP::Client;
use Encode qw/decode/;
use IO::Socket qw(:crlf);
use open ':std', ':encoding(UTF-8)';

my $user = 'my-user@gmail.com';
my $pwd = 'secret';

my $imap = Net::IMAP::Client->new(
    server          => 'imap.gmail.com',
    user            => $user,
    pass            => $pwd,
    ssl             => 1,      # (use SSL? default no)
    ssl_verify_peer => 1,      # (use ca to verify server, default yes)
    port            => 993
) or die "Could not connect to IMAP server: $!";

$imap->login or die('Login failed: ' . $imap->last_error);

# Add search_gmail method to Net::IMAP::Client
sub Net::IMAP::Client::search_gmail {
    my ($self, $criteria) = @_;

    my @crit;
    for my $key (keys %{$criteria}) {
        push @crit, join ":", $key, $criteria->{$key};
    }

    my $crit_str = join q{ }, @crit;

    my ($ok, $lines);
    ($ok, $lines) = $self->_tell_imap('SEARCH' => "CHARSET UTF-8 X-GM-RAW " . do {
        use bytes;
        sprintf qq{{%d}%s%s}, length($crit_str), $CRLF, $crit_str;
    });

    return unless $ok;

    for my $line (@{$lines->[1]}) {
        if ($line =~ s/^\*\s+SEARCH\s+//ig) {
            $line =~ s/\s*$//g;
            return [ map { $_ + 0 } split(/\s+/, $line) ];
        }
    }
}

# all the incoices from my telephone company
$imap->select('INBOX');
my $messages = $imap->search_gmail({
    from    => 'invoice@mgts.ru',
    #subject => '2020',
});

unless(defined($messages))
{
    $imap->logout();
    die "no messages";
}

foreach my $id (@$messages)
{
    my $summary = $imap->get_summaries([$id])->[0];

    my $subject = $summary->subject;
    $subject = decode('MIME-Header', $subject);
    print $subject."\n";
}