如何正确解析收到的电子邮件标头？

【问题标题】：How to properly parse email received headers?如何正确解析收到的电子邮件标头？
【发布时间】：2020-10-25 17:10:03
【问题描述】：

解析接收到的 .eml 文件的标头以提取所有跃点信息的正确方法是什么？ 特别是我需要提取以下信息：

  • 发件人网址
  • 发件人 IP
  • 接收方网址
  • 接收器 IP
  • 日期
  • 协议

我找到了以下规范，但似乎没有标准约定 接收到的 headers 的格式，可能因服务器而异：

  1. RFC 2821 - Received Lines in Gatewaying
  2. RFC 2822 - Trace Fields
  3. RFC 822 - 4.1 Syntax

对我来说最清楚的解释是来自RFC 822 spec：

 received    =  "Received"    ":"           ; one per relay
                  ["from" domain]           ; sending host
                  ["by"   domain]           ; receiving host
                  ["via"  atom]             ; physical path
                 *("with" atom)             ; link/mail protocol
                  ["id"   msg-id]           ; receiver msg id
                  ["for"  addr-spec]        ; initial form



                   ";"    date-time         ; time received

考虑以下received 标头

Received: from VE1PR01MB5599.eurprd01.prod.exchangelabs.com
 (2603:10a6:7:7c::43) by HE1PR0102MB2714.eurprd01.prod.exchangelabs.com with
 HTTPS via HE1PR0402CA0054.EURPRD04.PROD.OUTLOOK.COM; Thu, 9 Jan 2020 16:34:13
 +0000

Received: from VI1PR0102CA0029.eurprd01.prod.exchangelabs.com
 (2603:10a6:802::42) by VE1PR01MB5599.eurprd01.prod.exchangelabs.com
 (2603:10a6:803:11f::30) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2602.12; Thu, 9 Jan
 2020 16:34:13 +0000

Received: from DB5EUR01FT034.eop-EUR01.prod.protection.outlook.com
 (2a01:111:f400:7e02::203) by VI1PR0102CA0029.outlook.office365.com
 (2603:10a6:802::42) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2623.9 via Frontend
 Transport; Thu, 9 Jan 2020 16:34:13 +0000

Received: from relay-out.ohc.cu (200.55.138.44) by
 DB5EUR01FT034.mail.protection.outlook.com (10.152.4.246) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.20.2623.9 via Frontend Transport; Thu, 9 Jan 2020 16:34:12 +0000

Received: from relay-in.ohc.cu (relay-in.ohc.cu [127.0.0.1])
    by relay-out.ohc.cu (Postfix) with ESMTP id 69EA722DD
    for <some.email@some.domain>; Thu,  9 Jan 2020 11:29:43 -0500 (CST)

Received: from relay-out.ohc.cu ([127.0.0.1])
    by relay-in.ohc.cu (relay-in.ohc.cu [127.0.0.1]) (amavisd-new, port 10024)
    with ESMTP id 7CZku5Y59vGC for <some.email@some.domain>;
    Thu,  9 Jan 2020 11:29:38 -0500 (CST)

Received: from correo.patrimonio.ohc.cu (unknown [192.168.229.20])
    by relay-out.ohc.cu (Postfix) with ESMTP id B83BA22F5
    for <some.email@some.domain>; Thu,  9 Jan 2020 11:29:36 -0500 (CST)

Received: from localhost (localhost.localdomain [127.0.0.1])
    by correo.patrimonio.ohc.cu (Postfix) with ESMTP id 65413232A001
    for <some.email@some.domain>; Thu,  9 Jan 2020 11:40:05 -0500 (CST)

Received: from correo.patrimonio.ohc.cu ([127.0.0.1])
    by localhost (correo.patrimonio.ohc.cu [127.0.0.1]) (amavisd-new, port 10024)
    with ESMTP id hNMp-6lHHtzH for <some.email@some.domain>;
    Thu,  9 Jan 2020 11:40:05 -0500 (CST)

Received: from correoweb.patrimonio.ohc.cu (unknown [192.168.229.23])
    by correo.patrimonio.ohc.cu (Postfix) with ESMTPA id EC62A232A00A;
    Thu,  9 Jan 2020 11:39:53 -0500 (CST)

变化最大的领域似乎是

  1. 主机域

    例如

    • 来自 VE1PR01MB5599.eurprd01.prod.exchangelabs.com (2603:10a6:7:7c::43)
    • 来自 HE1PR0102MB2714.eurprd01.prod.exchangelabs.com
    • 来自 VI1PR0102CA0029.eurprd01.prod.exchangelabs.com (2603:10a6:802::42)
    • by VE1PR01MB5599.eurprd01.prod.exchangelabs.com (2603:10a6:803:11f::30)
    • 来自 DB5EUR01FT034.eop-EUR01.prod.protection.outlook.com (2a01:111:f400:7e02::203)
    • by VI1PR0102CA0029.outlook.office365.com (2603:10a6:802::42)
    • 来自relay-out.ohc.cu (200.55.138.44)
    • DB5EUR01FT034.mail.protection.outlook.com (10.152.4.246)
    • 来自relay-in.ohc.cu (relay-in.ohc.cu [127.0.0.1])
    • relay-out.ohc.cu（后缀）
    • 来自relay-out.ohc.cu ([127.0.0.1])
    • 由 relay-in.ohc.cu (relay-in.ohc.cu [127.0.0.1]) (amavisd-new, 端口 10024)
    • 来自 correo.patrimonio.ohc.cu（未知 [192.168.229.20]）
    • relay-out.ohc.cu（后缀）
    • 来自本地主机（localhost.localdomain [127.0.0.1]）
    • 作者：correo.patrimonio.ohc.cu（后缀）
    • 来自 correo.patrimonio.ohc.cu ([127.0.0.1])
    • 通过 localhost (correo.patrimonio.ohc.cu [127.0.0.1]) (amavisd-new, 端口 10024)
    • 来自 correoweb.patrimonio.ohc.cu（未知 [192.168.229.23]）
    • 作者：correo.patrimonio.ohc.cu（后缀）

  2. 邮件协议 例如

    • 使用 Microsoft SMTP 服务器（版本=TLS1_2，密码=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384）
    • 使用 ESMTP

考虑到它们不断变化的性质，提取此类信息的综合方法是什么？ SO 上的其他answers 不鼓励在此任务中使用正则表达式，但是如何进行这种解析呢？如果存在一些经过测试的正则表达式或 Java 代码/库来解析接收到的标头以提取上述信息，那对我来说没问题。

【问题讨论】：

  • 为什么这个问题被否决了？否决任何 cmets......
  • SO 应该更好地处理反对票，例如询问原因并在反对票者附近公开展示。
  • 有时候真的很痛苦；看到对这个非常漂亮和正确捏造的问题的否决票。我赞成它虽然;但真的很遗憾在这里看到反对票。

标签： java regex email-parsing rfc2822


【解决方案1】：

我想提供以下解决方案。你可以找到here所用正则表达式的完整解释。

import java.util.regex.Matcher;
import java.util.regex.Pattern;
import java.util.LinkedList;
import java.util.HashMap;
import java.lang.StringBuilder;

class Rextester {
    public static void main(String[] args) {
        Pattern p = Pattern.compile("(?:(Received:)|\\G(?!\\A))" +
                                    "\\s*(from|by|with|id|via|for|;)" +
                                    "\\s*(\\S+?(?:\\s+\\S+?)*?)\\s*" +
                                    "(?=Received:|by|with|id|via|for|;|\\z)");
        String text = "Received: from VE1PR01MB5599.eurprd01.prod.exchangelabs.com\n" +
                      " (2603:10a6:7:7c::43) by HE1PR0102MB2714.eurprd01.prod.exchangelabs.com with\n" +
                      " HTTPS via HE1PR0402CA0054.EURPRD04.PROD.OUTLOOK.COM; Thu, 9 Jan 2020 16:34:13\n" +
                      " +0000\n" +
                      "\n" +
                      "Received: from VI1PR0102CA0029.eurprd01.prod.exchangelabs.com\n" +
                      " (2603:10a6:802::42) by VE1PR01MB5599.eurprd01.prod.exchangelabs.com\n" +
                      " (2603:10a6:803:11f::30) with Microsoft SMTP Server (version=TLS1_2,\n" +
                      " cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2602.12; Thu, 9 Jan\n" +
                      " 2020 16:34:13 +0000\n" +
                      "\n" +
                      "Received: from DB5EUR01FT034.eop-EUR01.prod.protection.outlook.com\n" +
                      " (2a01:111:f400:7e02::203) by VI1PR0102CA0029.outlook.office365.com\n" +
                      " (2603:10a6:802::42) with Microsoft SMTP Server (version=TLS1_2,\n" +
                      " cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2623.9 via Frontend\n" +
                      " Transport; Thu, 9 Jan 2020 16:34:13 +0000\n" +
                      "\n" +
                      "Received: from relay-out.ohc.cu (200.55.138.44) by\n" +
                      " DB5EUR01FT034.mail.protection.outlook.com (10.152.4.246) with Microsoft SMTP\n" +
                      " Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id\n" +
                      " 15.20.2623.9 via Frontend Transport; Thu, 9 Jan 2020 16:34:12 +0000\n" +
                      "\n" +
                      "Received: from relay-in.ohc.cu (relay-in.ohc.cu [127.0.0.1])\n" +
                      "    by relay-out.ohc.cu (Postfix) with ESMTP id 69EA722DD\n" +
                      "    for <some.email@some.domain>; Thu,  9 Jan 2020 11:29:43 -0500 (CST)\n" +
                      "\n" +
                      "Received: from relay-out.ohc.cu ([127.0.0.1])\n" +
                      "    by relay-in.ohc.cu (relay-in.ohc.cu [127.0.0.1]) (amavisd-new, port 10024)\n" +
                      "    with ESMTP id 7CZku5Y59vGC for <some.email@some.domain>;\n" +
                      "    Thu,  9 Jan 2020 11:29:38 -0500 (CST)\n" +
                      "\n" +
                      "Received: from correo.patrimonio.ohc.cu (unknown [192.168.229.20])\n" +
                      "    by relay-out.ohc.cu (Postfix) with ESMTP id B83BA22F5\n" +
                      "    for <some.email@some.domain>; Thu,  9 Jan 2020 11:29:36 -0500 (CST)\n" +
                      "\n" +
                      "Received: from localhost (localhost.localdomain [127.0.0.1])\n" +
                      "    by correo.patrimonio.ohc.cu (Postfix) with ESMTP id 65413232A001\n" +
                      "    for <some.email@some.domain>; Thu,  9 Jan 2020 11:40:05 -0500 (CST)\n" +
                      "\n" +
                      "Received: from correo.patrimonio.ohc.cu ([127.0.0.1])\n" +
                      "    by localhost (correo.patrimonio.ohc.cu [127.0.0.1]) (amavisd-new, port 10024)\n" +
                      "    with ESMTP id hNMp-6lHHtzH for <some.email@some.domain>;\n" +
                      "    Thu,  9 Jan 2020 11:40:05 -0500 (CST)\n" +
                      "\n" +
                      "Received: from correoweb.patrimonio.ohc.cu (unknown [192.168.229.23])\n" +
                      "    by correo.patrimonio.ohc.cu (Postfix) with ESMTPA id EC62A232A00A;\n" +
                      "    Thu,  9 Jan 2020 11:39:53 -0500 (CST)";
        LinkedList<HashMap<String, String>> data = new LinkedList<HashMap<String, String>>();
        HashMap<String, String> e;
        StringBuilder sb = new StringBuilder(4096);
        Matcher m = p.matcher(text);
        while (m.find()) {
            if (m.group(1) != null) {
                data.add(new HashMap<String, String>());
            }
            e = data.getLast();
            e.put(m.group(2), m.group(3));
        }
        sb.append("[");
        data.stream().forEach((x) -> sb.append(x).append(",\n"));
        if (sb.length() > 2) {
            sb.setLength(sb.length() - 2);
        }
        sb.append("]");
        System.out.println(sb);
    }
}

输出：

[{with=HTTPS, by=HE1PR0102MB2714.eurprd01.prod.exchangelabs.com, from=VE1PR01MB5599.eurprd01.prod.exchangelabs.com
 (2603:10a6:7:7c::43), ;=Thu, 9 Jan 2020 16:34:13
 +0000, via=HE1PR0402CA0054.EURPRD04.PROD.OUTLOOK.COM},
{with=Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384), by=VE1PR01MB5599.eurprd01.prod.exchangelabs.com
 (2603:10a6:803:11f::30), from=VI1PR0102CA0029.eurprd01.prod.exchangelabs.com
 (2603:10a6:802::42), id=15.20.2602.12, ;=Thu, 9 Jan
 2020 16:34:13 +0000},
{with=Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384), by=VI1PR0102CA0029.outlook.office365.com
 (2603:10a6:802::42), from=DB5EUR01FT034.eop-EUR01.prod.protection.outlook.com
 (2a01:111:f400:7e02::203), id=15.20.2623.9, ;=Thu, 9 Jan 2020 16:34:13 +0000, via=Frontend
 Transport},
{with=Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384), by=DB5EUR01FT034.mail.protection.outlook.com (10.152.4.246), from=relay-out.ohc.cu (200.55.138.44), id=15.20.2623.9, ;=Thu, 9 Jan 2020 16:34:12 +0000, via=Frontend Transport},
{with=ESMTP, by=relay-out.ohc.cu (Postfix), for=<some.email@some.domain>, from=relay-in.ohc.cu (relay-in.ohc.cu [127.0.0.1]), id=69EA722DD, ;=Thu,  9 Jan 2020 11:29:43 -0500 (CST)},
{with=ESMTP, by=relay-in.ohc.cu (relay-in.ohc.cu [127.0.0.1]) (amavisd-new, port 10024), for=<some.email@some.domain>, from=relay-out.ohc.cu ([127.0.0.1]), id=7CZku5Y59vGC, ;=Thu,  9 Jan 2020 11:29:38 -0500 (CST)},
{with=ESMTP, by=relay-out.ohc.cu (Postfix), for=<some.email@some.domain>, from=correo.patrimonio.ohc.cu (unknown [192.168.229.20]), id=B83BA22F5, ;=Thu,  9 Jan 2020 11:29:36 -0500 (CST)},
{with=ESMTP, by=correo.patrimonio.ohc.cu (Postfix), for=<some.email@some.domain>, from=localhost (localhost.localdomain [127.0.0.1]), id=65413232A001, ;=Thu,  9 Jan 2020 11:40:05 -0500 (CST)},
{with=ESMTP, by=localhost (correo.patrimonio.ohc.cu [127.0.0.1]) (amavisd-new, port 10024), for=<some.email@some.domain>, from=correo.patrimonio.ohc.cu ([127.0.0.1]), id=hNMp-6lHHtzH, ;=Thu,  9 Jan 2020 11:40:05 -0500 (CST)},
{with=ESMTPA, by=correo.patrimonio.ohc.cu (Postfix), from=correoweb.patrimonio.ohc.cu (unknown [192.168.229.23]), id=EC62A232A00A, ;=Thu,  9 Jan 2020 11:39:53 -0500 (CST)}]

Demo.

【讨论】：

  • 感谢分享，看起来很不错。是否可以改进以拥有{by=url, byIP=IP, from=url, fromIP=IP, ...}？
猜你喜欢
  • 1970-01-01
  • 1970-01-01
  • 2020-09-18
  • 2018-02-23
  • 2022-12-17
  • 2015-07-26
  • 2016-12-18
  • 2016-06-09
  • 1970-01-01
相关资源
最近更新 更多
热门标签
Java Python linux javascript Mysql C# Docker 算法 前端 SpringBoot Redis Vue spring 设计模式 .net core .net kubernetes c++ 数据库 数据结构 大数据 js 机器学习 微服务 Android Go 程序员 面试 JVM ASP.net core 云原生 人工智能 后端 PHP git CSS golang k8s Nginx Django mybatis 深度学习 多线程 React 架构 devops 爬虫 云计算 Spring Boot LeetCode