UNIX grepping 安装日志中最新版本的文件

【问题标题】：UNIX grepping the latest version of files in a install logUNIX grepping 安装日志中最新版本的文件
【发布时间】：2014-04-07 03:09:05
【问题描述】：

我想从日志文件中获取最新版本的文件，并使用 unix 命令输出到一个新文件。

这是日志文件的示例。

补丁.txt

Wed Jan 15 10:42:26 MYT 2014, Start, SPIRTE_4.3.2.B1, OK
Wed Jan 15 10:42:28 MYT 2014, End, TILES_7.3.66B1, DONE
Wed Jan 15 10:42:35 MYT 2014, End, MOVIES_OPENING_4.3.2.B1, FAILED
Wed Jan 15 10:42:36 MYT 2014, Start, MOVEMENT_7.3.68B1, OK
Wed Jan 15 10:42:36 MYT 2014, Start, MOBS_7.3.2B1, OK
Wed Jan 15 10:42:37 MYT 2014, End, TILES_7.3.68B1, DONE
Wed Jan 15 10:42:38 MYT 2014, Start, MOVIES_CUTSCENE_4.3.2.57B1, OK
Wed Jan 15 10:42:38 MYT 2014, Start, GAME_4.3.2.57B1_SERVER, OK
Wed Jan 15 10:42:43 MYT 2014, End, GAME_4.2.57B2_CLIENT, DONE
Wed Jan 15 10:42:43 MYT 2014, End, GAME_4.2.57B4_SERVER, DONE
Wed Jan 15 10:42:45 MYT 2014, End, GAME_4.2.57B1_CLIENT, DONE
Wed Jan 15 10:42:45 MYT 2014, End, GAME_4.2.57B1_SERVER, DONE
Wed Jan 15 10:42:45 MYT 2014, End, DB_4.2.57C2_CLIENT, DONE
Wed Jan 15 10:42:45 MYT 2014, End, DB_4.2.57C2_SERVER, DONE

1.文件的最新日期指定最新版本，例如

Wed Jan 15 10:42:26 MYT 2014, Start, SPIRTE_4.3.2.B1, DONE
Wed Jan 16 10:42:26 MYT 2014, Start, SPIRTE_4.3.2.B1, DONE //latest
Wed Jan 17 10:42:26 MYT 2014, Start, SPIRTE_4.3.2.B1, FAILED //does not consider

2.DONE 在行尾指定补丁已安装。

3.每个文件都有这种格式[characters_][versionNumber_][CLIENT/SERVER]，[CLIENT/SERVER]不是强制性的。

4.如果安装了多个版本的文件，最新版本将覆盖之前的版本。例如

Wed Jan 15 10:42:37 MYT 2014, End, TILES_7.3.68B1, DONE //overwritten
Wed Jan 16 10:45:37 MYT 2014, End, TILES_7.3.70C1, DONE //latest

5. 后缀为 _CLIENT 的文件考虑另一个后缀为 _SERVER 的文件，即使它们以相同的名称开头（但仍受规则 #4 的约束）。 以此为例：

Wed Jan 15 10:39:23 MYT 2014, EndPatch, GAME_TILES_7_3_2_3B1_Client, DONE
Wed Jan 15 10:42:45 MYT 2014, EndPatch, MOB_CHARACTER_4.3.2.57B1, DONE
Wed Jan 15 10:41:39 MYT 2014, EndPatch, MOB_CHARACTER_4.3.2.42B4, DONE
Wed Jan 15 10:41:38 MYT 2014, EndPatch, GAME_TILES_7.3.2.33B3, DONE
Wed Jan 15 10:39:34 MYT 2014, EndPatch, GAME_TILES_7.3.2.1B1, DONE
Wed Jan 15 10:39:07 MYT 2014, EndPatch, GAME_TILES_7.3.2.ServicePackB1, DONE

GAME_TILES_7_3_2_3B1_Client 是一个文件。

GAME_TILES_7.3.2.33B3 是另一个文件。

GAME_TILES_7.3.2.1B1 与 GAME_TILES_7.3.2.33B3 和 GAME_TILES_7.3.2.ServicePackB1 相同

例如输出文件

Wed Jan 15 10:42:37 MYT 2014, End, TILES_7.3.68B1, DONE
Wed Jan 15 10:42:45 MYT 2014, End, GAME_4.2.57B1_CLIENT, DONE
Wed Jan 15 10:42:45 MYT 2014, End, GAME_4.2.57B1_SERVER, DONE
Wed Jan 15 10:42:45 MYT 2014, End, DB_4.2.57C2_CLIENT, DONE
Wed Jan 15 10:42:45 MYT 2014, End, DB_4.2.57C2_SERVER, DONE

策略：

1. sort all the files
2. grep all the lines that has DONE
3. check for rule #4 and rule #5 (how? I can't use tail for this)

我现在在哪里：grep '.* DONE'

测试文件：

Wed Jan 15 10:36:16 MYT 2014, StartInstall, MOB_CHARACTER_4.3.2B188, OK
Wed Jan 15 10:36:21 MYT 2014, StartInstall, GAME_TILES_7.3.2B467, OK
Wed Jan 15 10:36:21 MYT 2014, StartInstall, GAME_TILES_7.3.2B467, OK
Wed Jan 15 10:37:33 MYT 2014, EndInstall, GAME_TILES_7.3.2B467, DONE
Wed Jan 15 10:37:35 MYT 2014, StartInstall, MOVIE_CUTSCENE_4.3.2B246, OK
Wed Jan 15 10:37:35 MYT 2014, StartInstall, MOVIE_CUTSCENE_4.3.2B246, OK
Wed Jan 15 10:38:21 MYT 2014, EndInstall, MOVIE_CUTSCENE_4.3.2B246, DONE
Wed Jan 15 10:38:59 MYT 2014, EndInstall, MOB_CHARACTER_4.3.2B188, DONE
Wed Jan 15 10:39:00 MYT 2014, StartPatch, GAME_TILES_7.3.2.ServicePackB1, OK
Wed Jan 15 10:39:00 MYT 2014, StartPatch, GAME_TILES_7.3.2.ServicePackB1, OK
Wed Jan 15 10:39:07 MYT 2014, EndPatch, GAME_TILES_7.3.2.ServicePackB1, DONE
Wed Jan 15 10:39:08 MYT 2014, StartPatch, GAME_TILES_7_3_2_3B1_Server, OK
Wed Jan 15 10:39:08 MYT 2014, StartPatch, GAME_TILES_7_3_2_3B1_Server, OK
Wed Jan 15 10:39:16 MYT 2014, EndPatch, GAME_TILES_7_3_2_3B1_Server, DONE
Wed Jan 15 10:39:16 MYT 2014, StartPatch, GAME_TILES_7_3_2_3B1_Client, OK
Wed Jan 15 10:39:16 MYT 2014, StartPatch, GAME_TILES_7_3_2_3B1_Client, OK
Wed Jan 15 10:39:23 MYT 2014, EndPatch, GAME_TILES_7_3_2_3B1_Client, DONE
Wed Jan 15 10:39:24 MYT 2014, StartPatch, GAME_TILES_7.3.2.1B1, OK
Wed Jan 15 10:39:24 MYT 2014, StartPatch, GAME_TILES_7.3.2.1B1, OK
Wed Jan 15 10:39:34 MYT 2014, EndPatch, GAME_TILES_7.3.2.1B1, DONE
Wed Jan 15 10:39:35 MYT 2014, StartPatch, GAME_TILES_7.3.2.1B1_simulator, OK
Wed Jan 15 10:39:35 MYT 2014, StartPatch, GAME_TILES_7.3.2.1B1_simulator, OK
Wed Jan 15 10:39:45 MYT 2014, EndPatch, GAME_TILES_7.3.2.1B1_simulator, DONE
Wed Jan 15 10:39:46 MYT 2014, StartPatch, MOVIE_CUTSCENE_4.3.2.1B1, OK
Wed Jan 15 10:39:46 MYT 2014, StartPatch, MOVIE_CUTSCENE_4.3.2.1B1, OK
Wed Jan 15 10:39:53 MYT 2014, EndPatch, MOVIE_CUTSCENE_4.3.2.1B1, DONE
Wed Jan 15 10:39:54 MYT 2014, StartPatch, MOB_CHARACTER_4.3.2.1B1, OK
Wed Jan 15 10:39:54 MYT 2014, StartPatch, MOB_CHARACTER_4.3.2.1B1, OK
Wed Jan 15 10:40:03 MYT 2014, EndPatch, MOB_CHARACTER_4.3.2.1B1, DONE
Wed Jan 15 10:40:04 MYT 2014, StartPatch, GAME_TILES_7.3.2.2B1_Server, OK
Wed Jan 15 10:40:04 MYT 2014, StartPatch, GAME_TILES_7.3.2.2B1_Server, OK
Wed Jan 15 10:40:12 MYT 2014, EndPatch, GAME_TILES_7.3.2.2B1_Server, DONE
Wed Jan 15 10:40:13 MYT 2014, StartPatch, GAME_TILES_7.3.2.2B1_Client, OK
Wed Jan 15 10:40:13 MYT 2014, StartPatch, GAME_TILES_7.3.2.2B1_Client, OK
Wed Jan 15 10:40:21 MYT 2014, EndPatch, GAME_TILES_7.3.2.2B1_Client, DONE
Wed Jan 15 10:40:22 MYT 2014, StartPatch, GAME_TILES_7.3.2.4B2, OK
Wed Jan 15 10:40:22 MYT 2014, StartPatch, GAME_TILES_7.3.2.4B2, OK
Wed Jan 15 10:40:32 MYT 2014, EndPatch, GAME_TILES_7.3.2.4B2, DONE
Wed Jan 15 10:40:33 MYT 2014, StartPatch, MOB_CHARACTER_4.3.2.2B1, OK
Wed Jan 15 10:40:33 MYT 2014, StartPatch, MOB_CHARACTER_4.3.2.2B1, OK
Wed Jan 15 10:40:42 MYT 2014, EndPatch, MOB_CHARACTER_4.3.2.2B1, DONE
Wed Jan 15 10:40:43 MYT 2014, StartPatch, MOVIE_CUTSCENE_4.3.2.2B3, OK
Wed Jan 15 10:40:43 MYT 2014, StartPatch, MOVIE_CUTSCENE_4.3.2.2B3, OK
Wed Jan 15 10:40:50 MYT 2014, EndPatch, MOVIE_CUTSCENE_4.3.2.2B3, DONE
Wed Jan 15 10:40:51 MYT 2014, StartPatch, MOB_CHARACTER_4.3.2.12B2, OK
Wed Jan 15 10:40:51 MYT 2014, StartPatch, MOB_CHARACTER_4.3.2.12B2, OK
Wed Jan 15 10:40:51 MYT 2014, EndPatch, MOB_CHARACTER_4.3.2.12B2, DONE
Wed Jan 15 10:40:52 MYT 2014, StartPatch, GAME_TILES_7.3.2.23B2, OK
Wed Jan 15 10:40:52 MYT 2014, StartPatch, GAME_TILES_7.3.2.23B2, OK
Wed Jan 15 10:40:52 MYT 2014, EndPatch, GAME_TILES_7.3.2.23B2, DONE
Wed Jan 15 10:40:53 MYT 2014, StartPatch, MOB_CHARACTER_4.3.2.13B1, OK
Wed Jan 15 10:40:53 MYT 2014, StartPatch, MOB_CHARACTER_4.3.2.13B1, OK
Wed Jan 15 10:41:07 MYT 2014, EndPatch, MOB_CHARACTER_4.3.2.13B1, DONE
Wed Jan 15 10:41:07 MYT 2014, StartPatch, MOB_CHARACTER_4.3.2.17B3, OK
Wed Jan 15 10:41:07 MYT 2014, StartPatch, MOB_CHARACTER_4.3.2.17B3, OK
Wed Jan 15 10:41:08 MYT 2014, EndPatch, MOB_CHARACTER_4.3.2.17B3, DONE
Wed Jan 15 10:41:09 MYT 2014, StartPatch, GAME_TILES_7.3.2.30B4, OK
Wed Jan 15 10:41:09 MYT 2014, StartPatch, GAME_TILES_7.3.2.30B4, OK
Wed Jan 15 10:41:09 MYT 2014, EndPatch, GAME_TILES_7.3.2.30B4, DONE
Wed Jan 15 10:41:10 MYT 2014, StartPatch, GAME_TILES_7.3.2.41B1, OK
Wed Jan 15 10:41:10 MYT 2014, StartPatch, GAME_TILES_7.3.2.41B1, OK
Wed Jan 15 10:41:10 MYT 2014, EndPatch, GAME_TILES_7.3.2.41B1, DONE
Wed Jan 15 10:41:11 MYT 2014, StartPatch, MOB_CHARACTER_4.3.2.13B2, OK
Wed Jan 15 10:41:11 MYT 2014, StartPatch, MOB_CHARACTER_4.3.2.13B2, OK
Wed Jan 15 10:41:26 MYT 2014, EndPatch, MOB_CHARACTER_4.3.2.13B2, DONE
Wed Jan 15 10:41:27 MYT 2014, StartPatch, MOB_CHARACTER_4.3.2.16B1, OK
Wed Jan 15 10:41:27 MYT 2014, StartPatch, MOB_CHARACTER_4.3.2.16B1, OK
Wed Jan 15 10:41:29 MYT 2014, EndPatch, MOB_CHARACTER_4.3.2.16B1, DONE
Wed Jan 15 10:41:30 MYT 2014, StartPatch, MOB_CHARACTER_4.3.2.22B2, OK
Wed Jan 15 10:41:30 MYT 2014, StartPatch, MOB_CHARACTER_4.3.2.22B2, OK
Wed Jan 15 10:41:30 MYT 2014, EndPatch, MOB_CHARACTER_4.3.2.22B2, DONE
Wed Jan 15 10:41:31 MYT 2014, StartPatch, MOB_CHARACTER_4.3.2.24B2, OK
Wed Jan 15 10:41:31 MYT 2014, StartPatch, MOB_CHARACTER_4.3.2.24B2, OK
Wed Jan 15 10:41:31 MYT 2014, EndPatch, MOB_CHARACTER_4.3.2.24B2, DONE
Wed Jan 15 10:41:32 MYT 2014, StartPatch, MOB_CHARACTER_4.3.2.25B2, OK
Wed Jan 15 10:41:32 MYT 2014, StartPatch, MOB_CHARACTER_4.3.2.25B2, OK
Wed Jan 15 10:41:32 MYT 2014, EndPatch, MOB_CHARACTER_4.3.2.25B2, DONE
Wed Jan 15 10:41:33 MYT 2014, StartPatch, MOB_CHARACTER_4.3.2.28B2, OK
Wed Jan 15 10:41:33 MYT 2014, StartPatch, MOB_CHARACTER_4.3.2.28B2, OK
Wed Jan 15 10:41:33 MYT 2014, EndPatch, MOB_CHARACTER_4.3.2.28B2, DONE
Wed Jan 15 10:41:34 MYT 2014, StartPatch, GAME_TILES_7.3.2.40B1, OK
Wed Jan 15 10:41:34 MYT 2014, StartPatch, GAME_TILES_7.3.2.40B1, OK
Wed Jan 15 10:41:34 MYT 2014, EndPatch, GAME_TILES_7.3.2.40B1, DONE
Wed Jan 15 10:41:35 MYT 2014, StartPatch, GAME_TILES_7.3.2.43B2, OK
Wed Jan 15 10:41:35 MYT 2014, StartPatch, GAME_TILES_7.3.2.43B2, OK
Wed Jan 15 10:41:36 MYT 2014, EndPatch, GAME_TILES_7.3.2.43B2, DONE
Wed Jan 15 10:41:36 MYT 2014, StartPatch, MOB_CHARACTER_4.3.2.32B5, OK
Wed Jan 15 10:41:36 MYT 2014, StartPatch, MOB_CHARACTER_4.3.2.32B5, OK
Wed Jan 15 10:41:37 MYT 2014, EndPatch, MOB_CHARACTER_4.3.2.32B5, DONE
Wed Jan 15 10:41:37 MYT 2014, StartPatch, GAME_TILES_7.3.2.33B3, OK
Wed Jan 15 10:41:37 MYT 2014, StartPatch, GAME_TILES_7.3.2.33B3, OK
Wed Jan 15 10:41:38 MYT 2014, EndPatch, GAME_TILES_7.3.2.33B3, DONE
Wed Jan 15 10:41:39 MYT 2014, StartPatch, MOB_CHARACTER_4.3.2.42B4, OK
Wed Jan 15 10:41:39 MYT 2014, StartPatch, MOB_CHARACTER_4.3.2.42B4, OK
Wed Jan 15 10:41:39 MYT 2014, EndPatch, MOB_CHARACTER_4.3.2.42B4, DONE
Wed Jan 15 10:41:40 MYT 2014, StartPatch, GAME_TILES_7.3.2.62B1_Server, OK
Wed Jan 15 10:41:40 MYT 2014, StartPatch, GAME_TILES_7.3.2.62B1_Server, OK
Wed Jan 15 10:41:51 MYT 2014, EndPatch, GAME_TILES_7.3.2.62B1_Server, DONE
Wed Jan 15 10:41:53 MYT 2014, StartPatch, GAME_TILES_7.3.2.62B1_Client, OK
Wed Jan 15 10:41:53 MYT 2014, StartPatch, GAME_TILES_7.3.2.62B1_Client, OK
Wed Jan 15 10:42:02 MYT 2014, EndPatch, GAME_TILES_7.3.2.62B1_Client, DONE
Wed Jan 15 10:42:03 MYT 2014, StartPatch, MOVIE_CUTSCENE_4.3.2.15B1, OK
Wed Jan 15 10:42:03 MYT 2014, StartPatch, MOVIE_CUTSCENE_4.3.2.15B1, OK
Wed Jan 15 10:42:12 MYT 2014, EndPatch, MOVIE_CUTSCENE_4.3.2.15B1, DONE
Wed Jan 15 10:42:12 MYT 2014, StartPatch, MOB_CHARACTER_4.3.2.32B7, OK
Wed Jan 15 10:42:12 MYT 2014, StartPatch, MOB_CHARACTER_4.3.2.32B7, OK
Wed Jan 15 10:42:23 MYT 2014, EndPatch, MOB_CHARACTER_4.3.2.32B7, DONE
Wed Jan 15 10:42:24 MYT 2014, StartPatch, GAME_TILES_7.3.2.66B4, OK
Wed Jan 15 10:42:24 MYT 2014, StartPatch, GAME_TILES_7.3.2.66B4, OK
Wed Jan 15 10:42:25 MYT 2014, EndPatch, GAME_TILES_7.3.2.66B4, DONE
Wed Jan 15 10:42:26 MYT 2014, StartPatch, MOB_CHARACTER_4.3.2.55B1, OK
Wed Jan 15 10:42:26 MYT 2014, StartPatch, MOB_CHARACTER_4.3.2.55B1, OK
Wed Jan 15 10:42:35 MYT 2014, EndPatch, MOB_CHARACTER_4.3.2.55B1, DONE
Wed Jan 15 10:42:36 MYT 2014, StartPatch, GAME_TILES_7.3.2.68B1, OK
Wed Jan 15 10:42:36 MYT 2014, StartPatch, GAME_TILES_7.3.2.68B1, OK
Wed Jan 15 10:42:37 MYT 2014, EndPatch, GAME_TILES_7.3.2.68B1, DONE
Wed Jan 15 10:42:38 MYT 2014, StartPatch, MOB_CHARACTER_4.3.2.57B1, OK
Wed Jan 15 10:42:38 MYT 2014, StartPatch, MOB_CHARACTER_4.3.2.57B1, OK
Wed Jan 15 10:42:45 MYT 2014, EndPatch, MOB_CHARACTER_4.3.2.57B1, DONE

【问题讨论】：

  • 我不确定你只能用 bash 来做到这一点，或者至少不能有效地做到这一点，你需要的是一个 hashmap/dictionary 数据结构，据我所知 bash 缺乏，你可以实现一个真正的使用字典执行此操作的简单 python 脚本

标签： regex unix grep command


【解决方案1】：

好的，这是一个的方法：

awk '$9 == "DONE" { key = $8; add = ""; sub ("_CLIENT,$",",",key); if (key != $8) { add = "_CLIENT," } else { sub ("_SERVER,$",",",key); if (key != $8) { add = "_SERVER," } } sub ("_[^_]*$", "", key); key = key""add; assoc[key] = $0; } END { for (var in assoc) { print assoc[var] } }' inputFile.txt

现在这很难看，所以我将把它分解并添加 cmets，以便您希望了解它是如何工作的：

$9 == "DONE" {                        // Only want "DONE" lines.
    key = $8;                         // Get the key.
    add = ""                          // Save client/server/none and remove
    sub ("_CLIENT,$", ",", key);
    if (key != $8) {
        add = "_CLIENT,"
    } else {
        sub ("_SERVER,$", ",", key);  
        if (key != $8) {
            add = "_SERVER,"
        }
    }
    sub ("_[^_]*$", "", key);         // Remove version.
    key = key""add;                   // Put client/server/none back.
    assoc[key] = $0;                  // Store line.
}
END {
    for (var in assoc) {              // Print out every keyed line.
        print assoc[var]
    }
}

说实话，一旦awk 脚本变得如此复杂，您最好将它们放入script.awk 文件并使用awk -f 运行它——在命令行上执行这些长脚本有点麻烦痛。

这里的技巧是使用键计算和关联数组。从字段 8 中删除版本意味着它成为“功能”的有效键，例如GAME_SERVER 或TILES_。

这样做的方法如下：

  • sub 命令用给定变量 (key) 中的第二个参数（只是逗号）替换第一个参数（_CLIENT,$ 表示行尾的 _CLIENT,）。
  • 我们首先尝试客户端 - 如果替换有效，key 现在与 $8 不同，我们保存了这一事实，不用担心检查服务器。
  • 如果没有客户端，我们尝试服务器。如果找到，我们存储 那个 事实并继续。如果既没有找到客户端也没有找到服务器，我们将保持原始状态，即没有任何内容可以添加回密钥。
  • 现在我们有一个字符串，最后只有版本（但我们记得它最初是有客户端、服务器还是什么都没有）。删除版本的 sub 只是删除了最后一个 _ 字符之后（包括）之后的所有内容。
  • 然后我们添加回客户端或服务器字符串，这就是我们的密钥 - 原来的 $8，但删除了版本。

为该键设置关联数组值将创建一个新条目或覆盖当前条目，这意味着文件中具有相同键的后面的行将覆盖之前的行。

然后，最后，您只需遍历关联数组，输出值（输入文件中的行）。在我的系统上，您会看到：

Wed Jan 15 10:42:37 MYT 2014, End, TILES_7.3.68B1, DONE
Wed Jan 15 10:42:45 MYT 2014, End, DB_4.2.57C2_CLIENT, DONE
Wed Jan 15 10:42:45 MYT 2014, End, DB_4.2.57C2_SERVER, DONE
Wed Jan 15 10:42:45 MYT 2014, End, GAME_4.2.57B1_CLIENT, DONE
Wed Jan 15 10:42:45 MYT 2014, End, GAME_4.2.57B1_SERVER, DONE

根据需要。

【讨论】：

  • 努力+1，但就我而言，代码没有解决规则#5，我将提供一个测试日志文件供您测试。另请注意，文件名可能是 MOVIES_OPENING_4.3.2.B1 和 MOVIES_CUTSCENE_4.3.2.57B1，所以使用第一位，即 MOVIES 是不可行的。
  • @McKevin，它确实解决了规则 #5，该规则指出客户端/服务器将被视为不同的文件。至于_ 出现在分隔符区域以外的 位置的可能性，我会修改答案。
  • @McKevin：好的，它现在可以处理存在无关 _ 字符的情况。 _server/_client 位 必须 在末尾（如果存在）并且版本中不允许有 _ 字符。
  • 谢谢，你能解释一下这条线吗？ sub ("_CLIENT,$", ",", key); if (key != $8) {
猜你喜欢
  • 1970-01-01
  • 2018-10-23
  • 2019-05-25
  • 2020-03-25
  • 1970-01-01
  • 1970-01-01
  • 1970-01-01
  • 1970-01-01
  • 1970-01-01
相关资源
最近更新 更多
热门标签
Java Python linux javascript Mysql C# Docker 算法 前端 SpringBoot Redis Vue spring 设计模式 .net core .net kubernetes c++ 数据库 数据结构 大数据 js 机器学习 微服务 Android Go 程序员 面试 JVM ASP.net core 云原生 人工智能 后端 PHP git CSS golang k8s Nginx Django mybatis 深度学习 多线程 React 架构 devops 爬虫 云计算 Spring Boot LeetCode