从对象聚合和求和值

Question

我想创建一个算法，可以通过过滤和求和值将我的对象键：值对聚合到一个新对象中 我希望过滤名称以便它们只聚合一次，例如可能有多个 ana-json-to-parquet 值。 count 键是给定proc_nm 中成功、失败等的次数

1. 过滤 proc 名称（postgres-loader、ana-json-to-parquet 等）
2. 过滤状态（成功、失败、致命、警告等）
3. 过程名称的每个状态的总和计数

所以一个示例结果是

ana-json-to-parquet: {
  success: 500
  fail:523
  fatal:0
  ... 
  ...
},
postgres-loader: {
  success: 2313
  fail: 2131
  fatal: 1
  ... 
  ...
},

数据对象示例：

{
proc_nm: ana-json-to-parquet
count:2
date:2021-11-04 00:00:00
status:warning
},
{
proc_nm: ana-json-to-parquet
count:50
date:2021-11-04 00:00:00
status:fatal
},

有多个 proc_names，我知道日期是同一小时的，它们按小时查询，因此在一小时内可能有多个相同 proc_names 的条目具有不同的状态和计数

尝试：我可以对每个 proc_nm 的计数求和，并将数据存储在一个新对象中，但在这种情况下，我想为每个 proc_nm 创建一个对象并聚合每个 proc_nm 和状态的计数

const data = datastream.rows
var results = {};
var totals = {};
for (var d = 0; d < data.length; d++) {
                    var o = data[d];
                    var proc = o["proc_nm"];
                    var values = results[proc] || {};
                    // Since the column names may change, we need to loop over the object
                    for (var key in o) {
                        // remove the undeeded keys
                        if (o.hasOwnProperty(key) && key.toLowerCase() != "proc_nm" && key.toLowerCase() != "date_trunc") {
                            // Now find the entry in the values obj that matches
                            var column = values[key] || 0;
                            // And the corresponding entry in the totals object
                            var totalsColumn = totals[key] || 0;
                            column += parseFloat(o[key]);
                            totalsColumn += parseFloat(o[key]);
                            values[key] = column;
                            totals[key] = totalsColumn;
                        }
                    }
                    results[proc] = values;
                }

Answer 1

我是 Ramda 的忠实粉丝。 （免责声明：我是它的作者之一。）使用 Ramda，我会将其作为一系列转换来完成：

const {pipe, groupBy, prop, map, pluck, sum} = R

const extract = pipe (
  groupBy (prop ('proc_nm')),
  map (groupBy (prop ('status'))),
  map (map (pluck ('count'))),
  map (map (sum))
) 

const data = [{proc_nm: "ana-json-to-parquet", count: 2, date: "2021-11-04 00: 00: 00", status: "warning"}, {proc_nm: "ana-json-to-parquet", count: 50, date: "2021-11-04 00: 00: 00", status: "success"}, {proc_nm: "ana-foobar", count: 26, date: "2021-11-04 01: 00: 00", status: "success"}, {proc_nm: "ana-json-to-parquet", count: 73, date: "2021-11-04 01: 00: 00", status: "success"}, {proc_nm: "ana-json-to-parquet", count: 50, date: "2021-11-04 02: 00: 00", status: "warning"}, {proc_nm: "ana-foobar", count: 2, date: "2021-11-04 02: 00: 00", status: "fatal"}, {proc_nm: "ana-foobar", count: 34, date: "2021-11-04 02: 00: 00", status: "success"}]

console .log (extract (data))

.as-console-wrapper {max-height: 100% !important; top: 0}

<script src="//cdnjs.cloudflare.com/ajax/libs/ramda/0.27.1/ramda.min.js"></script>

您可以通过简单地注释掉后续步骤来查看每个步骤的作用。你可以在Ramda REPL 玩这个。

但是那些 Ramda 辅助函数很容易由我们自己编写。我们可以编写和维护我们自己的版本：

const pipe = (...fns) => (arg) =>
  fns .reduce ((a, fn) => fn (a), arg)
const prop = (s) => (o) => 
  o [s]
const map = (fn) => (xs) =>
  Array .isArray (xs) 
    ? xs .map (x => fn(x))
    : Object .fromEntries (Object .entries (xs) .map (([k, v]) => [k, fn (v)]))
const pluck = pipe (prop, map)
const sum = (xs) => 
  xs .reduce ((a, x) => a + x, 0)
const groupBy = (fn) => (xs) =>
  xs .reduce (
    (a, x, _, __, k = fn (x)) => ((a [k] = a [k] || []), (a[k] .push (x)), a), 
    {}
  )

const extract = pipe (
  groupBy (prop ('proc_nm')),
  map (groupBy (prop ('status'))),
  map (map (pluck ('count'))),
  map (map (sum))
) 

const data = [{proc_nm: "ana-json-to-parquet", count: 2, date: "2021-11-04 00: 00: 00", status: "warning"}, {proc_nm: "ana-json-to-parquet", count: 50, date: "2021-11-04 00: 00: 00", status: "success"}, {proc_nm: "ana-foobar", count: 26, date: "2021-11-04 01: 00: 00", status: "success"}, {proc_nm: "ana-json-to-parquet", count: 73, date: "2021-11-04 01: 00: 00", status: "success"}, {proc_nm: "ana-json-to-parquet", count: 50, date: "2021-11-04 02: 00: 00", status: "warning"}, {proc_nm: "ana-foobar", count: 2, date: "2021-11-04 02: 00: 00", status: "fatal"}, {proc_nm: "ana-foobar", count: 34, date: "2021-11-04 02: 00: 00", status: "success"}]

console .log (extract (data))

.as-console-wrapper {max-height: 100% !important; top: 0}

重要的是我们可以通过使用简单的可重用帮助器来轻松地制作自定义extract 函数。

更新

请注意，自定义解决方案并不太难：

const extract = (xs) =>
  xs .reduce ((a, x) => {
    const g = a [x .proc_nm] || {}
    const s = (g [x .status] || 0) + x .count
    g [x .status] = s
    a [x .proc_nm] = g
    return a
  }, {})

const data = [{proc_nm: "ana-json-to-parquet", count: 2, date: "2021-11-04 00: 00: 00", status: "warning"}, {proc_nm: "ana-json-to-parquet", count: 50, date: "2021-11-04 00: 00: 00", status: "success"}, {proc_nm: "ana-foobar", count: 26, date: "2021-11-04 01: 00: 00", status: "success"}, {proc_nm: "ana-json-to-parquet", count: 73, date: "2021-11-04 01: 00: 00", status: "success"}, {proc_nm: "ana-json-to-parquet", count: 50, date: "2021-11-04 02: 00: 00", status: "warning"}, {proc_nm: "ana-foobar", count: 2, date: "2021-11-04 02: 00: 00", status: "fatal"}, {proc_nm: "ana-foobar", count: 34, date: "2021-11-04 02: 00: 00", status: "success"}]

console .log (extract (data))

我觉得这更难阅读而且有点乏味，但并不可怕。