如何避免使用 GSON Streaming 将整个 JSON 加载到内存中？

Question

下面是我的 JSON，其中我只有三个 reportRecords 仅用于演示目的，但通常有时我们会得到非常大的 json，然后它没有三个 reportRecords，它有大量 @987654325 @。

{
   "aggRecords":{
      "reportRecords":[
         {
            "min":1.0,
            "max":1.0,
            "avg":1.0,
            "count":18,
            "sumSq":18.0,
            "stddev":0.0,
            "median":1.0,
            "percentileMap":{
               "95":1
            },
            "metricName":"TotalCount",
            "dimensions":{
               "env":"prod",
               "pool":"hawk",
               "Name":"CORE_utrade11",
               "Type":"Error"
            },
            "value":18.0
         },
         {
            "min":1.0,
            "max":1.0,
            "avg":1.0,
            "count":25968842,
            "sumSq":2.5968842E7,
            "stddev":0.0,
            "median":1.0,
            "percentileMap":{
               "95":1
            },
            "metricName":"TotalCount",
            "dimensions":{
               "env":"prod",
               "pool":"hawk",
               "Name":"ResponseHeaders",
               "Type":"ConnectionPool"
            },
            "value":2.5968842E7
         },
         {
            "min":1.0,
            "max":1.0,
            "avg":1.0,
            "count":44,
            "sumSq":44.0,
            "stddev":0.0,
            "median":1.0,
            "percentileMap":{
               "95":1
            },
            "metricName":"TotalCount",
            "dimensions":{
               "env":"prod",
               "pool":"hawk",
               "Name":"read-lookup",
               "Type":"ClientPool"
            },
            "value":44.0
         }
      ]
   },
   "minRecordsMap":{

   }
}

现在我正在尝试在 JSON 之上序列化以提取那些Type 是 ClientPool 和 ConnectionPool 的 reportRecords，所以我不想将所有内容加载到内存中。我正在考虑为此使用GSON Streaming，但到目前为止我无法完成这项工作。

private static List<HostClientMetrics> loadMetrics(String url) {
    List<HostClientMetrics> metrics = new ArrayList<HostClientMetrics>();
    try {
        InputStream input = new URL(url).openStream();
        JsonReader reader = new JsonReader(new InputStreamReader(input, "UTF-8"));

        // not sure what I should do here?
    } catch (Exception ex) {
        // log error
    }

    return metrics;
}

下面是我的HostClientMetrics类

public class HostClientMetrics {

    private String metricName;
    private Map<String, Integer> percentileMap;
    private String median;
    private String stddev;
    private String sumSq;
    private String count;
    private String avg;
    private String max;
    private String min;

    public String getMetricName() {
        return metricName;
    }

    public Map<String, Integer> getPercentileMap() {
        return percentileMap;
    }

    public String getMedian() {
        return median;
    }

    public String getStddev() {
        return stddev;
    }

    public String getSumSq() {
        return sumSq;
    }

    public String getCount() {
        return count;
    }

    public String getAvg() {
        return avg;
    }

    public String getMax() {
        return max;
    }

    public String getMin() {
        return min;
    }

    public Dimensions getDimensions() {
        return dimensions;
    }

    public Dimensions dimensions;

    public static class Dimensions {
        private String env;
        private String pool;
        @SerializedName("Name")
        private String name;

        public String getEnv() {
            return env;
        }

        public String getPool() {
            return pool;
        }

        public String getName() {
            return name;
        }
    }
}

我只需要提取那些Type 是ClientPool 和ConnectionPool 的reportRecords。在我的示例中如何使用GSON Streaming？

Answer 1

private static List<HostClientMetrics> loadMetrics(String url) {

    GsonBuilder gsonBuilder = new GsonBuilder();
    Gson gson = gsonBuilder.create();

    List<HostClientMetrics> metrics = new ArrayList<HostClientMetrics>();

    try {
        InputStream input = new URL(url).openStream();
        JsonReader reader = new JsonReader(new InputStreamReader(input, "UTF-8"));

        reader.beginObject();

        String jsonTag = null;

        while(reader.hasNext()) {
            jsonTag = reader.nextName();
            if("aggRecords".equals(jsonTag)) {
                reader.beginObject();

                while(reader.hasNext()) {
                    jsonTag = reader.nextName();
                    if("reportRecords".equals(jsonTag)) {
                        reader.beginArray();
                        while (reader.hasNext()) {
                            HostClientMetrics hostClientMetrics = gson.fromJson(reader, HostClientMetrics.class);
                            if ("ClientPool".equals(hostClientMetrics.dimensions.type) || "ConnectionPool".equals(hostClientMetrics.dimensions.type)) {
                                metrics.add(hostClientMetrics);
                            }
                        }
                        reader.endArray();
                    }
                }
                reader.endObject();
            } else if("minRecordsMap".equals(jsonTag)) {
                reader.beginObject();
                // skip
                reader.endObject();
            }
        }

        reader.endObject();

        reader.close();
        return metrics;
    } catch (Exception ex) {
        // log error
        System.out.println("ex:" + ex);
    }

    return metrics;
}

将类型字段添加到您的Diemensions POJO：

public static class Dimensions {
    private String env;
    private String pool;
    @SerializedName("Name")
    private String name;

    @SerializedName("Type")
    private String type;

    // Getters / Setters
}