【问题标题】:Improve performance of Graphql (graphene-django) for 4 levels of nested lists提高 Graphql (graphene-django) 4 级嵌套列表的性能
【发布时间】:2022-10-24 06:44:31
【问题描述】:

我将 Django 4 与 graphene-Django 3 一起使用,我需要构建一个响应,该响应必须是 4 级列表和最底层的字典。这里的实现

class FailureSensorType(DjangoObjectType):
      class Meta:
         model = FailureSensor
      spectra = graphene.Field(SpectraGraphType)
      ...
      rest of fields


class SpectraGraphType(graphene.ObjectType):
   name = graphene.List(graphene.String)
   spectra_z = graphene.List(graphene.List(graphene.List(SpectraZGraphType)))


class SpectraZGraphType(graphene.ObjectType):
   _id = graphene.String(required=False)
   collection_name = graphene.String(required=False)
   values = graphene.List(graphene.Float, required=False)
   values_names = graphene.List(graphene.Int, required=False)
   sidebands = graphene.List(graphene.Float, required=False)
   rpm_detected = graphene.Int(required=False)
   anomaly = graphene.Int()


def resolve_spectra(self, info):
        if self.anomaly_type == "spectra":
            spectra_name = set()
            for graph_y in self.get_map_Y():
                spectra_name.add(str(self.id) + '-' + graph_y.split()[-1])

            spectra_z_list = list()
            spectra_z_list_new = list()
            for i, x in enumerate(np.split(self.get_map_Z(), id_z)):
                spectra_z_list.append(x.tolist())


            for spectra_z in spectra_z_list:
                zero_index_list = list()
                for index_obj, graph_z_obj in enumerate(spectra_z):
                    zero_index = list()
                    for i, graph_z_value in enumerate(graph_z_obj):
                        if graph_z_value != '{"anomaly": 0}':
                            zero_index.append(i)

                    zero_index_list.append(zero_index)

                new_z_list = list() 
                for z_obj in spectra_z:
                    new_z = [v for i, v in enumerate(z_obj) if i in zero_index_set]
                    z_dict_list = list()
                    for dict_string in new_z:
                        r = json.loads(dict_string.replace("|", ",").replace("(", "[").replace(")", "]"))
                        if "_id" not in r:
                            r["_id"] = ""
                        if "collection_name" not in r:
                            r["collection_name"] = ""    
                        if "rpm_detected" not in r:
                            r["rpm_detected"] = -1    
                        if "values" in r:
                            r["values"] = json.loads(r["values"])
                        else:
                            r["values"] = []
                        if "values_names" in r:
                            r["values_names"] = json.loads(r["values_names"])
                        else:
                            r["values_names"] = []
                        if "sidebands" in r:
                            r["sidebands"] = json.loads(r["sidebands"])
                        else:
                            r["sidebands"] = []
                        z_dict_list.append(r)
                    new_z_list.append(z_dict_list)

                spectra_z_list_new.append(new_z_list)
                
            return {
                "name": spectra_name,
                "spectra_z": spectra_z_list_new
            }

这是 graphql 查询:

inspectSensorFailureBySystem(){
 failureSensors{
  anomalyType
  failureSensors{
    spectra{
      name
      spectraZ {
        Id
        collectionName
        rpmDetected
        anomaly
        values
        valuesNames
        sidebands
        
      }
    }
  }
}

这个查询的结果是:

{
 "data": {
   "inspectSensorFailureBySystem": [
     {
       "failureSensors": [
         {
        "anomalyType": "spectra",
        "failureSensors": [
          {
            "spectra": {
              "name": [
                "15339-envelope_spectra",
                "15339-envelope_spectra_timedomain",
                "15339-spectra_timedomain",
                "15339-spectra"
              ],
              "spectraZ": [
                [
                  [
                    {
                      "Id": "628bd17db4aff3060810a726",
                      "collectionName": "spectrum",
                      "rpmDetected": -1,
                      "anomaly": -1,
                      "values": [],
                      "valuesNames": [],
                      "sidebands": []
                    },
                    ...
                    {
                      "Id": "62e8d3119aa606584e88b228",
                      "collectionName": "timedomain",
                      "rpmDetected": 1256,
                      "anomaly": 1,
                      "values": [
                        261.1224,
                        522.2448,
                        783.3672,
                        1044.4896,
                        1305.612,
                        1566.7344,
                        1827.8568,
                        2088.9792,
                        2872.3464,
                        3133.4688
                      ],
                      "valuesNames": [
                        1,
                        2,
                        3,
                        4,
                        5,
                        6,
                        7,
                        8,
                        11,
                        12
                      ],
                      "sidebands": []
                        }
                      ]
                    ]
                  ]
                }
              }
            ]
          }
        ]
      }
    ]
  }
}

数据直接来自一个字符串化的字典列表 (spectra_z),该列表在之前的解析器中进行了解析,并且运行速度非常快(不到 0.2 秒)。因此,从数据库中检索数据,然后从字符串解析到所需的嵌套列表和最终字典非常快。

但是在解析器完成并将结果传递给 GraphQL 引擎后,GraphQL 需要 30 多秒来解析给定的结果并发送最终响应。

另一方面,如果我只是从数据库 (spectra_z) 中获取字符串化的值并将它们作为字符串传递(不涉及嵌套列表),显然 GraphGL 响应非常快。

我不是 GraphGL 专家,我想知道如何加快 graphene-Django 以更快地解析此响应(因为 30 秒得到响应是不可接受的)。 是否存在任何类型的设置参数来加速它或任何技巧?

【问题讨论】:

    标签: django graphql nested-lists graphene-django


    【解决方案1】:

    我建议如下:

    1. 为仅返回名称数组的光谱定义解析器
    2. 为spectraZ 定义一个解析器,它只返回spectraZ 列表
    3. 为spectralZ 对象中的所有字段定义单独的解析器。这将避免一长串if 语句,在这些语句中,您基本上每次通过键解析一个字段
    4. 让 GraphQL 将所有结果一起编组到返回对象中

      如果不是所有嵌套字段都被客户端请求,这将具有额外的好处,即不会过度计算所有嵌套字段。即使您只想要名称,其他所有内容也会被计算出来。

    【讨论】:

      猜你喜欢
      • 1970-01-01
      • 2013-01-29
      • 1970-01-01
      • 1970-01-01
      • 2020-05-31
      • 1970-01-01
      • 1970-01-01
      • 1970-01-01
      相关资源
      最近更新 更多