Firebase 大查询 - 如何从自定义事件表中检索数据

Question

我正在开发连接 Firebase 数据库的应用程序，我的应用程序基本上是一个市场应用程序。我已将所有用户搜索历史记录到 Firebase Analytics 作为自定义事件，我已将其链接到大查询以进行分析或建议。

这是我的自定义事件日志示例：

FirebaseAnalytics mFireBaseAnalytics = FirebaseAnalytics.getInstance(context);
        Bundle params = new Bundle();
        params.putString("SEARCHER_ID", "xxxx_id");
        params.putString("KEYWORDS", "Shoe");
        params.putDouble("LATITUDE", 12.00);
        params.putDouble("LONGITUDE", 80.00);
        mFireBaseAnalytics.logEvent("SEARCH_PRODUCT", params);

此数据将保存在 app_events 表中的 Biq 查询中。

从这个事件中，我必须查询我当地流行的关键字搜索列表，例如 5 公里范围内。我不确定如何实现这一目标。因为请看下面的结果，它实际上是如何存储在 Big 查询中的。

"event_dim": [      
      {
        "date": "20170718",
        "name": "SEARCH_PRODUCT",
        "params": [

          {
            "key": "LATITUDE",
            "value": {
              "string_value": null,
              "int_value": null,
              "float_value": null,
              "double_value": "12.9632747"
            }
          },

          {
            "key": "SEARCHER_ID",
            "value": {
              "string_value": "-KkphUBI3_v3kdGq6qTa",
              "int_value": null,
              "float_value": null,
              "double_value": null
            }
          },

          {
            "key": "LONGITUDE",
            "value": {
              "string_value": null,
              "int_value": null,
              "float_value": null,
              "double_value": "80.2131347"
            }
          },

          {
            "key": "KEYWORDS",
            "value": {
              "string_value": "shoe cloths",
              "int_value": null,
              "float_value": null,
              "double_value": null
            }
          },

        ],

      }
      ]

对于这种格式，我使用标准 SQL 查询进行了如下查询

SELECT
SQRT(POW(69.1*( (
  SELECT
    params.value.double_value
  FROM
    x.params
  WHERE
    params.key = 'LATITUDE') - 12.00), 2) + POW(69.1*(80.098 - (SELECT
    params.value.double_value
  FROM
    x.params
  WHERE
    params.key = 'LONGITUDE') ) * COS((SELECT
    params.value.double_value
  FROM
    x.params
  WHERE
    params.key = 'LATITUDE') / 57.3), 2)) AS distance,
  (
  SELECT
    params.value.string_value
  FROM
    x.params
  WHERE
    params.key = 'KEYWORDS') AS KEYWORDS,
  (
  SELECT
    params.value.string_value
  FROM
    x.params
  WHERE
    params.key = 'SEARCHER_ID') AS SEARCHER_ID,
  (
  SELECT
    params.value.string_value
  FROM
    x.params
  WHERE
    params.key = 'LATITUDE') AS LATITUDE,
  (
  SELECT
    params.value.string_value
  FROM
    x.params
  WHERE
    params.key = 'LONGITUDE') AS LONGITUDE
FROM
  `app_name.app_events_*`,
  UNNEST(event_dim) AS x
WHERE
  x.name = 'SEARCH_PRODUCT'
  AND (
  SELECT
    params.value.string_value As SearchId
  FROM
    x.params
  WHERE
    params.key = 'SEARCHER_ID' ) = '-xx_id'

通过此查询，我面临 2 个问题。 1) 我将距离设为 NULL 2) 我不确定如何在 5 KM 范围内过滤此结果。

帮我解决这个问题。

PS： 我有另一个临时 Biqquery 表，我直接将数据插入到该表中，然后我可以使用以下查询获得预期的结果。

SELECT
  SQRT(POW(111.1*( MAX(Lattitude) - 12.9632691), 2) + POW(111.1*(80.2140816 -  MAX(Longitude) ) * COS( MAX(Lattitude) / 57.3), 2)) AS distance,
  MAX(SearchKeyword) as SearchKeyword,
  SearcherArea,
  COUNT( SearcherArea) AS totalCount
FROM
  `HNF.LastFewHoursData`
WHERE
 SearchKeyword LIKE '%%'
GROUP BY
  SearcherArea

HAVING
  distance <= 5
ORDER BY
  totalCount DESC
LIMIT
  5

所以我只有查询由 Firebase Analytics 更新的事件表的问题。

Answer 1

此查询可能对您有用：

CREATE TEMP FUNCTION distance(lat1 FLOAT64, lat2 FLOAT64, lon1 FLOAT64, lon2 FLOAT64) AS((
WITH data AS(
SELECT POW(SIN((ACOS(-1) / 180 * (lat1 -lat2)) / 2), 2) + COS(ACOS(-1) / 180 * (lat1)) * COS(ACOS(-1) / 180 * (lat2)) * POW(SIN((ACOS(-1) / 180 * (lon1 -lon2)) / 2), 2) a
)
SELECT 6371 * 2 * ATAN2(SQRT((SELECT a FROM data)), SQRT(1 - (SELECT a FROM data)))
));

WITH data AS(
  SELECT ARRAY<STRUCT<name STRING, params ARRAY<STRUCT<key STRING, value STRUCT<string_value STRING, int_value INT64, float_value FLOAT64, double_value FLOAT64> > > > > [STRUCT("SEARCH_PRODUCT" AS name, [STRUCT("LATITUDE" AS key, STRUCT("" as string_value, 0 AS int_value, 0.0 AS float_value, 12.9632747 AS double_value) AS value), STRUCT("LONGITUDE" AS key, STRUCT("" as string_value, 0 AS int_value, 0.0 AS float_value, 80.2131347 AS double_value) AS value), STRUCT("SEARCHER_ID" AS key, STRUCT("-KkphUBI3_v3kdGq6qTa" as string_value, 0 AS int_value, 0.0 AS float_value, 0.0 AS double_value) AS value), STRUCT("KEYWORDS" AS key, STRUCT("shoes cloths" as string_value, 0 AS int_value, 0.0 AS float_value, 0.0 AS double_value) AS value)] AS params),
                                                                                                                                                                          STRUCT("SEARCH_RANDOM" AS name, [STRUCT("LATITUDE" AS key, STRUCT("" as string_value, 0 AS int_value, 0.0 AS float_value, 12.9632747 AS double_value) AS value), STRUCT("LONGITUDE" AS key, STRUCT("" as string_value, 0 AS int_value, 0.0 AS float_value, 80.2131347 AS double_value) AS value), STRUCT("SEARCHER_ID" AS key, STRUCT("-KkphUBI3_v3kdGq6qTa" as string_value, 0 AS int_value, 0.0 AS float_value, 0.0 AS double_value) AS value), STRUCT("KEYWORDS" AS key, STRUCT("shoes cloths" as string_value, 0 AS int_value, 0.0 AS float_value, 0.0 AS double_value) AS value)] AS params),
                                                                                                                                                                          STRUCT("SEARCH_PRODUCT" AS name, [STRUCT("LATITUDE" AS key, STRUCT("" as string_value, 0 AS int_value, 0.0 AS float_value, 12.9632747 AS double_value) AS value), STRUCT("LONGITUDE" AS key, STRUCT("" as string_value, 0 AS int_value, 0.0 AS float_value, 80.2131347 AS double_value) AS value), STRUCT("SEARCHER_ID" AS key, STRUCT("-xx_id" as string_value, 0 AS int_value, 0.0 AS float_value, 0.0 AS double_value) AS value), STRUCT("KEYWORDS" AS key, STRUCT("shoes cloths" as string_value, 0 AS int_value, 0.0 AS float_value, 0.0 AS double_value) AS value)] AS params)] event_dim UNION ALL

  SELECT ARRAY<STRUCT<name STRING, params ARRAY<STRUCT<key STRING, value STRUCT<string_value STRING, int_value INT64, float_value FLOAT64, double_value FLOAT64> > > > > [STRUCT("SEARCH_PRODUCT" AS name, [STRUCT("LATITUDE" AS key, STRUCT("" as string_value, 0 AS int_value, 0.0 AS float_value, 12.9632747 AS double_value) AS value), STRUCT("LONGITUDE" AS key, STRUCT("" as string_value, 0 AS int_value, 0.0 AS float_value, 80.2131347 AS double_value) AS value), STRUCT("SEARCHER_ID" AS key, STRUCT("-KkphUBI3_v3kdGq6qTa" as string_value, 0 AS int_value, 0.0 AS float_value, 0.0 AS double_value) AS value), STRUCT("KEYWORDS" AS key, STRUCT("shoes cloths" as string_value, 0 AS int_value, 0.0 AS float_value, 0.0 AS double_value) AS value)] AS params),
                                                                                                                                                                          STRUCT("SEARCH_RANDOM" AS name, [STRUCT("LATITUDE" AS key, STRUCT("" as string_value, 0 AS int_value, 0.0 AS float_value, 12.9632747 AS double_value) AS value), STRUCT("LONGITUDE" AS key, STRUCT("" as string_value, 0 AS int_value, 0.0 AS float_value, 80.2131347 AS double_value) AS value), STRUCT("SEARCHER_ID" AS key, STRUCT("-KkphUBI3_v3kdGq6qTa" as string_value, 0 AS int_value, 0.0 AS float_value, 0.0 AS double_value) AS value), STRUCT("KEYWORDS" AS key, STRUCT("shoes cloths" as string_value, 0 AS int_value, 0.0 AS float_value, 0.0 AS double_value) AS value)] AS params),
                                                                                                                                                                          STRUCT("SEARCH_PRODUCT" AS name, [STRUCT("LATITUDE" AS key, STRUCT("" as string_value, 0 AS int_value, 0.0 AS float_value, 12.9632747 AS double_value) AS value), STRUCT("LONGITUDE" AS key, STRUCT("" as string_value, 0 AS int_value, 0.0 AS float_value, 80.2131347 AS double_value) AS value), STRUCT("SEARCHER_ID" AS key, STRUCT("-xxx_id" as string_value, 0 AS int_value, 0.0 AS float_value, 0.0 AS double_value) AS value), STRUCT("KEYWORDS" AS key, STRUCT("shoes cloths" as string_value, 0 AS int_value, 0.0 AS float_value, 0.0 AS double_value) AS value)] AS params)] event_dim UNION ALL

  SELECT ARRAY<STRUCT<name STRING, params ARRAY<STRUCT<key STRING, value STRUCT<string_value STRING, int_value INT64, float_value FLOAT64, double_value FLOAT64> > > > > [STRUCT("SEARCH_PRODUCT" AS name, [STRUCT("LATITUDE" AS key, STRUCT("" as string_value, 0 AS int_value, 0.0 AS float_value, 12.01 AS double_value) AS value), STRUCT("LONGITUDE" AS key, STRUCT("" as string_value, 0 AS int_value, 0.0 AS float_value, 12.01 AS double_value) AS value), STRUCT("SEARCHER_ID" AS key, STRUCT("-xx_id" as string_value, 0 AS int_value, 0.0 AS float_value, 0.0 AS double_value) AS value), STRUCT("KEYWORDS" AS key, STRUCT("pants" as string_value, 0 AS int_value, 0.0 AS float_value, 0.0 AS double_value) AS value)] AS params),
                                                                                                                                                                          STRUCT("SEARCH_PRODUCT" AS name, [STRUCT("LATITUDE" AS key, STRUCT("" as string_value, 0 AS int_value, 0.0 AS float_value, 12.01 AS double_value) AS value), STRUCT("LONGITUDE" AS key, STRUCT("" as string_value, 0 AS int_value, 0.0 AS float_value, 12.01 AS double_value) AS value), STRUCT("SEARCHER_ID" AS key, STRUCT("-xx_id" as string_value, 0 AS int_value, 0.0 AS float_value, 0.0 AS double_value) AS value), STRUCT("KEYWORDS" AS key, STRUCT("shoes cloths" as string_value, 0 AS int_value, 0.0 AS float_value, 0.0 AS double_value) AS value)] AS params),
                                                                                                                                                                          STRUCT("SEARCH_PRODUCT" AS name, [STRUCT("LATITUDE" AS key, STRUCT("" as string_value, 0 AS int_value, 0.0 AS float_value, 12.01 AS double_value) AS value), STRUCT("LONGITUDE" AS key, STRUCT("" as string_value, 0 AS int_value, 0.0 AS float_value, 12.01 AS double_value) AS value), STRUCT("SEARCHER_ID" AS key, STRUCT("-xx_id" as string_value, 0 AS int_value, 0.0 AS float_value, 0.0 AS double_value) AS value), STRUCT("KEYWORDS" AS key, STRUCT("shoes cloths" as string_value, 0 AS int_value, 0.0 AS float_value, 0.0 AS double_value) AS value)] AS params)] event_dim



)

SELECT
  keyword,
  count(1) freq
FROM(
  SELECT
    ARRAY(SELECT AS STRUCT (SELECT value.string_value FROM UNNEST(params) WHERE key = 'KEYWORDS' AND EXISTS(SELECT 1 FROM UNNEST(params) WHERE key = 'SEARCHER_ID' and value.string_value = '-xx_id')) AS keyword, (SELECT value.double_value FROM UNNEST(params) WHERE key = 'LATITUDE' AND EXISTS(SELECT 1 FROM UNNEST(params) WHERE key = 'SEARCHER_ID' and value.string_value = '-xx_id')) latitude, (SELECT value.double_value FROM UNNEST(params) WHERE key = 'LONGITUDE' AND EXISTS(SELECT 1 FROM UNNEST(params) WHERE key = 'SEARCHER_ID' and value.string_value = '-xx_id')) longitude FROM UNNEST(event_dim) WHERE name = 'SEARCH_PRODUCT') data
  FROM data
  WHERE EXISTS(select 1 FROM UNNEST(event_dim) WHERE (SELECT COUNTIF(value.string_value = '-xx_id') FROM UNNEST(params) WHERE key = 'SEARCHER_ID' ) > 0)
),
UNNEST(data)
WHERE keyword IS NOT NULL AND latitude IS NOT NULL AND longitude IS NOT NULL
AND distance(latitude, X, longitude, Y) < 5 -- change X and Y to your desired location
GROUP BY keyword
ORDER BY freq DESC

我做了什么：

首先我们定义了函数distance，它计算两个空间点之间的距离（以度为单位，例如，（纬度12°，经度13°））。

data 只是对您在 Firebase 库中的相同数据的模拟。运行查询时，只需将 data 替换为您的实际 table。

最后，查询本身首先过滤掉名称不是SEARCH_PRODUCT 的事件，同时使用关键字、它们的纬度和经度构建ARRAY（还有一个过滤器可以删除不等于SEARCHER_IDs 的-xx_id )。

之后，只有一个 select 和一个 WHERE 子句，用于仅选择纬度和经度小于您要查询的位置的关键字（我在查询中将其称为 X 和 Y，您可以更改它对于您当前的位置，例如 12.09°、39.01°）。

让我知道这是否适合你。