雪花 SQL UDF 不支持子查询答案

【问题标题】：snowflake SQL UDF not supporting subquery雪花 SQL UDF 不支持子查询
【发布时间】：2023-02-09 05:39:26
【问题描述】：

Snowflake SQL UDF 在将硬编码值作为输入参数传递时工作正常，但在 SELECT 子句中通过将实际列作为输入参数传递来使用 UDF 时失败。

已在 snowflake 社区检查过这个错误，但似乎这个错误是 4 年前报告的，目前还没有解决方案。

请有人帮助提供替代解决方案来克服雪花的这个错误。

CREATE OR REPLACE FUNCTION UDF_GET_CURR_CONV_VALUES(BASE_NET_VALUE FLOAT,EX_PRICE_DATE DATE,EX_RATE_TYPE VARCHAR(20),FROM_CURR VARCHAR(10),TO_CURR VARCHAR(10))
RETURNS VARCHAR(16777216)
LANGUAGE SQL
COMMENT='This function will return Ex rate value, net value and converted net value based on the input parameter.'
AS $$
 
case 
    when FROM_CURR = TO_CURR then ('|'||BASE_NET_VALUE||'|'||BASE_NET_VALUE)
    
else 
    (select (EXCHANGE_RATE_VALUE||'|'||ACT_BASE_NET_VALUE||'|'||CONV_NET_VALUE) 
    from
        (select
            case 
                when ( (2 - CURRENCY.CURRENCY_DECIMAL_PLACES) < 0 ) then round((BASE_NET_VALUE / power(10, -1 * (2 - CURRENCY.CURRENCY_DECIMAL_PLACES)))/TCURF.FROM_CURRENCY_RATIO,2) 
                when ( (2 - CURRENCY.CURRENCY_DECIMAL_PLACES) > 0 ) then round((BASE_NET_VALUE * power(10, 1 * (2 - CURRENCY.CURRENCY_DECIMAL_PLACES)))/TCURF.FROM_CURRENCY_RATIO,2)
            else round(BASE_NET_VALUE,2) end as ACT_BASE_NET_VALUE
            ,round((ACT_BASE_NET_VALUE * EXRATE.EXCHANGE_RATE_VALUE),2) as CONV_NET_VALUE
            ,EXRATE.EXCHANGE_RATE_VALUE as EXCHANGE_RATE_VALUE
            
        from MY_SCHEMA.MY_EXCHANGE_RATES EXRATE
        LEFT JOIN MY_SCHEMA.CURRENCY CURRENCY ON CURRENCY.CURRENCY_KEY = FROM_CURR AND CURRENCY.DELETED = 'N'
        LEFT JOIN (select * from MY_SCHEMA.EXCHANGE_RATE_CONVERSION_FACTORS where DELETED = 'N'
                    QUALIFY ROW_NUMBER() OVER (PARTITION BY EXCHANGE_RATE_TYPE,FROM_CURRENCY,TO_CURRENCY,VALID_FROM ORDER BY VALID_FROM DESC) = 1) TCURF
                    ON TCURF.FROM_CURRENCY = FROM_CURR AND TCURF.TO_CURRENCY = TO_CURR AND TCURF.EXCHANGE_RATE_TYPE = EX_RATE_TYPE
                    
        where equal_null(FROM_CURR,EXRATE.SOURCE_CURRENCY) 
            and EXRATE.EXCHANGE_RATE_TYPE = EX_RATE_TYPE
            and (EX_PRICE_DATE BETWEEN EXRATE.EXCHANGE_RATE_DATE AND EXRATE.VALID_TO_DATE)
            and EXRATE.TARGET_CURRENCY = TO_CURR
            and EXRATE.DELETED = 'N'
        ))
end
$$;

当我使用作为输入参数传递的样本值调用 UDF 时，上述函数有效。

select 
     try_to_double(split_part(my_schema.UDF_GET_CURR_CONV_VALUES(44131.26,to_date('2020-04-24'),'M','EUR','USD'),'|',1)) as EX_RATE_VALUE 
    ,try_to_double(split_part(my_schema.UDF_GET_CURR_CONV_VALUES(44131.26,to_date('2020-04-24'),'M','EUR','USD'),'|',2)) as BASE_VALUE
    ,try_to_double(split_part(my_schema.UDF_GET_CURR_CONV_VALUES(44131.26,to_date('2020-04-24'),'M','EUR','USD'),'|',3)) as USD_BASE_VALUE
 
;

但是，当尝试通过将实际列作为输入参数传递给 select 子句时，也会出现“SQL 编译错误：无法评估不支持的子查询类型”错误。

select
    TXN_NO
    ,try_to_double(split_part(my_schema.UDF_GET_CURR_CONV_VALUES(NET_VALUE,PRICE_DATE,RATE_TYPE,SOURCE_CURRENCY,TARGET_CURRENCY),'|',1)) as EX_RATE_VALUE 
    ,try_to_double(split_part(my_schema.UDF_GET_CURR_CONV_VALUES(NET_VALUE,PRICE_DATE,RATE_TYPE,SOURCE_CURRENCY,TARGET_CURRENCY),'|',2)) as BASE_VALUE
    ,try_to_double(split_part(my_schema.UDF_GET_CURR_CONV_VALUES(NET_VALUE,PRICE_DATE,RATE_TYPE,SOURCE_CURRENCY,TARGET_CURRENCY),'|',3)) as USD_BASE_VALUE
 
FROM MY_SCHEMA.MY_TRANSACTION_TABLE
WHERE TXN_NO = 'ABCXYZ'
;

【问题讨论】：

您是否尝试过利用 CTE 而不是子查询？这通常可以让 Snowflake 以稍微不同的方式解释查询并避免该错误。
你的 UDF 感觉它缺少某种形式的聚合/行限制.. 其中一些像 EX_PRICE_DATE BETWEEN exrate.exchange_rate_date AND exrate.valid_to_date 感觉它会得到很多行。此外，您的 ROW_NUMBER 在分区依据和排序依据中都有 valid_from。
@MikeWalton，似乎 SQL UDF 不支持 CTE。无论如何，我可以通过将列作为输入参数传递来在函数中使用此连接并在 SELECT 子句中动态调用函数。 SimeonPilgrim，是的，我可以在 Partition 中删除 valid_from 并仅在 ORDER BY 子句中使用，使用 between 过滤的价格日期正在处理用例场景。是否有可能使用这些连接创建函数并在 SELECT 子句中即时使用它。
@SimeonPilgrim，在我的用例场景中EX_PRICE_DATE BETWEEN exrate.exchange_rate_date AND exrate.valid_to_date 将只返回一个匹配条目。由于给定的交易日期将介于汇率日期和有效日期之间。所以它不返回多行。我仍然在寻求任何帮助。我也在雪花中尝试使用 Python UDF，但无法在 python UDF 中执行查询。尝试了多种方法仍然无法解决这个问题。如上所述，CTE 概念在 SQL UDF 中不被接受，它似乎在雪花中。

标签： sql snowflake-cloud-data-platform user-defined-functions

【解决方案1】：

因此，为了将其转化为一个更清晰的示例，我首先重构了您的代码，并在某一时刻将其归结为需要一些相关值 (BASE_NET_VALUE)。

所以我做了一个新函数，只做数学位：

CREATE OR REPLACE FUNCTION udf_math_bits_curr_conv_values(base_net_value FLOAT, exchange_rate_value FLOAT, currency_decimal_places INT, from_currency_ratio FLOAT, from_curr VARCHAR(10), to_curr VARCHAR(10))
RETURNS TEXT
LANGUAGE SQL
AS $$
 
CASE 
    WHEN from_curr = to_curr THEN ( '|' || base_net_value || '|' || base_net_value ) 
ELSE 
    (SELECT ( exchange_rate_value || '|' || act_base_net_value || '|' || conv_net_value ) 
    FROM
        (SELECT
            CASE 
                WHEN ( (2 - currency_decimal_places) < 0 ) THEN round( (base_net_value / power(10, -1 * (2 - currency_decimal_places)) ) / from_currency_ratio, 2) 
                WHEN ( (2 - currency_decimal_places) > 0 ) THEN round( (base_net_value * power(10,  1 * (2 - currency_decimal_places)) ) / from_currency_ratio, 2)
                ELSE round(base_net_value, 2) 
            END AS act_base_net_value
            ,round((act_base_net_value * exchange_rate_value), 2) AS conv_net_value
        )
    )
END
$$;

这可用于构建您最想要的答案部分，例如旧 UDF 的工作方式，并与一些示例数据混合：

with MY_TRANSACTION_TABLE(NET_VALUE, SOURCE_CURRENCY, TARGET_CURRENCY) as (
    select * from values
    (1, 'USD', 'NZD'),
    (100, 'SAME', 'SAME')
)
select
    udf_math_bits_curr_conv_values(mtt.NET_VALUE,  1.023::float, 3, 1.023::float, mtt.SOURCE_CURRENCY, mtt.TARGET_CURRENCY) as r1
    ,split(r1, '|') as AA
    ,try_to_double(AA[0]::text) as EX_RATE_VALUE 
    ,try_to_double(AA[1]::text) as BASE_VALUE
    ,try_to_double(AA[2]::text) as USD_BASE_VALUE
FROM MY_TRANSACTION_TABLE as mtt

给出：

R1	AA	EX_RATE_VALUE	BASE_VALUE	USD_BASE_VALUE
1.023\|0.1\|0.1	[ "1.023", "0.1", "0.1" ]	1.023	0.1	0.1
\|100\|100	[ "", "100", "100" ]	null	100	100

这只是表明相同的货币 CASE 仍然有效。

然后我们将 UDF 的其余部分移动到一个可连接的表中。

像这样：

SELECT
    exrate.source_currency AS from_curr /* equal_null(from_curr , exrate.source_currency)  */
    ,exrate.target_currency AS to_curr
    ,exrate.exchange_rate_date
    ,exrate.valid_to_date
    ,exrate.exchange_rate_type as ex_rate_type
    ,currency.currency_decimal_places
    ,tcurf.from_currency_ratio
    ,exrate.exchange_rate_value
FROM my_schema.my_exchange_rates AS exrate
LEFT JOIN my_schema.currency AS currency 
    ON currency.currency_key = exrate.source_currency 
        AND currency.deleted = 'N'
LEFT JOIN (
    SELECT 
        from_currency, 
        to_currency, 
        exchange_rate_type, 
        from_currency_ratio
    FROM my_schema.exchange_rate_conversion_factors 
    WHERE deleted = 'N'
    QUALIFY ROW_NUMBER() OVER (PARTITION BY exchange_rate_type, from_currency, to_currency, valid_from ORDER BY valid_from DESC) = 1
) AS tcurf
    ON tcurf.from_currency = exrate.source_currency 
        AND tcurf.to_currency = exrate.target_currency 
        AND tcurf.exchange_rate_type = exrate.exchange_rate_type        
WHERE exrate.deleted = 'N'

我们将其加入到主 SQL 中，例如：

select
    udf_math_bits_curr_conv_values(mtt.NET_VALUE, x.exchange_rate_value, x.from_currency_ratio, x.from_currency_ratio, mtt.source_currency, mtt.target_currency) as r1
    ,split(r1, '|') as AA
    ,try_to_double(AA[0]::text) as EX_RATE_VALUE 
    ,try_to_double(AA[1]::text) as BASE_VALUE
    ,try_to_double(AA[2]::text) as USD_BASE_VALUE
FROM MY_TRANSACTION_TABLE as mtt
JOIN (
    SELECT
        exrate.source_currency AS from_curr
        ,exrate.target_currency AS to_curr
        ,exrate.exchange_rate_date
        ,exrate.valid_to_date
        ,exrate.exchange_rate_type as ex_rate_type
        ,currency.currency_decimal_places
        ,tcurf.from_currency_ratio
        ,exrate.exchange_rate_value
    FROM my_schema.my_exchange_rates AS exrate
    LEFT JOIN my_schema.currency AS currency 
        ON currency.currency_key = exrate.source_currency 
            AND currency.deleted = 'N'
    LEFT JOIN (
        SELECT 
            from_currency, 
            to_currency, 
            exchange_rate_type, 
            from_currency_ratio
        FROM my_schema.exchange_rate_conversion_factors 
        WHERE deleted = 'N'
        QUALIFY ROW_NUMBER() OVER (PARTITION BY exchange_rate_type, from_currency, to_currency, valid_from ORDER BY valid_from DESC) = 1
    ) AS tcurf
        ON tcurf.from_currency = exrate.source_currency 
            AND tcurf.to_currency = exrate.target_currency 
            AND tcurf.exchange_rate_type = exrate.exchange_rate_type        
    WHERE exrate.deleted = 'N'
) AS x
    ON mtt.price_date BETWEEN x.exchange_rate_date AND x.valid_to_date
        AND mtt.rate_type = x.ex_rate_type
        AND equal_null(mtt.source_currency, X.from_curr)
        AND mtt.target_currency = x.to_curr
WHERE mtt.txn_no = 'ABCXYZ'

我没有运行这个最终的 SQL，因为我没有表.. 但它应该工作/指向正确的方向。

如果你想避免大的子选择，那部分可以被推入 CTE 或 UDTF

【讨论】：

谢谢。但是，我希望这些逻辑在一个函数中执行，并希望将其与其他对象一起使用。似乎 SQL UDF 不支持 WITH 子句在 UDF 中尝试使用 WITH 子句时出现错误。我只是尝试使用此函数并通过在 select 语句中动态传递相应的输入参数来获取货币转换值和转换后的金额，同时从任何需要将金额转换为不同货币的对象中检索数据。

【解决方案2】：

我以某种方式尝试使用 UDTF 来实现此目的，但不确定这是否是最佳实践。如果这不是理想的方法，请回复。

UDTF

CREATE OR REPLACE FUNCTION UDTF_GET_CURR_CONV_VALUES(BASE_NET_VALUE FLOAT,EX_PRICE_DATE DATE,EX_RATE_TYPE VARCHAR(20),FROM_CURR VARCHAR(10),TO_CURR VARCHAR(10))
RETURNS TABLE (CONVERTED_NET_PRICE VARCHAR(16777216))
LANGUAGE SQL
COMMENT='This function will return Ex rate value, net value and converted net value based on the input parameter.'
AS $$
select
    case 
        when FROM_CURR = TO_CURR then ('1'||'|'||BASE_NET_VALUE)
        
    else 
        (EXCHANGE_RATE_VALUE||'|'||round(((case 
                                                when ( (2 - CURRENCY.CURRENCY_DECIMAL_PLACES) < 0 ) then round((BASE_NET_VALUE / power(10, -1 * (2 - CURRENCY.CURRENCY_DECIMAL_PLACES)))/TCURF.FROM_CURRENCY_RATIO,2)
                                                when ( (2 - CURRENCY.CURRENCY_DECIMAL_PLACES) > 0 ) then round((BASE_NET_VALUE * power(10, 1 * (2 - CURRENCY.CURRENCY_DECIMAL_PLACES)))/TCURF.FROM_CURRENCY_RATIO,2)
                                            else round(BASE_NET_VALUE,2) end) * EXRATE.EXCHANGE_RATE_VALUE),2))
    end as CONVERTED_NET_PRICE
                
from MY_SCHEMA.MY_EXCHANGE_RATES EXRATE
LEFT JOIN MY_SCHEMA.CURRENCY CURRENCY ON CURRENCY.CURRENCY_KEY = FROM_CURR AND CURRENCY.DELETED = 'N'
LEFT JOIN (select * from MY_SCHEMA.EXCHANGE_RATE_CONVERSION_FACTORS where DELETED = 'N'
            QUALIFY ROW_NUMBER() OVER (PARTITION BY EXCHANGE_RATE_TYPE,FROM_CURRENCY,TO_CURRENCY ORDER BY VALID_FROM DESC) = 1) TCURF
            ON TCURF.FROM_CURRENCY = FROM_CURR AND TCURF.TO_CURRENCY = TO_CURR AND TCURF.EXCHANGE_RATE_TYPE = EX_RATE_TYPE
            
where equal_null(FROM_CURR,EXRATE.SOURCE_CURRENCY) 
    and EXRATE.EXCHANGE_RATE_TYPE = EX_RATE_TYPE
    and (EX_PRICE_DATE BETWEEN EXRATE.EXCHANGE_RATE_DATE AND EXRATE.VALID_TO_DATE)
    and EXRATE.TARGET_CURRENCY = TO_CURR
    and EXRATE.DELETED = 'N'

$$;

工作正常的示例查询

SELECT
    TXN.TXN_NO,TXN.NET_VALUE,TXN.PRICE_DATE,TXN.RATE_TYPE,TXN.SOURCE_CURRENCY,TXN.TARGET_CURRENCY
    ,split_part(T.CONVERTED_NET_PRICE,'|',1) as EX_RATE_VALUE
    ,split_part(T.CONVERTED_NET_PRICE,'|',2) as USD_NET_VALUE
    
FROM MY_SCHEMA.MY_TRANSACTION_TABLE TXN
,TABLE(MY_SCHMEA.UDTF_GET_CURR_CONV_VALUES(TXN.NET_VALUE,TXN.PRICE_DATE,TXN.RATE_TYPE,TXN.SOURCE_CURRENCY,TXN.TARGET_CURRENCY)) T
WHERE TXN.TXN_NO = 'ABCXYZ'

【讨论】：