【问题标题】:Parent child hierarchy path without using CTE不使用 CTE 的父子层次结构路径
【发布时间】:2017-03-08 10:22:55
【问题描述】:

您好,我有以下表格:

create table Features
(
FeatureId bigint,
FeatureName varchar(255),
ParentId bigint
)

insert into Features values(10, 'Feature 1', 1);
insert into Features values(11, 'Feature 2', 10);
insert into Features values(12, 'Feature 3', 11);
insert into Features values(13, 'Feature 4', 2);
insert into Features values(14, 'Feature 5', 13);

insert into Features values(15, 'Feature 6', 3);
insert into Features values(16, 'Feature 7', 15);
insert into Features values(17, 'Feature 8', 16);
insert into Features values(18, 'Feature 9', 17);
insert into Features values(19, 'Feature 10', 18);
insert into Features values(20, 'Feature 11', 19);
insert into Features values(21, 'Feature 12', 12);


create table Scenarios
(
ScenarioId bigint,
ParentId bigint,
ScenarioTitle varchar(25)
)

insert into Scenarios values(1, 0, 'Scenario 1')
insert into Scenarios values(2, 0, 'Scenario 2')
insert into Scenarios values(3, 0, 'Scenario 3')

在这里,一个特征可以有另一个特征作为父级,也可以有一个场景作为父级。对于场景,父 ID 可以是 0,也可以是其他场景。

我想得到每个特征的路径如下:

FeatureId   ParentId    FeatureName PathString          PathLength
10          1           Feature 1   1                   0
11          10          Feature 2   1/10                1
12          11          Feature 3   1/10/11             2
13          2           Feature 4   2                   0
14          13          Feature 5   2/13                1
15          3           Feature 6   3                   0
16          15          Feature 7   3/15                1
17          16          Feature 8   3/15/16             2
18          17          Feature 9   3/15/16/17          3
19          18          Feature 10  3/15/16/17/18       4
20          19          Feature 11  3/15/16/17/18/19    5
21          12          Feature 12  1/10/11/12          3

由于我想将此结果收集到临时表中以进行进一步处理,因此我尝试了 select into 并且 Azure SQL DW 抛出 Using SELECT INTO statement is not supported in Parallel Data Warehouse. Modify the statement and re-try executing it.

这是我的查询(可能不是很好,因为我仍在计算递归 sql)

drop table FeaturesWithPath;
;WITH FeaturePaths (FeatureId, ParentId, FeatureName, PathString)
AS
(
-- Anchor member definition
    SELECT g.FeatureId, g.ParentId, g.FeatureName, cast(CAST(g.ParentId as nvarchar(max)) as varchar(max)) as PathString
    FROM dbo.Features AS g
    UNION ALL
-- Recursive member definition
    SELECT g.FeatureId, g.ParentId, g.FeatureName, PathString + '/' + cast(g.ParentId as varchar(max))
    FROM dbo.Features AS g
    INNER JOIN FeaturePaths AS gp
        ON g.ParentId = gp.FeatureId
)

SELECT FeatureId, ParentId, FeatureName, PathString into FeaturesWithPath FROM FeaturePaths;
--select * from FeaturesWithPath order by FeatureId;

drop table FeaturesWithPathLength;
select *, LEN(PathString) - LEN(REPLACE(PathString, '/', '')) as PathLength into FeaturesWithPathLength from FeaturesWithPath
--select * from FeaturesWithPathLength order by FeatureId

drop table MaxFeaturePathLenghtRowTable;

select * into MaxFeaturePathLenghtRowTable
from FeaturesWithPathLength
where PathLength = (select max(PathLength) from FeaturesWithPathLength as f where f.FeatureId = FeaturesWithPathLength.FeatureId)
or PathLength = (select max(PathLength) from FeaturesWithPathLength as f where f.FeatureId = FeaturesWithPathLength.FeatureId
    and PathLength > (select max(PathLength) from FeaturesWithPathLength as f2 where f2.FeatureId = FeaturesWithPathLength.FeatureId));

--select * from MaxFeaturePathLenghtRowTable order by FeatureId

drop table FeaturesPerParentTable
select FeatureId, [value] as NewParentId, FeatureName, COALESCE(NULLIF(SUBSTRING(PathString, 0, CHARINDEX('/', PathString)), ''), [value]) AS ScenarioId into FeaturesPerParentTable
    from MaxFeaturePathLenghtRowTable 
    cross apply STRING_SPLIT (PathString, '/') cs order by FeatureId

select * from FeaturesPerParentTable order by FeatureId;

我尝试将 CTE 转换为使用 CTAS,但这也不起作用。

这就是我尝试 CTAS 的方式:

;WITH FeaturePaths (FeatureId, ParentId, FeatureName, PathString)
AS
(
-- Anchor member definition
    SELECT g.FeatureId, g.ParentId, g.FeatureName, cast(CAST(g.ParentId as nvarchar(max)) as varchar(max)) as PathString
    FROM dbo.Features AS g
    --WHERE parentId=0
    UNION ALL
-- Recursive member definition
    SELECT g.FeatureId, g.ParentId, g.FeatureName, PathString + '/' + cast(g.ParentId as varchar(max))
    FROM dbo.Features AS g
    INNER JOIN FeaturePaths AS gp
        ON g.ParentId = gp.FeatureId
)

CREATE TABLE #tmp_fct
WITH
(
    DISTRIBUTION = ROUND_ROBIN
)
AS
SELECT  FeatureId, ParentId, FeatureName, PathString
FROM    FeaturePaths;

现在我想知道是否有办法获取Azure SQL DW 上每个功能的路径并将结果收集到表格中。

-- 更新--

SQL中的解决方案see this

这是 C# 中的解决方案

void Main()
{
    var scenarios = new List<Scenario> {
        new Scenario{Id = 1, Title = "Scenario 1", ParentId = 0},
        new Scenario{Id = 2, Title = "Scenario 2", ParentId = 0},
        new Scenario{Id = 3, Title = "Scenario 3", ParentId = 0},
    };

    var features = new List<Feature> {
            new Feature{Id =10, Title = "Feature 1", ParentId =1},
            new Feature{Id =11, Title = "Feature 2", ParentId =10},
            new Feature{Id =12, Title = "Feature 3", ParentId =11},
            new Feature{Id =13, Title = "Feature 4", ParentId =2},
            new Feature{Id =14, Title = "Feature 5", ParentId =13},

            new Feature{Id =15, Title = "Feature 6", ParentId =3},
            new Feature{Id =16, Title = "Feature 7", ParentId =15},
            new Feature{Id =17, Title = "Feature 8", ParentId =16},
            new Feature{Id =18, Title = "Feature 9", ParentId =17},
            new Feature{Id =19, Title = "Feature 10", ParentId =18},
            new Feature{Id =20, Title = "Feature 11", ParentId =19},
            new Feature{Id =21, Title = "Feature 12", ParentId =12}
        };

    var scenarioIds = new HashSet<long>(scenarios.Select(x => x.Id));

    //get path
    IList<Feature> withPath = features.Select(x => { x.Path = GetPath(x, features, scenarioIds); return x; }).ToList().Dump("With path");
}

private string GetPath(Feature f, IList<Feature> features, HashSet<long> scenarioIds)
{
    if (scenarioIds.Contains(f.ParentId))
    {
        return f.ParentId.ToString();
    }
    else
    {
        var parent = features.First(d => d.Id == f.ParentId);
        return GetPath(parent, features, scenarioIds) + "/" + f.ParentId;
    }
}

public class Scenario
{
    public long Id { get; set; }
    public string Title { get; set; }
    public long ParentId { get; set; }
}

public class Feature
{
    public long Id { get; set; }
    public string Title { get; set; }
    public long ParentId { get; set; }
    public string Path { get; set; } //temp
}

【问题讨论】:

  • 预先创建临时表,而不是构造 SELECT ... INTO &lt;temporary table&gt; ...。所以CREATE TABLE &lt;temporary table&gt; ...; INSERT INTO &lt;temporary table&gt;(...) SELECT ...。 PS:使用真正的临时表,即以# 开头的表名。例如CREATE TABLE #MaxFeaturePathLenghtRowTable(...).
  • @TT。如果我尝试插入,它会抛出Common table expressions followed by INSERT, UPDATE, DELETE or MERGE are not supported in this versionSelect Into 是唯一的工作声明。
  • 啊,是的,Azure 及其愚蠢的限制。抱歉,我被固定在 sql-server 和 sql 标签上...
  • 您能否提供一个示例,说明您在问题中对 CTAS 进行了哪些尝试,以及它产生的错误?查看至少应该在 Azure 中支持的 Features and Limitations of Common Table Expressions in SQL Data Warehouse and Parallel Data WarehouseA CTE can be specified in a CREATE TABLE AS SELECT (CTAS) statement.
  • 哦,擦掉最后一句话;来自同一链接:A common table expression that includes references to itself (a recursive common table expression) is not supported.。您将不得不求助于使用 CURSOR 的方法的解决方案...

标签: sql sql-server common-table-expression azure-sqldw


【解决方案1】:

由于目前 Azure SQL 数据仓库不支持递归 CTE 或游标,因此您可以使用良好的老式循环来执行此操作,例如:

-- Loop thru Features
DECLARE @counter INT = 1;

-- Insert first record where no parent exists
IF OBJECT_ID('tempdb..#features') IS NOT NULL DROP TABLE #features;

CREATE TABLE #features
WITH
    (
    DISTRIBUTION = HASH ( FeatureId ),
    LOCATION = USER_DB
    )
AS
WITH cte AS
(
SELECT 1 AS xlevel, p.FeatureId, p.ParentId, p.FeatureName, CAST( p.ParentId AS VARCHAR(255) ) AS PathString, 0 AS PathLength
FROM dbo.Features p
WHERE NOT EXISTS 
    (
    SELECT *
    FROM dbo.Features c
    WHERE p.ParentId = c.FeatureId
    )
)
SELECT *
FROM cte;


SELECT 'before' s, * FROM #features ORDER BY FeatureId;

-- Loop recursively through the child records
WHILE EXISTS (
SELECT *
    FROM #features p
        INNER JOIN dbo.features c ON p.FeatureId = c.ParentId
    WHERE p.xlevel = @counter
    )
BEGIN

    -- Insert next level
    INSERT INTO #features ( xlevel, FeatureId, ParentId, FeatureName, PathString, PathLength )
    SELECT @counter + 1 AS xlevel, c.FeatureId, c.ParentId, c.FeatureName, p.PathString + '/' + CAST( c.ParentId AS VARCHAR(255) ) AS PathString, @counter AS PathLength
    FROM #features p
        INNER JOIN dbo.features c ON p.FeatureId = c.ParentId
    WHERE p.xlevel = @counter;

    SET @counter += 1;

    -- Loop safety
    IF @counter > 99
    BEGIN 
        RAISERROR( 'Too many loops!', 16, 1 ) 
        BREAK 
    END;

END


SELECT 'after' s, *  FROM #features ORDER BY FeatureId;

包含设置的完整代码可用here

我的结果:

希望对您有所帮助。

【讨论】:

  • 谢谢@wBob 这是完美的
  • 我注意到它运行非常慢,可能是因为循环太多。
  • 好的,它在我的 ADW 上运行 3 秒。你有更大的数据集吗?我们可以考虑调整它,例如我的初始版本是'while @counter
  • 与上述相同的数据,在 Visual Studio 上运行,只需 30 秒!我的实际数据大约有 100K 行。
  • 什么 DWU?公平地说,我只跑了 200,数据集太小了。
【解决方案2】:

为什么不预先创建 FeaturesWithPath 表并使用以下伪代码插入其中?


CREATE TABLE FeaturesWithPath (FeatureId type, ParentId type, FeatureName type, PathString type)

;with FeaturePaths (FeatureId, ParentId, FeatureName, PathString)
AS
(
-- Anchor member definition
    SELECT g.FeatureId, g.ParentId, g.FeatureName, cast(CAST(g.ParentId as nvarchar(max)) as varchar(max)) as PathString
    FROM dbo.Features AS g
    UNION ALL
-- Recursive member definition
    SELECT g.FeatureId, g.ParentId, g.FeatureName, PathString + '/' + cast(g.ParentId as varchar(max))
    FROM dbo.Features AS g
    INNER JOIN FeaturePaths AS gp
        ON g.ParentId = gp.FeatureId
)
insert FeaturesWithPath (FeatureId, ParentId, FeatureName, PathString)
SELECT FeatureId, ParentId, FeatureName, PathString FROM FeaturePaths;

【讨论】:

  • 我得到了这个异常:Common table expressions followed by INSERT, UPDATE, DELETE or MERGE are not supported in this version.
  • 阿米特,参见this page 上的示例 F。似乎它会为您提供类似于您正在寻找的东西而无需插入语句。
  • 我仍然应该将结果收集到另一个表中以进行进一步处理。我上面分享的示例结果只是我要解决的问题的开始。
  • 我以前从未使用过 PDW,但如果它支持 while 循环,您可以使用它来代替递归 cte `while(没有未映射到其祖先的行) 开始更新功能集 pathstring = pathstring + '/' + parent where pathstring is not full填充 end `
猜你喜欢
  • 1970-01-01
  • 1970-01-01
  • 1970-01-01
  • 1970-01-01
  • 1970-01-01
  • 1970-01-01
  • 1970-01-01
  • 2011-04-23
  • 1970-01-01
相关资源
最近更新 更多