从逗号或管道运算符字符串中删除重复项

Question

我已经对此进行了一段时间的研究，但我找不到从 SQL Server 中的逗号分隔和管道分隔字符串中删除重复字符串的方法。

给定字符串

test1,test2,test1|test2,test3|test4,test4|test4

有谁知道你会怎么回复test1,test2,test3,test4?

Answer 1

接近

以下方法可用于对分隔的值列表进行重复数据删除。

1. 使用REPLACE() 函数将不同的分隔符转换为相同的分隔符。
2. 使用REPLACE()函数注入XML结束和开始标签来创建一个XML片段
3. 使用CAST(expr AS XML)函数将上述片段转换为XML数据类型
4. 使用OUTER APPLY 应用表值函数nodes() 将XML 片段拆分为其组成的XML 标记。这会在单独的行中返回每个 XML 标记。
5. 使用value() 函数仅从XML 标记中提取值，并使用指定的数据类型返回值。
6. 在上述值后附加逗号。
7. 请注意，这些值是在单独的行中返回的。 DISTINCT 关键字的使用现在删除重复的行（即值）。
8. 使用FOR XML PATH('') 子句将多行中的值连接成一行。

查询

将上述方法放在查询形式中：

SELECT DISTINCT PivotedTable.PivotedColumn.value('.','nvarchar(max)') + ',' 
FROM ( 
        -- This query returns the following in theDataXml column: 
        -- <tag>test1</tag><tag>test2</tag><tag>test1</tag><tag>test2</tag><tag>test3</tag><tag>test4</tag><tag>test4</tag><tag>test4</tag>
        -- i.e. it has turned the original delimited data into an XML fragment 
        SELECT 
          DataTable.DataColumn AS DataRaw 
        , CAST( 
            '<tag>' 
            -- First replace commas with pipes to have only a single delimiter 
            -- Then replace the pipe delimiters with a closing and opening tag 
            + replace(replace(DataTable.DataColumn, ',','|'), '|','</tag><tag>') 
            -- Add a final set of closing tags 
            + '</tag>' 
            AS XML) AS DataXml 
        FROM ( SELECT 'test1,test2,test1|test2,test3|test4,test4|test4' AS DataColumn) AS DataTable 
    ) AS x 
OUTER APPLY DataXml.nodes('tag') AS PivotedTable(PivotedColumn) 
-- Running the query without the following line will return the data in separate rows 
-- Running the query with the following line returns the rows concatenated, i.e. it returns: 
-- test1,test2,test3,test4, 
FOR XML PATH('') 

输入和结果

给定输入：

test1,test2,test1|test2,test3|test4,test4|test4

上面的查询会返回结果：

测试1，测试2，测试3，测试4，

注意末尾的逗号。我会把它作为练习留给你删除它。

---

编辑：重复次数

OP 在评论中请求“我如何获得重复的计数？在单独的列中”。

最简单的方法是使用上述查询但删除最后一行FOR XML PATH('')。然后，计算上述查询中SELECT 表达式返回的所有值和不同值（即PivotedTable.PivotedColumn.value('.','nvarchar(max)')）。所有值的计数与不同值的计数之间的差异是重复值的计数。

SELECT 
    COUNT(PivotedTable.PivotedColumn.value('.','nvarchar(max)'))            AS CountOfAllValues 
  , COUNT(DISTINCT PivotedTable.PivotedColumn.value('.','nvarchar(max)'))   AS CountOfUniqueValues 
    -- The difference of the previous two counts is the number of duplicate values 
  , COUNT(PivotedTable.PivotedColumn.value('.','nvarchar(max)')) 
    - COUNT(DISTINCT PivotedTable.PivotedColumn.value('.','nvarchar(max)')) AS CountOfDuplicateValues 
FROM ( 
        -- This query returns the following in theDataXml column: 
        -- <tag>test1</tag><tag>test2</tag><tag>test1</tag><tag>test2</tag><tag>test3</tag><tag>test4</tag><tag>test4</tag><tag>test4</tag>
        -- i.e. it has turned the original delimited data into an XML fragment 
        SELECT 
          DataTable.DataColumn AS DataRaw 
        , CAST( 
            '<tag>' 
            -- First replace commas with pipes to have only a single delimiter 
            -- Then replace the pipe delimiters with a closing and opening tag 
            + replace(replace(DataTable.DataColumn, ',','|'), '|','</tag><tag>') 
            -- Add a final set of closing tags 
            + '</tag>' 
            AS XML) AS DataXml 
        FROM ( SELECT 'test1,test2,test1|test2,test3|test4,test4|test4' AS DataColumn) AS DataTable 
    ) AS x 
OUTER APPLY DataXml.nodes('tag') AS PivotedTable(PivotedColumn) 

对于上面显示的相同输入，此查询的输出是：

CountOfAllValues CountOfUniqueValues CountOfDuplicateValues
---------------- ------------------- ----------------------
8                4                   4

Answer 2

我刚刚尝试过完美运行以下脚本：

declare @List VARCHAR(MAX)='test1,test2,test1|test2,test3|test4,test4|test4'
declare @Delim CHAR=','
DECLARE @ParsedList TABLE
(
Item VARCHAR(MAX)
)
DECLARE @list1 VARCHAR(MAX), @Pos INT, @rList VARCHAR(MAX)
set @List=Replace(@List,'|',',')
SET @list = LTRIM(RTRIM(@list)) + @Delim
SET @pos = CHARINDEX(@delim, @list, 1)
WHILE @pos > 0
BEGIN
SET @list1 = LTRIM(RTRIM(LEFT(@list, @pos - 1)))
IF @list1 <> ''
INSERT INTO @ParsedList VALUES (CAST(@list1 AS VARCHAR(MAX)))
SET @list = SUBSTRING(@list, @pos+1, LEN(@list))
SET @pos = CHARINDEX(@delim, @list, 1)
END
SELECT @rlist = COALESCE(@rlist+',','') + item
FROM (SELECT DISTINCT Item FROM @ParsedList) t
Select @rlist

Answer 3

尝试以下 SQL 脚本：

declare @List nvarchar(max)='test1,test2,test1|test2,test3|test4,test4|test4';
declare @Delimiter CHAR(1) =','
declare @XML AS XML
declare @result varchar(max)
set @List=Replace(@List,'|',',')
--Select @List

SET @XML = CAST(('<X>'+REPLACE(@List,@Delimiter ,'</X><X>')+'</X>') AS XML)
DECLARE @temp TABLE (Data nvarchar(100))
INSERT INTO @temp
SELECT N.value('.', 'nvarchar(100)') AS Data FROM @XML.nodes('X') AS T(N)
--SELECT distinct * FROM @temp

IF OBJECT_ID('tempdb..#temp') IS NOT NULL DROP TABLE #temp
Select distinct Data into #temp from @temp

SET @result = ''
select @result = @result + Data + ', ' from #temp
select SUBSTRING(@result, 0, LEN(@result))

Answer 4

您的问题的解决方案如下：

DECLARE @Data_String AS VARCHAR(1000), @Result as varchar(1000)=''
SET @Data_String = 'test1,test2,test1|test2,test3|test4,test4|test4'
SET @Data_String = REPLACE(@Data_String,'|',',')
SELECT @Result=@Result+col+',' from(
SELECT DISTINCT t.c.value('.','varchar(100)') col from(
SELECT cast('<A>'+replace(@Data_String,',','</A><A>')+'</A>' as     xml)col1)data cross apply col1.nodes('/A') as t(c))Data
SELECT LEFT(@Result,LEN(@Result)-1)

结果

test1,test2,test3,test4

Answer 5

    DECLARE @string AS VARCHAR(1000) 
    SET @string = 'test1,test2,test1|test2,test3|test4,test4|test4'
    SET @string = REPLACE(@string,'|',',')
    DECLARE @t TABLE (val VARCHAR(MAX)) 

    DECLARE @xml XML
    SET @xml = N'<root><r>' + REPLACE(@string, ',', '</r><r>') +         '</r></root>'
    INSERT INTO @t(val) SELECT r.value('.','VARCHAR(MAX)') as Item FROM         @xml.nodes('//root/r') AS RECORDS(r)
    ;WITH cte
    AS (SELECT ROW_NUMBER() OVER (PARTITION BY val ORDER BY val desc) RN
    FROM  @t)
    DELETE FROM cte
    WHERE  RN > 1