如何合并大稀疏矩阵

Question

我有一个包含 25 个稀疏矩阵的大列表（它们真的很大——其中一个包含 100M 或更多元素），我需要将它们合并成一个大的稀疏矩阵。

例如：一个矩阵 A 可能看起来像这样（它是我的 100M 个元素的真实矩阵的子矩阵）：

> A
5 x 4 sparse Matrix of class "dgCMatrix"
              SKU
CustomerID         404     457     547     558     
  100002_24655       1       .       .       .       
  100003_46919       .       1       1       .       
  100007_46702       .       .       .       .       
  100012_47709       .       .       .       .       
  100013_46132       1       1       1       1 

> dput(A)
new("dgCMatrix"
    , i = c(0L, 4L, 1L, 4L, 1L, 4L, 4L)
    , p = c(0L, 2L, 4L, 6L, 7L)
    , Dim = c(5L, 4L)
    , Dimnames = structure(list(CustomerID = c("100002_24655", "100003_46919", 
"100007_46702", "100012_47709", "100013_46132"), SKU = c("404", 
"457", "547", "558")), .Names = c("CustomerID", "SKU"
))
    , x = c(1, 1, 1, 1, 1, 1, 1)
    , factors = list()
)

另一个 B 可能如下所示：

> B
7 x 5 sparse Matrix of class "dgCMatrix"
               SKU
CustomerID          191     404     558     715     787        
  100002_24655        .       .       .       .       .              
  100007_46702        1       1       1       1       1              
  100012_47709        .       .       1       .       .              
  100013_46132        .       .       .       .       1              
  100014_46400        .       .       .       .       .             
  100014_605414       1       1       1       .       .              
  100014_631294       .       .       1       1       1              

> dput(B)
new("dgCMatrix"
    , i = c(1L, 5L, 1L, 5L, 1L, 2L, 5L, 6L, 1L, 6L, 1L, 3L, 6L)
    , p = c(0L, 2L, 4L, 8L, 10L, 13L)
    , Dim = c(7L, 5L)
    , Dimnames = structure(list(CustomerID = c("100002_24655", "100007_46702", 
"100012_47709", "100013_46132", "100014_46400", "100014_605414", 
"100014_631294"), SKU = c("191", "404", "558", "715", 
"787")), .Names = c("CustomerID", "SKU"))
    , x = c(1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1)
    , factors = list()
)

输出应该是这样的：（第一部分是第一个矩阵，第二部分是第二个矩阵 - 为了更好地查看，我将其除以空间）

12 x 7 sparse Matrix of class "dgCMatrix"    
             404  457  547  558  191  715  787    
     [1, ]     1    .    .    .    .    .    .     
     [2, ]     .    1    1    .    .    .    .
     [3, ]     .    .    .    .    .    .    .
     [4, ]     .    .    .    .    .    .    .
     [5, ]     1    1    1    1    .    .    .

     [6, ]     .    .    .    .    .    .    .
     [7, ]     1    .    .    1    1    1    1
     [8, ]     .    .    .    1    .    .    .
     [9, ]     .    .    .    .    .    .    1
     [10,]     .    .    .    .    .    .    .
     [11,]     1    .    .    1    1    .    .
     [12,]     .    .    .    1    .    1    1

这意味着我想按列名合并。那么如何合并所有 25 个稀疏矩阵呢？

Answer 1

所以我编辑了一点dww answear 以避免我在评论中提到的错误。但这有点慢。但我有非常大的矩阵。

> proc.time() - ptm
   user  system elapsed 
572.384 213.179 793.550

这是修改后的代码：

merge.sparse = function(M.list) {
  A = M.list[[1]]

  for (i in 2:length(M.list)){ #i indexes of matrices
    # finding what's missing
    misA = colnames(M.list[[i]])[!colnames(M.list[[i]]) %in% colnames(A)]
    misB = colnames(A)[!colnames(A) %in% colnames(M.list[[i]])]

    misAl = as.vector(numeric(length(misA)), "list")
    names(misAl) = misA
    misBl = as.vector(numeric(length(misB)), "list")
    names(misBl) = misB

    ## adding missing columns to initial matrices
    An = Reduce(cbind, c(A, misAl))
    if (length(misA) > 0)
       {
       lenA <- ncol(An)-length(misA)+1
       colnames(An)[lenA:ncol(An)] = names(misAl)
       }

    Bn = Reduce(cbind, c(M.list[[i]], misBl))
    if(length(misB) > 0)
       {
       lenB <- ncol(Bn)-length(misB)+1
       colnames(Bn)[lenB:ncol(Bn)] = names(misBl)
       }

    Bn <- Bn[,colnames(An)]

    # final bind
    A = rbind(An, Bn, use.names = T)
    print(c(length(M.list), i))
  } 
  A
}