【发布时间】:2016-08-02 14:02:03
【问题描述】:
我正在编写一个 ETL 脚本,在某一时刻,有几个提取的表都应该具有相同的特定维度(16 行,9 列)。
有时会由于上游数据问题而丢失一行,即 15x9。我需要在正确的位置添加缺失的行,用它们的静态、非唯一标签值填充前 2 列。
我尝试创建一个模板 data.frame 并使用sqldf 将非缺失值加入到模板中,将缺失的行与适当的标签和NA 值留在缺失数据的位置。然而,连接的on 条件不明显这一事实导致数据形状发生变化。
# ill-fated attempt, doubles the number of rows
require(sqldf)
sqldf("select a.label1, a.label2, b.data1,
b.data2, b.data3, b.data4, b.data5,
b.data6, b.data7 from t a join d1 b
on a.label1 = b.label1 and a.label2 = b.label2")
可重复的数据如下。上例中的d1 对应于下面的第二个data.frame,template 对应于第三个。
非问题数据示例(此处没有数据丢失):
structure(list(label1 = c("BS", "Certificates", "MS", "PHD",
"BS", "Certificates", "MS", "PHD", "BS", "Certificates", "MS",
"PHD", "BS", "Certificates", "MS", "PHD"), label2 = c("INTL",
"INTL", "INTL", "INTL", "US", "US", "US", "US", "INTL", "INTL",
"INTL", "INTL", "US", "US", "US", "US"), data1 = c("fake data",
"fake data", "fake data", "fake data", "fake data", "fake data",
"fake data", "fake data", "fake data", "fake data", "fake data",
"fake data", "fake data", "fake data", "fake data", "fake data"
), data2 = c(1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15,
16), data3 = c("09B", "09B", "09B", "09B", "09B", "09B", "09B",
"09B", "09B", "09B", "09B", "09B", "09B", "09B", "09B", "09B"
), data4 = c(887L, 220L, 3633L, 1491L, 3331L, 379L, 9521L, 3785L,
624L, 188L, 1842L, 863L, 3119L, 428L, 7868L, 3138L), data5 = c(531L,
62L, 1114L, 625L, 2454L, 286L, 6298L, 2512L, 521L, 161L, 1344L,
649L, 2795L, 397L, 6984L, 2651L), data6 = c(35L, 12L, 389L, 253L,
468L, 140L, 3143L, 1244L, 36L, 7L, 337L, 167L, 621L, 135L, 3116L,
1222L), data7 = c(27L, 10L, 313L, 214L, 404L, 128L, 2853L, 1111L,
22L, 5L, 224L, 123L, 459L, 103L, 2650L, 984L)), .Names = c("label1",
"label2", "data1", "data2", "data3", "data4", "data5", "data6",
"data7"), row.names = c(NA, -16L), class = "data.frame")
问题数据示例(缺失 2 行):
structure(list(label1 = c("BS", "Certificates", "MS", "PHD",
"BS", "MS", "PHD", "BS", "Certificates", "MS", "PHD", "Certificates",
"MS", "PHD"), label2 = c("INTL", "INTL", "INTL", "INTL", "US",
"US", "US", "INTL", "INTL", "INTL", "INTL", "US", "US", "US"),
data1 = c("fake data", "fake data", "fake data", "fake data",
"fake data", "fake data", "fake data", "fake data", "fake data",
"fake data", "fake data", "fake data", "fake data", "fake data"
), data2 = c(1, 2, 3, 4, 5, 7, 8, 9, 10, 11, 12, 14, 15,
16), data3 = c("09B", "09B", "09B", "09B", "09B", "09B",
"09B", "09B", "09B", "09B", "09B", "09B", "09B", "09B"),
data4 = c(887L, 220L, 3633L, 1491L, 3331L, 9521L, 3785L,
624L, 188L, 1842L, 863L, 428L, 7868L, 3138L), data5 = c(531L,
62L, 1114L, 625L, 2454L, 6298L, 2512L, 521L, 161L, 1344L,
649L, 397L, 6984L, 2651L), data6 = c(35L, 12L, 389L, 253L,
468L, 3143L, 1244L, 36L, 7L, 337L, 167L, 135L, 3116L, 1222L
), data7 = c(27L, 10L, 313L, 214L, 404L, 2853L, 1111L, 22L,
5L, 224L, 123L, 103L, 2650L, 984L)), .Names = c("label1",
"label2", "data1", "data2", "data3", "data4", "data5", "data6",
"data7"), row.names = c(NA, -14L), class = "data.frame")
模板示例:
structure(list(label1 = structure(c(1L, 2L, 3L, 4L, 1L, 2L, 3L,
4L, 1L, 2L, 3L, 4L, 1L, 2L, 3L, 4L), .Label = c("BS", "Certificates",
"MS", "PHD"), class = "factor"), label2 = structure(c(1L, 1L,
1L, 1L, 2L, 2L, 2L, 2L, 1L, 1L, 1L, 1L, 2L, 2L, 2L, 2L), .Label = c("INTL",
"US"), class = "factor"), data1 = c(NA, NA, NA, NA, NA, NA, NA,
NA, NA, NA, NA, NA, NA, NA, NA, NA), data2 = c(NA, NA, NA, NA,
NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA), data3 = c(NA,
NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA),
data4 = c(NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA,
NA, NA, NA, NA), data5 = c(NA, NA, NA, NA, NA, NA, NA, NA,
NA, NA, NA, NA, NA, NA, NA, NA), data6 = c(NA, NA, NA, NA,
NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA), data7 = c(NA,
NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA
)), .Names = c("label1", "label2", "data1", "data2", "data3",
"data4", "data5", "data6", "data7"), row.names = c(NA, -16L), class = "data.frame")
【问题讨论】:
标签: r