【问题标题】:Better way to optimize my code for getting NOAA climate data优化我的代码以获取 NOAA 气候数据的更好方法
【发布时间】:2018-09-19 11:47:35
【问题描述】:

所以我一直致力于从 NOAA 的 GHCN 气象站网络获取气候数据(特别是温度、降水)。我已经设法获得了与我所在地区相关的站点列表(约 200 个),并建立了一个循环,以便在指定的最小和最大日期之间的每一天为该列表中的每个站点获取特定的气候变量。最终我需要大约 10 年的数据。然而,我的简单循环需要永远获取这些数据,我想知道是否有更好的方法来优化它?此外,我真的想访问每月数据而不是每天,但 rnoaa 似乎没有 GHCN 每月数据的选项,因为唯一可用的函数是 ghcnd_search()。如果有人也知道如何挖掘每月而不是每日数据,那将不胜感激

车站列表:

df<-c("US1FLAL0048", "US1FLBK0003", "US1FLBV0002", "US1FLBV0006", 
"US1FLBV0023", "US1FLBV0040", "US1FLBW0099", "US1FLCT0012", "US1FLDV0051", 
"US1FLFR0006", "US1FLHL0003", "US1FLHN0009", "US1FLLB0001", "US1FLLE0005", 
"US1FLLK0012", "US1FLLN0004", "US1FLLN0018", "US1FLMN0013", "US1FLMR0012", 
"US1FLMR0033", "US1FLOK0017", "US1FLOR0028", "US1FLPS0002", "US1FLPS0018", 
"US1FLPT0007", "US1FLSJ0012", "US1FLSM0008", "US1FLSS0044", "US1FLST0014", 
"US1FLSW0008", "US1FLVL0035", "US1FLWK0001", "USC00080228", "USC00080236", 
"USC00080369", "USC00080414", "USC00080478", "USC00080598", "USC00080737", 
"USC00080945", "USC00080992", "USC00081163", "USC00081276", "USC00081306", 
"USC00081544", "USC00081641", "USC00081651", "USC00081978", "USC00082008", 
"USC00082046", "USC00082150", "USC00082229", "USC00082288", "USC00082298", 
"USC00082391", "USC00082418", "USC00082441", "USC00082850", "USC00082915", 
"USC00082944", "USC00083020", "USC00083153", "USC00083163", "USC00083168", 
"USC00083207", "USC00083209", "USC00083470", "USC00083874", "USC00083909", 
"USC00083956", "USC00083986", "USC00084050", "USC00084095", "USC00084210", 
"USC00084289", "USC00084320", "USC00084366", "USC00084394", "USC00084412", 
"USC00084461", "USC00084625", "USC00084662", "USC00084731", "USC00084802", 
"USC00085076", "USC00085099", "USC00085184", "USC00085275", "USC00085359", 
"USC00085377", "USC00085539", "USC00085612", "USC00085667", "USC00085879", 
"USC00085895", "USC00085973", "USC00086065", "USC00086078", "USC00086129", 
"USC00086240", "USC00086315", "USC00086406", "USC00086414", "USC00086618", 
"USC00086657", "USC00086764", "USC00086767", "USC00086828", "USC00086842", 
"USC00086999", "USC00087020", "USC00087025", "USC00087205", "USC00087228", 
"USC00087261", "USC00087304", "USC00087397", "USC00087429", "USC00087760", 
"USC00087826", "USC00087851", "USC00087869", "USC00087886", "USC00087982", 
"USC00088368", "USC00088529", "USC00088620", "USC00088756", "USC00088782", 
"USC00088824", "USC00088942", "USC00089120", "USC00089176", "USC00089219", 
"USC00089401", "USC00089430", "USC00089566", "USC00089640", "USC00089795", 
"USR0000FBLO", "USR0000FCAC", "USR0000FCEN", "USR0000FCHE", "USR0000FLSU", 
"USR0000FMER", "USR0000FMIL", "USR0000FNAV", "USR0000FOAS", "USR0000FOCH", 
"USR0000FOLU", "USR0000FRAC", "USR0000FSAN", "USR0000FSTM", "USR0000FSUM", 
"USR0000FWIL", "USW00003818", "USW00003853", "USW00012812", "USW00012815", 
"USW00012816", "USW00012818", "USW00012819", "USW00012832", "USW00012833", 
"USW00012834", "USW00012835", "USW00012836", "USW00012838", "USW00012839", 
"USW00012841", "USW00012842", "USW00012843", "USW00012844", "USW00012849", 
"USW00012850", "USW00012854", "USW00012871", "USW00012873", "USW00012876", 
"USW00012882", "USW00012885", "USW00012888", "USW00012894", "USW00012895", 
"USW00012896", "USW00012897", "USW00013884", "USW00013889", "USW00013899", 
"USW00053847", "USW00053853", "USW00053860", "USW00092805", "USW00092806", 
"USW00092809", "USW00092811", "USW00092821", "USW00093805", "USW00093837", 
"USW00093841")

代码:

library(rnoaa)
options(noaakey = "your api key")
data<-matrix(, nrow=0, ncol=0) #create empty matrix
for (i in 1:length(df)){
  a<-ghcnd_search(stationid=df[1],var='TMAX',date_min='2010-1-30',date_max='2015-12-31')
  data=rbind(data,a$tmax)

}

【问题讨论】:

  • 难以优化,但在循环的每次迭代中增加data 矩阵的成本很高。由于您知道您的日期范围,并且数据是每天的,以及将查询多少个站点,您可以预先分配一个适当大小的矩阵并随时填写矩阵的块。就目前而言,您正在一遍又一遍地破坏和创建数据框。
  • 您确定使用rnoaa 可以获得10 年的数据吗? IIRC 他们有 1000 天的最大限制

标签: r rnoaa


【解决方案1】:

假设站 ID 存储在名为 dat 的向量中,我们可以使用 包中的函数下载数据并创建数据框。

# Load packages
library(rnoaa)
library(purrr)

# Download the data and create a data frame. 
dat_df <- map(dat, ghcnd_search, 
              var='TMAX', date_min = '2010-1-30', date_max = '2015-12-31') %>%
          map_dfr("tmax")

数据

dat<-c("US1FLAL0048", "US1FLBK0003", "US1FLBV0002", "US1FLBV0006", 
      "US1FLBV0023", "US1FLBV0040", "US1FLBW0099", "US1FLCT0012", "US1FLDV0051", 
      "US1FLFR0006", "US1FLHL0003", "US1FLHN0009", "US1FLLB0001", "US1FLLE0005", 
      "US1FLLK0012", "US1FLLN0004", "US1FLLN0018", "US1FLMN0013", "US1FLMR0012", 
      "US1FLMR0033", "US1FLOK0017", "US1FLOR0028", "US1FLPS0002", "US1FLPS0018", 
      "US1FLPT0007", "US1FLSJ0012", "US1FLSM0008", "US1FLSS0044", "US1FLST0014", 
      "US1FLSW0008", "US1FLVL0035", "US1FLWK0001", "USC00080228", "USC00080236", 
      "USC00080369", "USC00080414", "USC00080478", "USC00080598", "USC00080737", 
      "USC00080945", "USC00080992", "USC00081163", "USC00081276", "USC00081306", 
      "USC00081544", "USC00081641", "USC00081651", "USC00081978", "USC00082008", 
      "USC00082046", "USC00082150", "USC00082229", "USC00082288", "USC00082298", 
      "USC00082391", "USC00082418", "USC00082441", "USC00082850", "USC00082915", 
      "USC00082944", "USC00083020", "USC00083153", "USC00083163", "USC00083168", 
      "USC00083207", "USC00083209", "USC00083470", "USC00083874", "USC00083909", 
      "USC00083956", "USC00083986", "USC00084050", "USC00084095", "USC00084210", 
      "USC00084289", "USC00084320", "USC00084366", "USC00084394", "USC00084412", 
      "USC00084461", "USC00084625", "USC00084662", "USC00084731", "USC00084802", 
      "USC00085076", "USC00085099", "USC00085184", "USC00085275", "USC00085359", 
      "USC00085377", "USC00085539", "USC00085612", "USC00085667", "USC00085879", 
      "USC00085895", "USC00085973", "USC00086065", "USC00086078", "USC00086129", 
      "USC00086240", "USC00086315", "USC00086406", "USC00086414", "USC00086618", 
      "USC00086657", "USC00086764", "USC00086767", "USC00086828", "USC00086842", 
      "USC00086999", "USC00087020", "USC00087025", "USC00087205", "USC00087228", 
      "USC00087261", "USC00087304", "USC00087397", "USC00087429", "USC00087760", 
      "USC00087826", "USC00087851", "USC00087869", "USC00087886", "USC00087982", 
      "USC00088368", "USC00088529", "USC00088620", "USC00088756", "USC00088782", 
      "USC00088824", "USC00088942", "USC00089120", "USC00089176", "USC00089219", 
      "USC00089401", "USC00089430", "USC00089566", "USC00089640", "USC00089795", 
      "USR0000FBLO", "USR0000FCAC", "USR0000FCEN", "USR0000FCHE", "USR0000FLSU", 
      "USR0000FMER", "USR0000FMIL", "USR0000FNAV", "USR0000FOAS", "USR0000FOCH", 
      "USR0000FOLU", "USR0000FRAC", "USR0000FSAN", "USR0000FSTM", "USR0000FSUM", 
      "USR0000FWIL", "USW00003818", "USW00003853", "USW00012812", "USW00012815", 
      "USW00012816", "USW00012818", "USW00012819", "USW00012832", "USW00012833", 
      "USW00012834", "USW00012835", "USW00012836", "USW00012838", "USW00012839", 
      "USW00012841", "USW00012842", "USW00012843", "USW00012844", "USW00012849", 
      "USW00012850", "USW00012854", "USW00012871", "USW00012873", "USW00012876", 
      "USW00012882", "USW00012885", "USW00012888", "USW00012894", "USW00012895", 
      "USW00012896", "USW00012897", "USW00013884", "USW00013889", "USW00013899", 
      "USW00053847", "USW00053853", "USW00053860", "USW00092805", "USW00092806", 
      "USW00092809", "USW00092811", "USW00092821", "USW00093805", "USW00093837", 
      "USW00093841")

【讨论】:

  • 嘿@www 我最终在 rnoaa 中使用了一个不同的搜索函数,它的 for 循环格式非常不同。它涉及在 NOAA 挖掘 GSOM 数据集。我想知道是否有一种方法可以与您分享我的循环,因为我需要一些关于如何使用 purrr 函数合并它的建议。遗憾的是,字数限制不允许我复制和粘贴。
  • @LeoOhyama 我认为您可以提出一个关于 GSOM 数据集可重现的新问题。无需共享整个数据集,只需其中一部分就足够了。
猜你喜欢
  • 2020-11-25
  • 1970-01-01
  • 1970-01-01
  • 2021-02-27
  • 1970-01-01
  • 2022-08-18
  • 2011-11-06
  • 1970-01-01
  • 2019-10-04
相关资源
最近更新 更多