如果我理解您的操作正确，这些是您尝试执行的步骤：
- 获取网址
- 对于响应中的每个团队，提取所有玩家的列表
- 对于每个玩家，更新他们在数据库中的数据
稍微改组你的代码并将await batch.commit()移出for循环（这使你的代码在移动到下一个之前等待每个批处理完成），给出：
const response = await fetch(url)
if (!response.ok || response.status === 204) {
// A 204 code will break response.json() with a parsing error
// You might want to check for the 429 status here
throw new Error(`Unexpected status code ${response.status}!`)
}
const json = await response.json() // note: empty bodies will throw a parsing error
if (!json.hasOwnProperty("data")) {
throw new Error(`Unexpected response body!`, json)
}
const teams = json["data"]
const players = teams.flatMap((team) => {
return team.squad.data // the array of players in this team
})
const playersInBatches = [];
while (players.length > 0) {
const thisPlayerBatch = players.splice(0, 500)
playersInBatches.push(thisPlayerBatch)
}
const batches = playersInBatches.map((playersInThisBatch) => {
const dbBatch = database.batch()
for (let player of playersInThisBatch) {
const reference = database
.collection("players")
.doc(`${player.player_id}`)
dbBatch.set(reference, player, { merge: true })
}
return dbBatch
}
// commit all batches in parallel and wait for them to finish
await Promise.all(batches.map((b) => b.commit()))
console.log("Synced successfully!")
注意事项：
- 作为对您所做工作的扩展，您可能希望存储响应的缓存标头，例如
ETag 或 Last-Modified。这使您可以在下载数据之前询问第三方服务器数据是否发生了变化。
- 我已将您的
if (condition) { /* do lots of work */ } else { /* do small amount of work to handle error */ } 替换为if (!condition) { /* do small amount of work to handle error */ return; } /* do lots of work */。这被称为“快速失败”，用于防止大型和/或嵌套的 if-else 树，同时还在导致错误的原因旁边显示您的错误处理。
- 如果任何一个批次失败，那么其他批次可能无法成功写入数据库，但这个错误也存在于您的原始代码中。您可以将最后几行更改为以下内容，以使它们不会杀死其他批次：
// commit all batches in parallel and wait for them to finish
const results = await Promise.all(batches.map(
(b) => b.commit().then(
() => ({success: true}),
(error) => ({success: false, error})
)
))
let succeeded = 0, failed = 0
results.forEach(result => result.success ? succeeded++ : failed++)
if (failed > 0) {
console.log(`Synced ${succeeded}/${results.length} batches of players successfully!`)
return
}
console.log("Synced all players successfully!")
您也不是第一个遇到批量写入数据库的问题的人。有一个 MultiBatch 实用程序类为您处理批处理是很常见的。
const response = await fetch(url)
if (!response.ok || response.status === 204) {
// A 204 code will break response.json() with a parsing error
// You might want to check for the 429 status here
throw new Error(`Unexpected status code ${response.status}!`)
}
const json = await response.json() // note: empty bodies will throw a parsing error
if (!json.hasOwnProperty("data")) {
throw new Error(`Unexpected response body!`, json)
}
const teams = json["data"]
const multiBatch = new MultiBatch(database)
const playersColRef = database.collection("players")
teams.forEach((team) => {
team.squad.data // the array of players in this team
.forEach(player => {
const reference = playersColRef.doc(`${player.player_id}`)
multiBatch.set(reference, player, { merge: true })
})
})
await multiBatch.commit(/* pass true here to suppress errors */)
console.log("Synced successfully!")