有哪些可靠的机制可以防止 CoreData CloudKit 中的数据重复？答案

【问题标题】：What are some reliable mechanism to prevent data duplication in CoreData CloudKit?有哪些可靠的机制可以防止 CoreData CloudKit 中的数据重复？
【发布时间】：2022-06-11 02:19:02
【问题描述】：

我们的每个数据行都包含一个唯一的uuid 列。

以前，在采用 CloudKit 之前，uuid 列具有唯一约束。这使我们能够防止数据重复。

现在，我们开始将 CloudKit 集成到我们现有的 CoreData 中。这样的唯一约束被删除。以下用户流，会造成数据重复。

使用CloudKit时导致数据重复的步骤

首次启动应用。
由于有空数据，所以生成一个预定义uuid的预定义数据。
预定义数据同步到 iCloud。
应用已卸载。
应用程序已重新安装。
首次启动应用。
由于有空数据，所以会生成带有预定义uuid的预定义数据。
第 3 步中以前的旧预定义数据已同步到设备。
我们现在有 2 个具有相同 uuid 的预定义数据！ :(

我想知道，我们有没有办法防止这种重复？

在第 8 步中，我们希望有一种方法可以在写入 CoreData 之前执行此类逻辑

检查CoreData中是否存在这样的uuid。如果没有，请写入 CoreData。如果没有，我们将选择具有最新更新日期的那个，然后覆盖现有数据。

我曾经尝试将上述逻辑插入 https://developer.apple.com/documentation/coredata/nsmanagedobject/1506209-willsave 。为了防止保存，我使用self.managedObjectContext?.rollback()。但它只是崩溃。

您有什么想法，我可以使用哪些可靠的机制来防止 CoreData CloudKit 中的数据重复？

附加信息：

在采用 CloudKit 之前

我们正在使用以下 CoreData 堆栈

class CoreDataStack {
    static let INSTANCE = CoreDataStack()
    
    private init() {
    }
    
    private(set) lazy var persistentContainer: NSPersistentContainer = {
        precondition(Thread.isMainThread)
        
        let container = NSPersistentContainer(name: "xxx", managedObjectModel: NSManagedObjectModel.wenote)
        
        container.loadPersistentStores(completionHandler: { (storeDescription, error) in
            if let error = error as NSError? {
                // This is a serious fatal error. We will just simply terminate the app, rather than using error_log.
                fatalError("Unresolved error \(error), \(error.userInfo)")
            }
        })
        
        // So that when backgroundContext write to persistent store, container.viewContext will retrieve update from
        // persistent store.
        container.viewContext.automaticallyMergesChangesFromParent = true
        
        // TODO: Not sure these are required...
        //
        //container.viewContext.mergePolicy = NSMergeByPropertyObjectTrumpMergePolicy
        //container.viewContext.undoManager = nil
        //container.viewContext.shouldDeleteInaccessibleFaults = true
        
        return container
    }()

我们的 CoreData 数据架构有

唯一约束。
拒绝删除关系规则。
非空字段没有默认值。

采用 CloudKit 后

class CoreDataStack {
    static let INSTANCE = CoreDataStack()
    
    private init() {
    }
    
    private(set) lazy var persistentContainer: NSPersistentContainer = {
        precondition(Thread.isMainThread)
        
        let container = NSPersistentCloudKitContainer(name: "xxx", managedObjectModel: NSManagedObjectModel.wenote)
        
        container.loadPersistentStores(completionHandler: { (storeDescription, error) in
            if let error = error as NSError? {
                // This is a serious fatal error. We will just simply terminate the app, rather than using error_log.
                fatalError("Unresolved error \(error), \(error.userInfo)")
            }
        })
        
        // So that when backgroundContext write to persistent store, container.viewContext will retrieve update from
        // persistent store.
        container.viewContext.automaticallyMergesChangesFromParent = true
        
        // TODO: Not sure these are required...
        //
        //container.viewContext.mergePolicy = NSMergeByPropertyObjectTrumpMergePolicy
        //container.viewContext.undoManager = nil
        //container.viewContext.shouldDeleteInaccessibleFaults = true
        
        return container
    }()

我们将 CoreData 数据架构更改为

没有唯一约束。
取消关系的删除规则。
具有非空字段的默认值。

根据来自https://developer.apple.com/forums/thread/699634?login=true 的开发者技术支持工程师的反馈，他提到我们可以

通过使用存储持久历史记录检测相关更改
删除重复数据

但是，它应该如何实现并不完全清楚，因为提供的 github 链接已损坏。

【问题讨论】：

你在使用NSPersistentCloudKitContainer吗？
是的。对不起。请允许我通过更新我的问题来提供更多技术细节。

标签： ios swift core-data cloudkit

【解决方案1】：

与 CloudKit 集成后，没有唯一约束功能。

解决此限制的方法是

一旦 CloudKit 插入后检测到重复，我们将执行重复数据删除。

此解决方法的挑战性部分是，我们如何在 CloudKit 执行插入时收到通知？

这里是关于如何在 CloudKit 执行插入时得到通知的分步说明。

在 CoreData 中开启 NSPersistentHistoryTrackingKey 功能。
在 CoreData 中开启 NSPersistentStoreRemoteChangeNotificationPostOptionKey 功能。
设置viewContext.transactionAuthor = "app"。这是一个重要的步骤，因此当我们查询事务历史记录时，我们可以知道哪个 DB 事务是由我们的应用发起的，以及哪个 DB 事务是由 CloudKit 发起的。
每当我们通过NSPersistentStoreRemoteChangeNotificationPostOptionKey功能自动通知我们时，我们将开始查询交易历史。查询将根据交易作者和最后查询令牌进行过滤。详情请参考代码示例。
一旦我们检测到事务是insert，并且它在我们关注的entity上操作，我们将开始执行基于关注entity的重复数据删除

代码示例

import CoreData

class CoreDataStack: CoreDataStackable {
    let appTransactionAuthorName = "app"
    
    /**
     The file URL for persisting the persistent history token.
    */
    private lazy var tokenFile: URL = {
        return UserDataDirectory.token.url.appendingPathComponent("token.data", isDirectory: false)
    }()
    
    /**
     Track the last history token processed for a store, and write its value to file.
     
     The historyQueue reads the token when executing operations, and updates it after processing is complete.
     */
    private var lastHistoryToken: NSPersistentHistoryToken? = nil {
        didSet {
            guard let token = lastHistoryToken,
                let data = try? NSKeyedArchiver.archivedData( withRootObject: token, requiringSecureCoding: true) else { return }
            
            if !UserDataDirectory.token.url.createCompleteDirectoryHierarchyIfDoesNotExist() {
                return
            }
            
            do {
                try data.write(to: tokenFile)
            } catch {
                error_log(error)
            }
        }
    }
    
    /**
     An operation queue for handling history processing tasks: watching changes, deduplicating tags, and triggering UI updates if needed.
     */
    private lazy var historyQueue: OperationQueue = {
        let queue = OperationQueue()
        queue.maxConcurrentOperationCount = 1
        return queue
    }()
    
    var viewContext: NSManagedObjectContext {
        persistentContainer.viewContext
    }
    
    static let INSTANCE = CoreDataStack()
    
    private init() {
        // Load the last token from the token file.
        if let tokenData = try? Data(contentsOf: tokenFile) {
            do {
                lastHistoryToken = try NSKeyedUnarchiver.unarchivedObject(ofClass: NSPersistentHistoryToken.self, from: tokenData)
            } catch {
                error_log(error)
            }
        }
    }
    
    deinit {
        deinitStoreRemoteChangeNotification()
    }
    
    private(set) lazy var persistentContainer: NSPersistentContainer = {
        precondition(Thread.isMainThread)
        
        let container = NSPersistentCloudKitContainer(name: "xxx", managedObjectModel: NSManagedObjectModel.xxx)
        
        // turn on persistent history tracking
        let description = container.persistentStoreDescriptions.first
        description?.setOption(true as NSNumber, forKey: NSPersistentHistoryTrackingKey)
        description?.setOption(true as NSNumber, forKey: NSPersistentStoreRemoteChangeNotificationPostOptionKey)
        
        container.loadPersistentStores(completionHandler: { (storeDescription, error) in
            if let error = error as NSError? {
                // This is a serious fatal error. We will just simply terminate the app, rather than using error_log.
                fatalError("Unresolved error \(error), \(error.userInfo)")
            }
        })
        
        // Provide transaction author name, so that we can know whether this DB transaction is performed by our app
        // locally, or performed by CloudKit during background sync.
        container.viewContext.transactionAuthor = appTransactionAuthorName
        
        // So that when backgroundContext write to persistent store, container.viewContext will retrieve update from
        // persistent store.
        container.viewContext.automaticallyMergesChangesFromParent = true
        
        // TODO: Not sure these are required...
        //
        //container.viewContext.mergePolicy = NSMergeByPropertyObjectTrumpMergePolicy
        //container.viewContext.undoManager = nil
        //container.viewContext.shouldDeleteInaccessibleFaults = true
        
        // Observe Core Data remote change notifications.
        initStoreRemoteChangeNotification(container)
        
        return container
    }()
    
    private(set) lazy var backgroundContext: NSManagedObjectContext = {
        precondition(Thread.isMainThread)
        
        let backgroundContext = persistentContainer.newBackgroundContext()

        // Provide transaction author name, so that we can know whether this DB transaction is performed by our app
        // locally, or performed by CloudKit during background sync.
        backgroundContext.transactionAuthor = appTransactionAuthorName
        
        // Similar behavior as Android's Room OnConflictStrategy.REPLACE
        // Old data will be overwritten by new data if index conflicts happen.
        backgroundContext.mergePolicy = NSMergeByPropertyObjectTrumpMergePolicy
        
        // TODO: Not sure these are required...
        //backgroundContext.undoManager = nil
        
        return backgroundContext
    }()
    
    private func initStoreRemoteChangeNotification(_ container: NSPersistentContainer) {
        // Observe Core Data remote change notifications.
        NotificationCenter.default.addObserver(
            self,
            selector: #selector(storeRemoteChange(_:)),
            name: .NSPersistentStoreRemoteChange,
            object: container.persistentStoreCoordinator
        )
    }
    
    private func deinitStoreRemoteChangeNotification() {
        NotificationCenter.default.removeObserver(self)
    }
    
    @objc func storeRemoteChange(_ notification: Notification) {
        // Process persistent history to merge changes from other coordinators.
        historyQueue.addOperation {
            self.processPersistentHistory()
        }
    }
    
    /**
     Process persistent history, posting any relevant transactions to the current view.
     */
    private func processPersistentHistory() {
        backgroundContext.performAndWait {
            
            // Fetch history received from outside the app since the last token
            let historyFetchRequest = NSPersistentHistoryTransaction.fetchRequest!
            historyFetchRequest.predicate = NSPredicate(format: "author != %@", appTransactionAuthorName)
            let request = NSPersistentHistoryChangeRequest.fetchHistory(after: lastHistoryToken)
            request.fetchRequest = historyFetchRequest

            let result = (try? backgroundContext.execute(request)) as? NSPersistentHistoryResult
            guard let transactions = result?.result as? [NSPersistentHistoryTransaction] else { return }

            if transactions.isEmpty {
                return
            }
            
            for transaction in transactions {
                if let changes = transaction.changes {
                    for change in changes {
                        let entity = change.changedObjectID.entity.name
                        let changeType = change.changeType
                        let objectID = change.changedObjectID
                        
                        if entity == "NSTabInfo" && changeType == .insert {
                            deduplicateNSTabInfo(objectID)
                        }
                    }
                }
            }
            
            // Update the history token using the last transaction.
            lastHistoryToken = transactions.last!.token
        }
    }
    
    private func deduplicateNSTabInfo(_ objectID: NSManagedObjectID) {
        do {
            guard let nsTabInfo = try backgroundContext.existingObject(with: objectID) as? NSTabInfo else { return }
            
            let uuid = nsTabInfo.uuid
            
            guard let nsTabInfos = NSTabInfoRepository.INSTANCE.getNSTabInfosInBackground(uuid) else { return }
            
            if nsTabInfos.isEmpty {
                return
            }
            
            var bestNSTabInfo: NSTabInfo? = nil
            
            for nsTabInfo in nsTabInfos {
                if let _bestNSTabInfo = bestNSTabInfo {
                    if nsTabInfo.syncedTimestamp > _bestNSTabInfo.syncedTimestamp {
                        bestNSTabInfo = nsTabInfo
                    }
                } else {
                    bestNSTabInfo = nsTabInfo
                }
            }
            
            for nsTabInfo in nsTabInfos {
                if nsTabInfo === bestNSTabInfo {
                    continue
                }
                
                // Remove old duplicated data!
                backgroundContext.delete(nsTabInfo)
            }
            
            RepositoryUtils.saveContextIfPossible(backgroundContext)
        } catch {
            error_log(error)
        }
    }
}

参考

https://developer.apple.com/documentation/coredata/synchronizing_a_local_store_to_the_cloud - 在示例代码中，文件CoreDataStack.swift 说明了一个类似的示例，说明如何在云同步后删除重复数据。
https://developer.apple.com/documentation/coredata/consuming_relevant_store_changes - 交易历史信息。
What's the best approach to prefill Core Data store when using NSPersistentCloudKitContainer? - 一个类似的问题

【讨论】：