Core Data + CloudKit app using NSPersistentCloudKitContainer only syncing at launch and not during execution

June 21, 2020

Hello visitor from Google, I hope you are well, but most likely you are not, because you are fighting with syncing issues.

I posted this on Stack Overflow earlier today, but it actually also makes sense to have it here on my blog.

In a new app I’m working on, I had syncing issues (not upon launch, when it synced fine, but only during the app being active but idle) in a project that uses Core Data + CloudKit with NSPersistentCloudKitContainer.

My app was built using Xcode’s 11 Master-Detail template with Core Data + CloudKit from the start, so I had to do very little to have syncing work initially:

1. Enable Remote Notifications Background Mode in Signing & Capabilities for my target;
2. Add the iCloud capability for CloudKit;
3. Select the container iCloud.com.domain.AppName
4. Add viewContext.automaticallyMergesChangesFromParent = true

Basically, I followed Getting Started With NSPersistentCloudKitContainer by Andrew Bancroft and this was enough to have the MVP sync between devices (Catalina, iOS 13, iPadOS 13) not only upon launch, but also when the app was running and active (thanks to step 4 above) and another device edited/added/deleted an object.
Being the Xcode template, it did not have the additional customisations / advanced behaviours of WWDC 2019’s sample project, but it actually accomplished the goal pretty well and I was satisfied, so I moved on to other parts of this app’s development and stopped thinking about sync.

A few days ago, I noticed that the iOS/iPadOS app was now only syncing upon launch, and not while the app was active and idle on screen; on macOS the behaviour was slightly different, because a simple command-tab triggered sync when reactivating the app, but again, if the Mac app was frontmost, no syncing for changes coming from other devices.

I initially blamed a couple of modifications I did in the meantime:

• In order to have the sqlite accessible in a Share Extension, I moved the container in an app group by subclassing NSPersistentCloudKitContainer;
• I changed the capitalisation in the name of the app and, since I could not delete the CloudKit database, I created a new container named iCloud.com.domain.AppnameApp (CloudKit is case insensitive, apparently, and yes, I should really start to care less about such things).

While I was pretty sure that I saw syncing work as well as before after each one of these changes, having sync (apparently) suddenly break convinced me, for at least a few hours, that either one of those modification from the default path caused the notifications to stop being received while the app was active, and that then the merge would only happen upon launch as I was seeing because the running app was not made aware of changes.

I should mention, because this could help others in my situation, that I was sure notifications were triggered upon Core Data context saves because CloudKit Dashboard was showing the notifications being sent:

So, I tried a few times clearing Derived Data (one never knows), deleting the apps on all devices and resetting the Development Environment in CloudKit’s Dashboard (something I already did periodically during development), but I still had the issue of the notifications not being received.

Finally, I realised that resetting the CloudKit environment and deleting the apps was indeed useful (and I actually rebooted everything just to be safe ;) but I also needed to delete the app data from iCloud (from iCloud’s Settings screen on the last iOS device where the app was still installed, after deleting from the others) if I really wanted a clean slate; otherwise, my somewhat messed up database would sync back to the newly installed app.

And indeed, a truly clean slate with a fresh Development Environment, newly installed apps and rebooted devices resumed the notifications being detected from the devices also when the apps are frontmost.
So, if you feel your setup is correct and have already read enough times that viewContext.automaticallyMergesChangesFromParent = true is the only thing you need, but still can’t see changes come from other devices, don’t exclude that something could have been messed up beyond your control (don’t get me wrong: I’m 100% sure that it must have been something that I did!) and try to have a fresh start… it might seem obscure, but what isn’t with the syncing method we are choosing for our app?

---

2020.12.02 - Update for Users who set NSPersistentStoreDescription to interact with public databases and are noticing delays in the updates.

A few days ago, Ben Radler brought to my attention a behavior that those who use public databases with CloudKit (I was using the private database in the original post) need to take into account when testing:

If you are setting the NSPersistentStoreDescription to interact with the public database rather than the private database (previous default before iOS 14 I believe) via description.cloudKitContainerOptions?.databaseScope = .public, it will only poll for changes from cloudkit every 30 minutes by design (see 10:30 into the WWDC 2020 session video on this topic).

I want to thank Ben for pointing this out, since it might also be a cause for head-scratching when testing CloudKit sync!

Core Data + cloud sync = dilemma

July 11, 2017

My friend Becky recently wrote a post about the dilemma she’s facing with a new, interesting app she’s making.

She likes Core Data, and would like to use it for her project; she also wants to add sync capabilities, as that’s a requirement for most modern apps.

Here lies her dilemma. And my dilemma. And many developers’ dilemma… There’s no clear path towards a Core Data app with cloud sync.

Becky doesn’t want to use Core Data + iCloud since it’s deprecated. I might add, I don’t want a friend to use it, because I value her sanity… when I was setting up Tasktic’s sync mechanism, I spent 4 months and 3 complete rewrites before realizing my code wasn’t the problem: it was the actual API that was an almost un-debuggable black box that sometimes, very rarely but still too often, lost an object during the sync process with apparently no reason, and no way to get it back.

A few weeks ago, I suggested Ensembles to Becky, since it was the solution I adopted for Tasktic and given how well it behaved for me, restoring my sanity after those awful 4 months. But the free version of Ensembles still uses Core Data + iCloud under the hood***, so it’s not very future proof given the deprecation mentioned above, and on top of that there’s a more modern, faster solution provided by Apple that everybody want to use: CloudKit.

CloudKit is very tempting because everyone has experienced how fast and reliable it is with Notes.app and Photos.app for macOS and iOS; the most important thing for an app that sync is to never, ever lose users’ data, and CloudKit passes that test with flying colors. It is also very modern, with private/public data, web support via CloudKitJS, and most important, it is what Apple has chosen for the future, and following Apple’s lead is always a good idea.

The thing is, in order to get CloudKit to play nice with Core Data, you have to write most of the sync logic yourself, converting NSManagedObjects to CKRecords, handling updates, reacting to duplication, etc. It can be done, and many developers do it “by hand”.

As Becky notes, it can also be done via libraries like Seam3, which is currently the best open source implementation I found (and I looked really long and hard) of a Core Data - CloudKit bridge, albeit only if you don’t have many-to-many relationships in your schema…

The fact is, I really don’t want a dependency for my sync code anymore, especially when starting a new project. If I were willing to accept the risk of my sync code being abandoned someday in the future, I might as well look into Realm (not that such a loved mobile stack is going anywhere, but as we’ve seen in the past, companies get acquired, or sometimes move on to different projects…).

Where does this leave us? To Apple, of course.

Apple made such an amazing object graph and persistence framework with Core Data, and a fantastic syncing backend with CloudKit. They never connected those dots officially, though.

Becky’s post reminded me of something I really wanted (needed) for a really long time: an Apple sample project showing their idea of the best, most modern and Swifty implementation of Core Data for local persistence + CloudKit for sync implementation.

This is a step they really should take: while it’s great that they provide sample projects for ARKit, data persistence and sync is top priority for a lot of developers, and it would only be appropriate for Apple to show how they think a “great” implementation should look like.

So, if you have a friend who works at the best fruit company in the world, pass along the message: the indie developer community, and especially us beginners, would love some help in this area!

** This is true only if you want to stick with native cloud solutions, like I prefer, otherwise you can also pair Ensembles with your own backend or use Dropbox: the great thing about Ensembles is that it’s actually backend agnostic, but for CloudKit you need Ensembles 2, which isn’t free and so it’s not an option for most indie developers like myself.