Mistakes were made, let's learn from them
March 15, 2021
GlanceCam 3 launched last week and, for such a niche product, I couldn’t be happier:
- Lots of Users reached out via email to let me know they’ve been waiting for multi-windows for a long time and loved the implementation;
- No dangerous bug or crash ruined the experience;
- Sales spiked more than I have expected (the ratio among lifetime unlocks and yearly subscriptions also surprised me: it’s currently 4 to 1); of course the rush is now almost over, but still, it was nice while it lasted;
- Cherry on top: iMore wrote about it!
So, I’ve decided to celebrate by reflecting on my mistakes in the 2 years that took me to launch GlanceCam 3, or I should say the mistakes I made causing me to take 2 years to ship it.
I’ll try to keep this of reasonable length and actionable to other indie developers:
- Lack of focus: I started working on multi-windows in March 2019 and since then I’ve shipped 5 medium-size updates to GlanceCam, launched 3 small new apps (PhotosUpload, Link HUB and ClipBar), updated Walk More and Tame Time, and spent 5 months on an unfinished Catalyst project I still very much hope to complete some day (it’s almost there…).
With a day job (I work in a completely unrelated field to programming) I can’t do that: if I’m lucky (and able to actually get up at 4.30 a.m.) I have 2.5 hours per morning during the week and a little more time on Saturdays and Sundays to program.
I need to pick my battles carefully and I can’t give myself the illusion of being productive because I work on too many things at once.
- You guessed it, as it goes hand in hand with the first mistake: lack of motivation. If you don’t make progress on something for a long time, you start procrastinating… maybe even launch 3 different apps instead of working on the only one that generates a little revenue.
When you planned over and over to work on something and don’t make tangible progress, it’s a very dangerous place for that project to be in.
Things need to keep moving or they die.
I feel lucky to have been able to finally find the willpower to really put my head down and work for almost 300 hours on GlanceCam this year (since January, outside of my day job I haven’t done anything but coding, talking walks with Milla, eat, work and do very stupid things to my sleep-cycle), but I realise it was an unreasonable and unbalanced way to get both my and my app’s mojos back.
- Branching proliferation: I wanted a big 3.0 with multi-windows, new Preferences and much more, so my idea was to keep updating GlanceCam while working on the other features. Then, when I got stuck on difficult problems connected to multi-windows development, I started from scratch again and again (branches: multiwindows_take2, _take3 and _take4, which eventually shipped) while also concurrently working on Preferences and even custom one-off features I sent to some Users who asked for tweaks. Merging was a nightmare, because in the meantime the shipping app progressed.
If time is constrained, I must do ONE thing, finish it, and then move to the next one.
- Aiming for the stars: a big 3.0 was a reasonable desire, and indeed I don’t think some press would have mentioned a camera viewer if it was a smaller update, but before this long release-cycle I was always smart enough to build small updates and ship them in a timely fashion.
I’m returning to my better habits already, as GlanceCam 3.1 has already been submitted for review, but finding balance between frequent releases and their depth is very important, and probably will continue to be a struggle in the future, for instance when I’ll start thinking of GlanceCam 4 (or maybe of the much requested iOS version that’s more of a new app altogether… see, I am learning: I can foresee that, while promising in terms of market size, that’s also dangerous for focus and branching proliferation).
- Managing project-related informations: during the weekend I asked other developers about their tools for keeping track of their work, feature requests, bugs and ideas and got very good answers. In that thread I’ve also shared where my journey on this ended (not far from where it started, OmniFocus, but with better structure).
Working on different things at the same time, in different directions and on multiple apps, I’ve let my to-do lists slip a bit, then more than a bit, until they got stale and their periodical review became painful and stopped happening.
I know better than that: my mind works best if I have a single stop between home and work, a single and up-to-date bucket for everything that’s related on planning and developing a project.
I sure have made plenty of other mistakes (letting a single feature request derail my plans, allowing a review to mess with my mood, starting work on something before doing the appropriate research and due diligence, leaving marketing to the last few days, even building a whole new website the day before launch…), but this is probably enough introspection for me to go through in one sitting, and for you to read… thank you for coming along, hopefully there was some lesson for you to extract from my mistakes!
Core Data + CloudKit app using NSPersistentCloudKitContainer only syncing at launch and not during execution
June 21, 2020
Hello visitor from Google, I hope you are well, but most likely you are not, because you are fighting with syncing issues.
I posted this on Stack Overflow earlier today, but it actually also makes sense to have it here on my blog.
In a new app I’m working on, I had syncing issues (not upon launch, when it synced fine, but only during the app being active but idle) in a project that uses Core Data + CloudKit with
My app was built using Xcode’s 11 Master-Detail template with Core Data + CloudKit from the start, so I had to do very little to have syncing work initially:
- Enable Remote Notifications Background Mode in Signing & Capabilities for my target;
- Add the iCloud capability for CloudKit;
- Select the container iCloud.com.domain.AppName
viewContext.automaticallyMergesChangesFromParent = true
Basically, I followed Getting Started With NSPersistentCloudKitContainer by Andrew Bancroft and this was enough to have the MVP sync between devices (Catalina, iOS 13, iPadOS 13) not only upon launch, but also when the app was running and active (thanks to step 4 above) and another device edited/added/deleted an object.
Being the Xcode template, it did not have the additional customisations / advanced behaviours of WWDC 2019’s sample project, but it actually accomplished the goal pretty well and I was satisfied, so I moved on to other parts of this app’s development and stopped thinking about sync.
A few days ago, I noticed that the iOS/iPadOS app was now only syncing upon launch, and not while the app was active and idle on screen; on macOS the behaviour was slightly different, because a simple command-tab triggered sync when reactivating the app, but again, if the Mac app was frontmost, no syncing for changes coming from other devices.
I initially blamed a couple of modifications I did in the meantime:
- In order to have the sqlite accessible in a Share Extension, I moved the container in an app group by subclassing NSPersistentCloudKitContainer;
- I changed the capitalisation in the name of the app and, since I could not delete the CloudKit database, I created a new container named iCloud.com.domain.AppnameApp (CloudKit is case insensitive, apparently, and yes, I should really start to care less about such things).
While I was pretty sure that I saw syncing work as well as before after each one of these changes, having sync (apparently) suddenly break convinced me, for at least a few hours, that either one of those modification from the default path caused the notifications to stop being received while the app was active, and that then the merge would only happen upon launch as I was seeing because the running app was not made aware of changes.
I should mention, because this could help others in my situation, that I was sure notifications were triggered upon Core Data context saves because CloudKit Dashboard was showing the notifications being sent:
So, I tried a few times clearing Derived Data (one never knows), deleting the apps on all devices and resetting the Development Environment in CloudKit’s Dashboard (something I already did periodically during development), but I still had the issue of the notifications not being received.
Finally, I realised that resetting the CloudKit environment and deleting the apps was indeed useful (and I actually rebooted everything just to be safe ;) but I also needed to delete the app data from iCloud (from iCloud’s Settings screen on the last iOS device where the app was still installed, after deleting from the others) if I really wanted a clean slate; otherwise, my somewhat messed up database would sync back to the newly installed app.
And indeed, a truly clean slate with a fresh Development Environment, newly installed apps and rebooted devices resumed the notifications being detected from the devices also when the apps are frontmost.
So, if you feel your setup is correct and have already read enough times that
viewContext.automaticallyMergesChangesFromParent = true is the only thing you need, but still can’t see changes come from other devices, don’t exclude that something could have been messed up beyond your control (don’t get me wrong: I’m 100% sure that it must have been something that I did!) and try to have a fresh start… it might seem obscure, but what isn’t with the syncing method we are choosing for our app?
2020.12.02 - Update for Users who set
NSPersistentStoreDescription to interact with public databases and are noticing delays in the updates.
A few days ago, Ben Radler brought to my attention a behavior that those who use public databases with CloudKit (I was using the private database in the original post) need to take into account when testing:
If you are setting the
NSPersistentStoreDescriptionto interact with the public database rather than the private database (previous default before iOS 14 I believe) via
description.cloudKitContainerOptions?.databaseScope = .public, it will only poll for changes from cloudkit every 30 minutes by design (see 10:30 into the WWDC 2020 session video on this topic).
I want to thank Ben for pointing this out, since it might also be a cause for head-scratching when testing CloudKit sync!
How to symbolicate macOS apps crash logs
April 6, 2018
Alternate title: TIL Xcode does not symbolicate Mac applications .crash files, that only works for iOS apps.
I made a pretty obscure mistake in GlanceCam 1.3 that lead to a crash during its app review.
I’m very sorry for wasting app review some time (they have been amazing, both understanding and allowing an hedge-case for sandboxing and giving GlanceCam lighting fast reviews, usually less than a day!) , but they have been so kind to attach a crash report and I had to symbolicate it (“Symbolication replaces memory addresses with human-readable function names and line numbers”).
Having done that a few times in the past directly in Xcode, I went straight there, but as I mentioned above, no luck. There’s plenty of resources online on how to do that “by hand”, but I want to point out a couple, very useful, solutions for other fellow developers:
Bob Matcuk’s gist perfectly describes all the steps to manually symbolicate a crash report, including checking the build number; this involves using the atos command in Terminal, which is not bad at all, but it can get a bit repetitive if you want to check more addresses (lines). Tomaz’s Symbolicator is a Swift Xcode project that compiled right out the box (sometimes, with Swift the language fast-paced evolution makes you work a bit before a project from the Internet actually builds) generating a very easy to use command line application that automatically fetches the DSYM files from the Xcode archives and overwrites the crash report with functions and line numbers that a simple human being like myself can actually understand.
Core Data + cloud sync = dilemma
July 11, 2017
My friend Becky recently wrote a post about the dilemma she’s facing with a new, interesting app she’s making.
She likes Core Data, and would like to use it for her project; she also wants to add sync capabilities, as that’s a requirement for most modern apps.
Here lies her dilemma. And my dilemma. And many developers’ dilemma… There’s no clear path towards a Core Data app with cloud sync.
Becky doesn’t want to use Core Data + iCloud since it’s deprecated. I might add, I don’t want a friend to use it, because I value her sanity… when I was setting up Tasktic’s sync mechanism, I spent 4 months and 3 complete rewrites before realizing my code wasn’t the problem: it was the actual API that was an almost un-debuggable black box that sometimes, very rarely but still too often, lost an object during the sync process with apparently no reason, and no way to get it back.
A few weeks ago, I suggested Ensembles to Becky, since it was the solution I adopted for Tasktic and given how well it behaved for me, restoring my sanity after those awful 4 months. But the free version of Ensembles still uses Core Data + iCloud under the hood***, so it’s not very future proof given the deprecation mentioned above, and on top of that there’s a more modern, faster solution provided by Apple that everybody want to use: CloudKit.
CloudKit is very tempting because everyone has experienced how fast and reliable it is with Notes.app and Photos.app for macOS and iOS; the most important thing for an app that sync is to never, ever lose users’ data, and CloudKit passes that test with flying colors. It is also very modern, with private/public data, web support via CloudKitJS, and most important, it is what Apple has chosen for the future, and following Apple’s lead is always a good idea.
The thing is, in order to get CloudKit to play nice with Core Data, you have to write most of the sync logic yourself, converting NSManagedObjects to CKRecords, handling updates, reacting to duplication, etc. It can be done, and many developers do it “by hand”.
As Becky notes, it can also be done via libraries like Seam3, which is currently the best open source implementation I found (and I looked really long and hard) of a Core Data - CloudKit bridge, albeit only if you don’t have many-to-many relationships in your schema…
The fact is, I really don’t want a dependency for my sync code anymore, especially when starting a new project. If I were willing to accept the risk of my sync code being abandoned someday in the future, I might as well look into Realm (not that such a loved mobile stack is going anywhere, but as we’ve seen in the past, companies get acquired, or sometimes move on to different projects…).
Where does this leave us? To Apple, of course.
Apple made such an amazing object graph and persistence framework with Core Data, and a fantastic syncing backend with CloudKit. They never connected those dots officially, though.
Becky’s post reminded me of something I really wanted (needed) for a really long time: an Apple sample project showing their idea of the best, most modern and Swifty implementation of Core Data for local persistence + CloudKit for sync implementation.
This is a step they really should take: while it’s great that they provide sample projects for ARKit, data persistence and sync is top priority for a lot of developers, and it would only be appropriate for Apple to show how they think a “great” implementation should look like.
So, if you have a friend who works at the best fruit company in the world, pass along the message: the indie developer community, and especially us beginners, would love some help in this area!
** This is true only if you want to stick with native cloud solutions, like I prefer, otherwise you can also pair Ensembles with your own backend or use Dropbox: the great thing about Ensembles is that it’s actually backend agnostic, but for CloudKit you need Ensembles 2, which isn’t free and so it’s not an option for most indie developers like myself.