The Mac you build on cannot reproduce these bugs
The morning after a launch, a user wrote to say the app would not open. Just "the application cannot be opened" — no explanation. He had uninstalled, restarted, reinstalled, and got the same thing every time. He had already checked whether it was a security warning, and it wasn't. He tried his second Mac. Same.
Two machines, two different Apple Silicon chips, same result.
On my machine it opened fine. codesign --verify --deep --strict passed. Notarization was accepted. Every tool I knew how to run said the build was correct, because on my machine it was correct.
The difference was one line of entitlements.
One entitlement, and who is allowed to have it
The build had picked up keychain-access-groups. That entitlement — and the whole com.apple.developer.* family — has to be granted by a provisioning profile. An App Store build gets one attached. A Developer ID build for direct download does not embed one at all.
So the app asked for something nothing on the user's machine could authorise, and macOS refused to launch it. No Gatekeeper dialog, because it is not a Gatekeeper decision. No crash report, because it never started. Reinstalling changes nothing, because the artifact itself is what is wrong.
Removing that one key fixed it. The fix shipped the same day.
Why it worked on my machine
A developer machine already has the team's provisioning profiles installed. Xcode puts them there. The entitlement validates, the app launches, and nothing looks wrong.
The dividing line is not "clean install versus upgrade" — the user had been running an earlier version for weeks. It is not the hardware, or the OS version. It is your machine versus everyone else's.
That is a nasty property for a bug to have. You cannot test your way out of it on the machine in front of you, and neither can the handful of people who have been testing your app since before the problem existed, because they are the same population.
The second one, which nobody reported at all
Months later, during release prep, I ran a command by hand:
spctl --assess --type open --context context:primary-signature -v MyApp.dmg
# rejected
# source=Unnotarized Developer IDThe app inside the disk image had been notarized and stapled correctly. The disk image never had been. Gatekeeper inspects the disk image, because the disk image is what the user opens first. So every download had been blocked — for months, across shipped releases.
Nobody told me. Not one email.
Why that one does not reproduce locally either
Gatekeeper only evaluates files carrying the com.apple.quarantine extended attribute. A browser sets it on a download. Your build system does not set it on the artifact it just produced. The disk image sitting in your build/ directory is not quarantined, so opening it locally proves nothing: you are testing a file Gatekeeper has decided to ignore.
This one you can fake, at least:
xattr -w com.apple.quarantine "0081;00000000;Safari;" MyApp.dmg
open MyApp.dmgA detail that will waste an afternoon
Once you fix it, you will want to check the fix. Do not use --type execute on a disk image. It answers rejected for a correct image too:
broken rejected source=Unnotarized Developer ID
correct rejected (the code is valid but does not seem to be an app)Same verdict, different parenthetical. Use the --type open form above and read the exit status.
And the order matters. Notarize the app, staple it, then build the disk image; notarize the image, staple that too. Stapling rewrites the file — in one measured case the disk image grew by 2257 bytes — so anything computed from its bytes, such as a Sparkle appcast's length and EdDSA signature, has to be generated after stapling. Get that backwards and you have fixed one silent failure by creating another.
The shape
Neither of these failed a build. The archive succeeded, the upload succeeded, CI stayed green. One was reported by a user who happened to be persistent enough to try a second machine. The other was never reported by anyone.
That combination is the whole problem. The population that can observe the bug is exactly the population you are trying to reach — people installing for the first time, on machines that have never been set up for development — and that population does not file reports. They delete the download.
Once you see the shape you find the rest of the family:
The update nobody is offered. Sparkle compares CFBundleVersion, not the marketing version. Bump 2.4.0 to 2.4.1 and leave the build number alone, and every existing user is told they are up to date. Your own machine, running a build straight out of Xcode, never asks.
The appcast that describes a different build. Generate it from your build settings rather than from the artifact you produced, and the feed advertises a version that does not match the binary behind it. Whichever branch you had checked out decides what the world is told.
The crash reports you cannot read. Ship without keeping the matching dSYM and every crash from that version arrives as raw addresses, for as long as the version is live. This is the worst of them, because it is the only one you cannot fix afterwards: rebuild and the UUIDs no longer match. That release is permanently unreadable.
Every one of them is invisible from the machine that produced it.
You do not need a clean Mac
Here is the part that took me embarrassingly long to notice.
I went looking for a machine that had never had the app installed. That was the wrong instinct, because the evidence was inside the artifact the whole time.
For the entitlement failure, three facts settle it, and all three are readable from the bundle:
signed for direct distribution (Developer ID)
AND an entitlement that only a provisioning profile can grant
AND no Contents/embedded.provisionprofile
→ this will not launch on a machine without your team's profileNo clean Mac. No second machine. No user report. Just codesign -d --entitlements and a file existence test.
The same is true of the others. Whether the disk image is stapled, whether the build number moved, whether the appcast agrees with the bytes on disk, whether a dSYM exists whose UUID matches the binary — all of it is in the artifact you are about to publish.
So I wrote the checker
Not a build system; there are plenty. Something that reads the artifact and tells you what a user will experience:
$ mac-release-verify MyApp-3.2.1.dmg
target MyApp-3.2.1.dmg
app MyApp.app 3.2.1 (214)
channel direct distribution (Developer ID)
PASS code signature
PASS Gatekeeper assessment (app)
PASS hardened runtime and secure timestamp
FAIL entitlements are backed by a provisioning profile
The bundle asks for entitlements that only a provisioning profile can
grant (keychain-access-groups) but carries no
Contents/embedded.provisionprofile. macOS refuses to launch it on any
machine that does not already have your team's profile installed —
which is every machine except a developer's. It opens normally on the
machine that built it, so this cannot be reproduced locally.
PASS notarization ticket (app)
FAIL notarization ticket (disk image)
The app is stapled but the disk image is not. Gatekeeper checks the
disk image when the user opens the download, so this blocks every
download even though the app inside is fine.
FAIL Gatekeeper assessment (disk image)
Gatekeeper rejects the disk image itself: rejected,
source=Unnotarized Developer ID.
...
3 failed, 9 passed, 1 warning(s), 3 skippedThat is the real output, run against the build that would not open — the same disk image that shipped. Only the app's name and version numbers are changed. Both failures were in it, and the second one had already been there for months.
It only reads. It never signs, notarizes, uploads, publishes, tags or modifies anything, it never asks for credentials, and it is one shell script using tools that already ship with macOS — so you can read all of it before pointing it at a signed release.
Every check in it is a mistake that actually shipped. It is MIT licensed: github.com/cyber937/mac-release-verify
brew install cyber937/tap/mac-release-verifyIf you have a failure mode that belongs in there, open an issue and describe what shipped and what the user saw. Especially if it is one your own Mac would never have shown you.