Notarizing App for macOS Catalina

[kovidgoyal]

Regarding embedded python, as far as I can tell, the following of points should apply:

1) Turn off the use of __pycache__ and instead precompile the bytecode at embedding time. This is because I am pretty sure notarized apps are not allowed to change things inside their .app folders. IIRC gatekeeper will periodically recheck the app folder to enforece this.

2) The hardened runtime for some stupid reason seems to only care about machine code. Loading pure python plugins/extensions should therefore be no problem

3) I think the exceptions for dlopening and execing unsigend code should also allow loading third party python C extensions from outseide the app bundle, but am not sure. It may be that plugins that use native code wont be workable on macOS. This will require experimentation.

[odamizu]

Not sure if this makes much difference in the end ...

https://9to5mac.com/2019/09/03/apple...ization-delay/

Apple today announced that it is [temporarily] relaxing some notarization requirements for Mac apps. For apps outside of the Mac App Store, macOS Catalina will only run notarized software. However, developers have found it difficult to adopt notarization.

To ease the transition, Apple is temporarily relaxing some of the things that the company previously mandated for an app to be notarized.

... However, Apple has only promised to delay the full enforcement until January 2020, a mere four months away.

And here: https://developer.apple.com/news/?id=09032019a

... To make this transition easier and to protect users on macOS Catalina who continue to use older versions of software, we’ve adjusted the notarization prerequisites until January 2020.

You can now notarize Mac software that:

• Doesn’t have the Hardened Runtime capability enabled.
• Has components not signed with your Developer ID.
• Doesn’t include a secure timestamp with your code-signing signature.
• Was built with an older SDK.
• Includes the com.apple.security.get-task-allow entitlement with the value set to any variation of true.

Make sure to submit all versions of your software. While Xcode 10 or later is still required to submit, you don’t need to rebuild or re-sign your software before submission.

[KevinH]

Thanks for the links.

FWIW, I still strongly believe Apple is totally in the wrong with any type of "notarization requirement" and should not be trying to force developers who already sign their code but distribute outside the Mac app store.

I can understand it for apps inside the Mac app store, but not independent apps that have nothing to do with the app store.

To complicate things it seems they only accept dmg, zip, and pkg packaging instead of the .tar.xz Sigil just moved to because of its much better compression.

[KevinH]

When final Catalina is finally released, if you decide to upgrade, please report any successes or failures installing Sigil, Calibre, Python3, etc, as Sigil will not be moving its dev machine away from macOS 10.13.6 for the foreseeable future.

[odamizu]

Once final Catalina is released, I plan to install and boot from an external drive, whose sole purpose will be to test how Sigil, Calibre, Alf, Kindle for Mac, etc. fare on Catalina. I will definitely report back.

I'm still hopeful that, as a last resort, it will be possible to override Catalina's Gatekeeper to allow unnotarized apps to run, even if that means they can't be signed (crazy, I know). The few articles I've seen suggest Apple does not intend to close this avenue.

That said, I plan to keep my actual Mac on High Sierra for as long as possible. The external Catalina drive will just be for experimentation

[KevinH]

It would be interesting to see the results from installing from a .tar.xz (.txz) archive as this seems to be flying below gatekeepers radar at the moment even on High Sierra and Mojave.

[kovidgoyal]

If you dont download the dmg using safari, it wont trigger gatekeeper as safari is what sets the attributes of the dmg file that tell gatekeeper that the app was downloaded form the internet and needs to be checked. And ust FYI on newer versions of macOS you can use UFLO rmat for dmg which greatly improves compression.

[KevinH]

So use this one from the hdiutil man page, right?

ULFO - UDIF lzfse-compressed image (OS X 10.11+ only)

or is there a newer "UFLO" available but not documented in the man page?

Originally Posted by kovidgoyal

If you dont download the dmg using safari, it wont trigger gatekeeper as safari is what sets the attributes of the dmg file that tell gatekeeper that the app was downloaded form the internet and needs to be checked. And ust FYI on newer versions of macOS you can use UFLO rmat for dmg which greatly improves compression.

edit:

Okay using this command line to generate a dmg:

hdiutil create ./Sigil-0.9.18-Mac-Package.dmg -volname "Sigil" -srcfolder ./Sigil.app -format ULFO

gives me

124508032 6 Sep 11:07 Sigil-0.9.18-Mac-Package.dmg


Using .tar.xz (.txz) on the app itself gives me

78071020 3 Sep 11:22 Sigil.app-0.9.18-Mac.txz

So I am saving roughly 40 to 46 meg of download by using tar.xz just on Sigil.app

Am I simply using the wrong command options on hdiutil?

[KevinH]

Ah, that explains why. If I use Safari to download a .tar.xz and then use the unxz command line from a self compiled xzutuils, it creates the .tar archive and deletes the original tar.xz file.

This also seems to remove any extended attribute quarantine flags on the .tar itself. So if it is clean being tarred, it will be clean being untarred.

Very interesting to know. So their entire gatekeeper extra attributes approach is really full of BS.

Originally Posted by kovidgoyal

If you dont download the dmg using safari, it wont trigger gatekeeper as safari is what sets the attributes of the dmg file that tell gatekeeper that the app was downloaded form the internet and needs to be checked. And ust FYI on newer versions of macOS you can use UFLO rmat for dmg which greatly improves compression.

[kovidgoyal]

No ULFO is correct. tar.xz will still compress better because in a dmg individual files are compressed, while in a tar.xz all files are first tarred and then compressed.

[odamizu]

Originally Posted by KevinH

It would be interesting to see the results from installing from a .tar.xz (.txz) archive as this seems to be flying below gatekeepers radar at the moment even on High Sierra and Mojave.

I'm in over my head on this topic, so my ignorance is probably showing.

I'm still on High Sierra, and I think gatekeeper is still catching .txz on my Mac, unless I'm misunderstanding what gatekeeper is.

• downloaded Sigil 0.9.18 .txz using Safari
• unpacked txz using the Terminal command
• copied Sigil app into my Applications folder
• launched Sigil
• got a pop-up asking, "Sigil is an application downloaded from the Internet. Are you sure you want to open it?"

When I did the same with the test builds you sent me for 0.9.17, I got a similar message, with the addition that the test build was unsigned, so I needed to confirm that I wanted to open an unsigned app.

This is all with High Sierra. Isn't this gatekeeper in action? or am I using the wrong terminology?

Originally Posted by kovidgoyal

If you dont download the dmg using safari, it wont trigger gatekeeper as safari is what sets the attributes of the dmg file that tell gatekeeper that the app was downloaded form the internet and needs to be checked ...

I tried the above again except downloading the .txz with Chrome instead of Safari. Got the same result.

Also, all of the above has always happened with Sigil .dmg (which I know is expected behavior)

[JSWolf]

Originally Posted by odamizu

I'm in over my head on this topic, so my ignorance is probably showing.

I'm still on High Sierra, and I think gatekeeper is still catching .txz on my Mac, unless I'm misunderstanding what gatekeeper is.

• downloaded Sigil 0.9.18 .txz using Safari
• unpacked txz using the Terminal command
• copied Sigil app into my Applications folder
• launched Sigil
• got a pop-up asking, "Sigil is an application downloaded from the Internet. Are you sure you want to open it?"

When I did the same with the test builds you sent me for 0.9.17, I got a similar message, with the addition that the test build was unsigned, so I needed to confirm that I wanted to open an unsigned app.

This is all with High Sierra. Isn't this gatekeeper in action? or am I using the wrong terminology?



I tried the above again except downloading the .txz with Chrome instead of Safari. Got the same result.

Also, all of the above has always happened with Sigil .dmg (which I know is expected behavior)

Try downloading with Firefox. I'd like to know if it works better in this case.

[odamizu]

Originally Posted by JSWolf

Try downloading with Firefox. I'd like to know if it works better in this case.

Same result with Firefox.

The browser that downloads Sigil's .txz does not appear to make a difference. I get a Gatekeeper pop-up when downloaded with Safari, Chrome and Firefox. The only difference is the pop-up will say "Safari downloaded this file ..." or "Chrome downloaded this file ..." or "Firefox downloaded this file ..."

Note: I am not having any trouble launching Sigil. Just sharing the fact that on my Mac running High Sierra, installing Sigil .txz via Terminal results in the same Gatekeeper pop-up as installing Sigil from a .dmg. Then I click "Open" and all is good.

Also, my Security preferences are set to "Allow apps downloaded from App Store and identified developers."

Then again, maybe this isn't what KevinH and kovidgoyal are referring to in the above discussion.

[KevinH]

Move the Sigil.app*.txz from Downloads to your Desktop:

Then use Terminal.app to check out the file attributes as follows:

cd Desktop
ls -a@l *.txz

It will show you any extended attributes.

You are using Apple's tar program to both uncompress and unpack. I use a compiled opensource program from xz utils to run unxz on the .txz to get a .tar file. In this case the extended file attributes seem to get lost.

[odamizu]

Ah. I see. Interesting, and thank you.

In any case, I hope there is a way around the notarization requirement, one way or another.

I'll keep you posted once I start experimenting with Catalina.