Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre > Development

Notices

Reply
 
Thread Tools Search this Thread
Old 06-30-2019, 08:05 PM   #16
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 34,914
Karma: 12669638
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
Regarding embedded python, as far as I can tell, the following of points should apply:

1) Turn off the use of __pycache__ and instead precompile the bytecode at embedding time. This is because I am pretty sure notarized apps are not allowed to change things inside their .app folders. IIRC gatekeeper will periodically recheck the app folder to enforece this.

2) The hardened runtime for some stupid reason seems to only care about machine code. Loading pure python plugins/extensions should therefore be no problem

3) I think the exceptions for dlopening and execing unsigend code should also allow loading third party python C extensions from outseide the app bundle, but am not sure. It may be that plugins that use native code wont be workable on macOS. This will require experimentation.
kovidgoyal is offline   Reply With Quote
Old 09-05-2019, 04:04 PM   #17
odamizu
just an egg
odamizu ought to be getting tired of karma fortunes by now.odamizu ought to be getting tired of karma fortunes by now.odamizu ought to be getting tired of karma fortunes by now.odamizu ought to be getting tired of karma fortunes by now.odamizu ought to be getting tired of karma fortunes by now.odamizu ought to be getting tired of karma fortunes by now.odamizu ought to be getting tired of karma fortunes by now.odamizu ought to be getting tired of karma fortunes by now.odamizu ought to be getting tired of karma fortunes by now.odamizu ought to be getting tired of karma fortunes by now.odamizu ought to be getting tired of karma fortunes by now.
 
odamizu's Avatar
 
Posts: 981
Karma: 1831520
Join Date: Mar 2015
Device: Kindle Oasis1 & Voyage, iOS
Not sure if this makes much difference in the end ...

https://9to5mac.com/2019/09/03/apple...ization-delay/

Quote:
Apple today announced that it is [temporarily] relaxing some notarization requirements for Mac apps. For apps outside of the Mac App Store, macOS Catalina will only run notarized software. However, developers have found it difficult to adopt notarization.

To ease the transition, Apple is temporarily relaxing some of the things that the company previously mandated for an app to be notarized.

... However, Apple has only promised to delay the full enforcement until January 2020, a mere four months away.
And here: https://developer.apple.com/news/?id=09032019a

Quote:
... To make this transition easier and to protect users on macOS Catalina who continue to use older versions of software, we’ve adjusted the notarization prerequisites until January 2020.

You can now notarize Mac software that:
  • Doesn’t have the Hardened Runtime capability enabled.
  • Has components not signed with your Developer ID.
  • Doesn’t include a secure timestamp with your code-signing signature.
  • Was built with an older SDK.
  • Includes the com.apple.security.get-task-allow entitlement with the value set to any variation of true.

Make sure to submit all versions of your software. While Xcode 10 or later is still required to submit, you don’t need to rebuild or re-sign your software before submission.

Last edited by odamizu; 09-05-2019 at 08:56 PM. Reason: clarity
odamizu is offline   Reply With Quote
Old 09-05-2019, 10:15 PM   #18
KevinH
Wizard
KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.
 
Posts: 3,652
Karma: 2209324
Join Date: Nov 2009
Device: many
Thanks for the links.

FWIW, I still strongly believe Apple is totally in the wrong with any type of "notarization requirement" and should not be trying to force developers who already sign their code but distribute outside the Mac app store.

I can understand it for apps inside the Mac app store, but not independent apps that have nothing to do with the app store.

To complicate things it seems they only accept dmg, zip, and pkg packaging instead of the .tar.xz Sigil just moved to because of its much better compression.

Last edited by KevinH; 09-05-2019 at 10:19 PM.
KevinH is offline   Reply With Quote
Old 09-05-2019, 10:18 PM   #19
KevinH
Wizard
KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.
 
Posts: 3,652
Karma: 2209324
Join Date: Nov 2009
Device: many
When final Catalina is finally released, if you decide to upgrade, please report any successes or failures installing Sigil, Calibre, Python3, etc, as Sigil will not be moving its dev machine away from macOS 10.13.6 for the foreseeable future.
KevinH is offline   Reply With Quote
Old 09-05-2019, 11:52 PM   #20
odamizu
just an egg
odamizu ought to be getting tired of karma fortunes by now.odamizu ought to be getting tired of karma fortunes by now.odamizu ought to be getting tired of karma fortunes by now.odamizu ought to be getting tired of karma fortunes by now.odamizu ought to be getting tired of karma fortunes by now.odamizu ought to be getting tired of karma fortunes by now.odamizu ought to be getting tired of karma fortunes by now.odamizu ought to be getting tired of karma fortunes by now.odamizu ought to be getting tired of karma fortunes by now.odamizu ought to be getting tired of karma fortunes by now.odamizu ought to be getting tired of karma fortunes by now.
 
odamizu's Avatar
 
Posts: 981
Karma: 1831520
Join Date: Mar 2015
Device: Kindle Oasis1 & Voyage, iOS
Once final Catalina is released, I plan to install and boot from an external drive, whose sole purpose will be to test how Sigil, Calibre, Alf, Kindle for Mac, etc. fare on Catalina. I will definitely report back.

I'm still hopeful that, as a last resort, it will be possible to override Catalina's Gatekeeper to allow unnotarized apps to run, even if that means they can't be signed (crazy, I know). The few articles I've seen suggest Apple does not intend to close this avenue.

That said, I plan to keep my actual Mac on High Sierra for as long as possible. The external Catalina drive will just be for experimentation
odamizu is offline   Reply With Quote
Old 09-06-2019, 09:18 AM   #21
KevinH
Wizard
KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.
 
Posts: 3,652
Karma: 2209324
Join Date: Nov 2009
Device: many
It would be interesting to see the results from installing from a .tar.xz (.txz) archive as this seems to be flying below gatekeepers radar at the moment even on High Sierra and Mojave.
KevinH is offline   Reply With Quote
Old 09-06-2019, 09:30 AM   #22
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 34,914
Karma: 12669638
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
If you dont download the dmg using safari, it wont trigger gatekeeper as safari is what sets the attributes of the dmg file that tell gatekeeper that the app was downloaded form the internet and needs to be checked. And ust FYI on newer versions of macOS you can use UFLO rmat for dmg which greatly improves compression.
kovidgoyal is offline   Reply With Quote
Old 09-06-2019, 11:01 AM   #23
KevinH
Wizard
KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.
 
Posts: 3,652
Karma: 2209324
Join Date: Nov 2009
Device: many
So use this one from the hdiutil man page, right?

ULFO - UDIF lzfse-compressed image (OS X 10.11+ only)

or is there a newer "UFLO" available but not documented in the man page?


Quote:
Originally Posted by kovidgoyal View Post
If you dont download the dmg using safari, it wont trigger gatekeeper as safari is what sets the attributes of the dmg file that tell gatekeeper that the app was downloaded form the internet and needs to be checked. And ust FYI on newer versions of macOS you can use UFLO rmat for dmg which greatly improves compression.
edit:

Okay using this command line to generate a dmg:

hdiutil create ./Sigil-0.9.18-Mac-Package.dmg -volname "Sigil" -srcfolder ./Sigil.app -format ULFO

gives me

124508032 6 Sep 11:07 Sigil-0.9.18-Mac-Package.dmg


Using .tar.xz (.txz) on the app itself gives me

78071020 3 Sep 11:22 Sigil.app-0.9.18-Mac.txz

So I am saving roughly 40 to 46 meg of download by using tar.xz just on Sigil.app

Am I simply using the wrong command options on hdiutil?

Last edited by KevinH; 09-06-2019 at 11:17 AM.
KevinH is offline   Reply With Quote
Old 09-06-2019, 11:23 AM   #24
KevinH
Wizard
KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.
 
Posts: 3,652
Karma: 2209324
Join Date: Nov 2009
Device: many
Ah, that explains why. If I use Safari to download a .tar.xz and then use the unxz command line from a self compiled xzutuils, it creates the .tar archive and deletes the original tar.xz file.

This also seems to remove any extended attribute quarantine flags on the .tar itself. So if it is clean being tarred, it will be clean being untarred.

Very interesting to know. So their entire gatekeeper extra attributes approach is really full of BS.


Quote:
Originally Posted by kovidgoyal View Post
If you dont download the dmg using safari, it wont trigger gatekeeper as safari is what sets the attributes of the dmg file that tell gatekeeper that the app was downloaded form the internet and needs to be checked. And ust FYI on newer versions of macOS you can use UFLO rmat for dmg which greatly improves compression.
KevinH is offline   Reply With Quote
Old 09-06-2019, 11:35 AM   #25
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 34,914
Karma: 12669638
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
No ULFO is correct. tar.xz will still compress better because in a dmg individual files are compressed, while in a tar.xz all files are first tarred and then compressed.
kovidgoyal is offline   Reply With Quote
Old 09-06-2019, 03:44 PM   #26
odamizu
just an egg
odamizu ought to be getting tired of karma fortunes by now.odamizu ought to be getting tired of karma fortunes by now.odamizu ought to be getting tired of karma fortunes by now.odamizu ought to be getting tired of karma fortunes by now.odamizu ought to be getting tired of karma fortunes by now.odamizu ought to be getting tired of karma fortunes by now.odamizu ought to be getting tired of karma fortunes by now.odamizu ought to be getting tired of karma fortunes by now.odamizu ought to be getting tired of karma fortunes by now.odamizu ought to be getting tired of karma fortunes by now.odamizu ought to be getting tired of karma fortunes by now.
 
odamizu's Avatar
 
Posts: 981
Karma: 1831520
Join Date: Mar 2015
Device: Kindle Oasis1 & Voyage, iOS
Quote:
Originally Posted by KevinH View Post
It would be interesting to see the results from installing from a .tar.xz (.txz) archive as this seems to be flying below gatekeepers radar at the moment even on High Sierra and Mojave.
I'm in over my head on this topic, so my ignorance is probably showing.

I'm still on High Sierra, and I think gatekeeper is still catching .txz on my Mac, unless I'm misunderstanding what gatekeeper is.
  • downloaded Sigil 0.9.18 .txz using Safari
  • unpacked txz using the Terminal command
  • copied Sigil app into my Applications folder
  • launched Sigil
  • got a pop-up asking, "Sigil is an application downloaded from the Internet. Are you sure you want to open it?"

When I did the same with the test builds you sent me for 0.9.17, I got a similar message, with the addition that the test build was unsigned, so I needed to confirm that I wanted to open an unsigned app.

This is all with High Sierra. Isn't this gatekeeper in action? or am I using the wrong terminology?

Quote:
Originally Posted by kovidgoyal View Post
If you dont download the dmg using safari, it wont trigger gatekeeper as safari is what sets the attributes of the dmg file that tell gatekeeper that the app was downloaded form the internet and needs to be checked ...
I tried the above again except downloading the .txz with Chrome instead of Safari. Got the same result.

Also, all of the above has always happened with Sigil .dmg (which I know is expected behavior)
odamizu is offline   Reply With Quote
Old 09-06-2019, 05:17 PM   #27
JSWolf
Resident Curmudgeon
JSWolf ought to be getting tired of karma fortunes by now.JSWolf ought to be getting tired of karma fortunes by now.JSWolf ought to be getting tired of karma fortunes by now.JSWolf ought to be getting tired of karma fortunes by now.JSWolf ought to be getting tired of karma fortunes by now.JSWolf ought to be getting tired of karma fortunes by now.JSWolf ought to be getting tired of karma fortunes by now.JSWolf ought to be getting tired of karma fortunes by now.JSWolf ought to be getting tired of karma fortunes by now.JSWolf ought to be getting tired of karma fortunes by now.JSWolf ought to be getting tired of karma fortunes by now.
 
JSWolf's Avatar
 
Posts: 53,049
Karma: 49154381
Join Date: Nov 2006
Location: Roslindale, Massachusetts
Device: Kobo Aura H2O, Sony PRS-650, Sony PRS-T1, nook STR, iPad 4, iPhone 5
Quote:
Originally Posted by odamizu View Post
I'm in over my head on this topic, so my ignorance is probably showing.

I'm still on High Sierra, and I think gatekeeper is still catching .txz on my Mac, unless I'm misunderstanding what gatekeeper is.
  • downloaded Sigil 0.9.18 .txz using Safari
  • unpacked txz using the Terminal command
  • copied Sigil app into my Applications folder
  • launched Sigil
  • got a pop-up asking, "Sigil is an application downloaded from the Internet. Are you sure you want to open it?"

When I did the same with the test builds you sent me for 0.9.17, I got a similar message, with the addition that the test build was unsigned, so I needed to confirm that I wanted to open an unsigned app.

This is all with High Sierra. Isn't this gatekeeper in action? or am I using the wrong terminology?



I tried the above again except downloading the .txz with Chrome instead of Safari. Got the same result.

Also, all of the above has always happened with Sigil .dmg (which I know is expected behavior)
Try downloading with Firefox. I'd like to know if it works better in this case.
JSWolf is offline   Reply With Quote
Old 09-06-2019, 06:09 PM   #28
odamizu
just an egg
odamizu ought to be getting tired of karma fortunes by now.odamizu ought to be getting tired of karma fortunes by now.odamizu ought to be getting tired of karma fortunes by now.odamizu ought to be getting tired of karma fortunes by now.odamizu ought to be getting tired of karma fortunes by now.odamizu ought to be getting tired of karma fortunes by now.odamizu ought to be getting tired of karma fortunes by now.odamizu ought to be getting tired of karma fortunes by now.odamizu ought to be getting tired of karma fortunes by now.odamizu ought to be getting tired of karma fortunes by now.odamizu ought to be getting tired of karma fortunes by now.
 
odamizu's Avatar
 
Posts: 981
Karma: 1831520
Join Date: Mar 2015
Device: Kindle Oasis1 & Voyage, iOS
Quote:
Originally Posted by JSWolf View Post
Try downloading with Firefox. I'd like to know if it works better in this case.
Same result with Firefox.

The browser that downloads Sigil's .txz does not appear to make a difference. I get a Gatekeeper pop-up when downloaded with Safari, Chrome and Firefox. The only difference is the pop-up will say "Safari downloaded this file ..." or "Chrome downloaded this file ..." or "Firefox downloaded this file ..."

Note: I am not having any trouble launching Sigil. Just sharing the fact that on my Mac running High Sierra, installing Sigil .txz via Terminal results in the same Gatekeeper pop-up as installing Sigil from a .dmg. Then I click "Open" and all is good.

Also, my Security preferences are set to "Allow apps downloaded from App Store and identified developers."

Then again, maybe this isn't what KevinH and kovidgoyal are referring to in the above discussion.
odamizu is offline   Reply With Quote
Old 09-06-2019, 06:11 PM   #29
KevinH
Wizard
KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.
 
Posts: 3,652
Karma: 2209324
Join Date: Nov 2009
Device: many
Move the Sigil.app*.txz from Downloads to your Desktop:

Then use Terminal.app to check out the file attributes as follows:

cd Desktop
ls -a@l *.txz

It will show you any extended attributes.

You are using Apple's tar program to both uncompress and unpack. I use a compiled opensource program from xz utils to run unxz on the .txz to get a .tar file. In this case the extended file attributes seem to get lost.
KevinH is offline   Reply With Quote
Old 09-06-2019, 08:30 PM   #30
odamizu
just an egg
odamizu ought to be getting tired of karma fortunes by now.odamizu ought to be getting tired of karma fortunes by now.odamizu ought to be getting tired of karma fortunes by now.odamizu ought to be getting tired of karma fortunes by now.odamizu ought to be getting tired of karma fortunes by now.odamizu ought to be getting tired of karma fortunes by now.odamizu ought to be getting tired of karma fortunes by now.odamizu ought to be getting tired of karma fortunes by now.odamizu ought to be getting tired of karma fortunes by now.odamizu ought to be getting tired of karma fortunes by now.odamizu ought to be getting tired of karma fortunes by now.
 
odamizu's Avatar
 
Posts: 981
Karma: 1831520
Join Date: Mar 2015
Device: Kindle Oasis1 & Voyage, iOS
Ah. I see. Interesting, and thank you.

In any case, I hope there is a way around the notarization requirement, one way or another.

I'll keep you posted once I start experimenting with Catalina.
odamizu is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
MacOS 10.15 Catalina Beta Discussion Thread OtinG Apple Devices 40 08-21-2019 07:51 AM
Calibre 3.41.3 for macOS 10.14.4 adrianf Library Management 2 04-23-2019 05:15 AM
MacOS Mojave Books App and Calibre datostar Apple Devices 2 10-26-2018 08:25 PM
Touch Have you tried the MacOS App for Android? Nate the great Barnes & Noble NOOK 0 02-29-2012 01:49 PM
Mysterious Missile Launched Near santa Catalina Island PhilipChen Lounge 2 11-09-2010 02:34 PM


All times are GMT -4. The time now is 07:27 AM.


MobileRead.com is a privately owned, operated and funded community.