Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre > Recipes

Notices

Reply
 
Thread Tools Search this Thread
Old 02-27-2024, 01:14 AM   #1
mkgtu
Zealot
mkgtu is generous with chocolatemkgtu is generous with chocolatemkgtu is generous with chocolatemkgtu is generous with chocolatemkgtu is generous with chocolatemkgtu is generous with chocolatemkgtu is generous with chocolatemkgtu is generous with chocolatemkgtu is generous with chocolatemkgtu is generous with chocolatemkgtu is generous with chocolate
 
Posts: 129
Karma: 33000
Join Date: Feb 2010
Device: Currently:Voyage, Oasis 3, Kindle mobile apps, andKindle Fire
New Yorker too large to email

Don't know what, if anything, can be done about this. But today for instance (Feb 26) the downloaded New Yorker was 93mb. Way too large for emailing. Even after editing the epub by compressing images to the default (20 and 40) level the resulting file is still 83mb. Still too big.

So I converted the 83mb epub to mobi and the result was a 15mb file. I then the converted that mobi to epub and emailed it. Quite a difference! But at least it emailed ok to kindle and displays as it should.

The New Yorker does occasionally create overly large files. Just wondering if there is some way that the recipe can prevent that. I suppose one can create custom recipes that omit images or whatever. But those custom recipes begin to fail when the base recipe is updated. But in this case even severely compressing the images didn't shrink the file by much, so images may not have been the problem.

It's beyond my novice ability to fix.
mkgtu is offline   Reply With Quote
Old 02-27-2024, 09:30 PM   #2
DNSB
Bibliophagist
DNSB ought to be getting tired of karma fortunes by now.DNSB ought to be getting tired of karma fortunes by now.DNSB ought to be getting tired of karma fortunes by now.DNSB ought to be getting tired of karma fortunes by now.DNSB ought to be getting tired of karma fortunes by now.DNSB ought to be getting tired of karma fortunes by now.DNSB ought to be getting tired of karma fortunes by now.DNSB ought to be getting tired of karma fortunes by now.DNSB ought to be getting tired of karma fortunes by now.DNSB ought to be getting tired of karma fortunes by now.DNSB ought to be getting tired of karma fortunes by now.
 
DNSB's Avatar
 
Posts: 35,324
Karma: 145435140
Join Date: Jul 2010
Location: Vancouver
Device: Kobo Sage, Forma, Clara HD, Lenovo M8 FHD, Paperwhite 4, Tolino epos
If you use the https://www.amazon.ca/sendtokindle site (change the .ca to the appropriate Amazon site for your country), you can send up to a 200MB file.
DNSB is offline   Reply With Quote
Advert
Old 02-27-2024, 11:56 PM   #3
unkn0wn
Evangelist
unkn0wn can do the Funky Gibbon.unkn0wn can do the Funky Gibbon.unkn0wn can do the Funky Gibbon.unkn0wn can do the Funky Gibbon.unkn0wn can do the Funky Gibbon.unkn0wn can do the Funky Gibbon.unkn0wn can do the Funky Gibbon.unkn0wn can do the Funky Gibbon.unkn0wn can do the Funky Gibbon.unkn0wn can do the Funky Gibbon.unkn0wn can do the Funky Gibbon.
 
Posts: 444
Karma: 82686
Join Date: May 2021
Device: kindle
https://github.com/unkn0w7n/calibre/...1c870c0fdb8334
unkn0wn is offline   Reply With Quote
Old 02-28-2024, 12:07 AM   #4
DNSB
Bibliophagist
DNSB ought to be getting tired of karma fortunes by now.DNSB ought to be getting tired of karma fortunes by now.DNSB ought to be getting tired of karma fortunes by now.DNSB ought to be getting tired of karma fortunes by now.DNSB ought to be getting tired of karma fortunes by now.DNSB ought to be getting tired of karma fortunes by now.DNSB ought to be getting tired of karma fortunes by now.DNSB ought to be getting tired of karma fortunes by now.DNSB ought to be getting tired of karma fortunes by now.DNSB ought to be getting tired of karma fortunes by now.DNSB ought to be getting tired of karma fortunes by now.
 
DNSB's Avatar
 
Posts: 35,324
Karma: 145435140
Join Date: Jul 2010
Location: Vancouver
Device: Kobo Sage, Forma, Clara HD, Lenovo M8 FHD, Paperwhite 4, Tolino epos
Thanks! An nice fix to the recipe to allow continuing to automate the procedure.
DNSB is offline   Reply With Quote
Old 02-28-2024, 11:10 PM   #5
mkgtu
Zealot
mkgtu is generous with chocolatemkgtu is generous with chocolatemkgtu is generous with chocolatemkgtu is generous with chocolatemkgtu is generous with chocolatemkgtu is generous with chocolatemkgtu is generous with chocolatemkgtu is generous with chocolatemkgtu is generous with chocolatemkgtu is generous with chocolatemkgtu is generous with chocolate
 
Posts: 129
Karma: 33000
Join Date: Feb 2010
Device: Currently:Voyage, Oasis 3, Kindle mobile apps, andKindle Fire
Sounds good because it preserves the automation. But I'm curious about this line in the new recipe:
self.log.warn('Kindle Output profile being used, reducing image quality to keep file size below amazon email threshold')

Does this mean that in order for the necessary compression to happen the Common Options "Page Setup" has to be set to one of the "Kindle" options. Otherwise how does calibre know this is a "Kindle Output"?

As has long been the case, if you set the Page Setup as any "Kindle", the resulting filename only includes the Title itself, but none of the Date information. For that reason, I have always used Generic E-ink HD as the page setup because that allows the inclusion of the date. Kindle itself has no problem displaying the dates, but calibre still prevents the dates from appearing in any specifically "Kindle" outputs. And, at least to me, the dates are really important on my Kindle so I know which issue to open.

I would hope that this doesn't create an either/or situation: you can either have dates, or files that are small enough to email, but you can't have both?

One solution is to remove the restriction on including dates in the "Kindle" outputs if that's possible. I'm wondering if that's a leftover from long ago when maybe the Kindle mobi files weren't compatible with the dates. But mobi is dead. Amazon now accepts epubs.

And I'd also note that while the New Yorker is a frequent culprit in creating gigantic files, The New York Times, The Spectator, and The Economist also occasionally break the bank - especially for "double" issues.

I use Fastmail because it allows a total email size of 70mb, but with encoding etc that usually means an attachment of 50mb. GMX is also 50mb, GMail only 25mb.
mkgtu is offline   Reply With Quote
Advert
Old 02-29-2024, 01:06 AM   #6
unkn0wn
Evangelist
unkn0wn can do the Funky Gibbon.unkn0wn can do the Funky Gibbon.unkn0wn can do the Funky Gibbon.unkn0wn can do the Funky Gibbon.unkn0wn can do the Funky Gibbon.unkn0wn can do the Funky Gibbon.unkn0wn can do the Funky Gibbon.unkn0wn can do the Funky Gibbon.unkn0wn can do the Funky Gibbon.unkn0wn can do the Funky Gibbon.unkn0wn can do the Funky Gibbon.
 
Posts: 444
Karma: 82686
Join Date: May 2021
Device: kindle
yes, output profile needs to be kindle and I think you wouldn't need compression after this update.

I also face the title-date problem, especially when calibre shows that the mobi is already present when i connect my device, but then its the yesterdays mobi i left there to read.

set profile as kindle and use this code.. the titile will include date too.
Code:
    def __init__(self, *args, **kwargs):
        BasicNewsRecipe.__init__(self, *args, **kwargs)
	from datetime import date
        if self.output_profile.short_name.startswith('kindle'):
            # Reduce image sizes to get file size below amazon's email
            # sending threshold
            self.web2disk_options.compress_news_images = True
            self.web2disk_options.compress_news_images_auto_size = 5
            self.log.warn('Kindle Output profile being used, reducing image quality to keep file size below amazon email threshold')
            self.title = self.title + ' [' + date.today().strftime('%b %d, %Y') + ']'
we can add this code to all the recipes you mentioned.
unkn0wn is offline   Reply With Quote
Old 02-29-2024, 02:39 AM   #7
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 43,848
Karma: 22666666
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
There is no compression needed. And if you connect your Kindle with a USB and send downloaded news directly you will see why kindle profile causes date to be removed. That's not going not change. If you are sending EPUBs anyway, which will be *converted* by amazon's sub par conversion service, there is no reason to use Kindle profile.
kovidgoyal is offline   Reply With Quote
Old 02-29-2024, 11:14 AM   #8
unkn0wn
Evangelist
unkn0wn can do the Funky Gibbon.unkn0wn can do the Funky Gibbon.unkn0wn can do the Funky Gibbon.unkn0wn can do the Funky Gibbon.unkn0wn can do the Funky Gibbon.unkn0wn can do the Funky Gibbon.unkn0wn can do the Funky Gibbon.unkn0wn can do the Funky Gibbon.unkn0wn can do the Funky Gibbon.unkn0wn can do the Funky Gibbon.unkn0wn can do the Funky Gibbon.
 
Posts: 444
Karma: 82686
Join Date: May 2021
Device: kindle
i know date shows up under the periodical in kindle but if i have past editions of the periodical, calibre thinks that the book is already present, then i get confused if the new edition in the library is actually synced to kindle or not.

With different titles (of date) I can check which one's of the same recipe are synced and still on my kindle.

thats why I actaully added a piece of code to change title + date if output profile is kindle for some of my recipes.
unkn0wn is offline   Reply With Quote
Old 02-29-2024, 09:48 PM   #9
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 43,848
Karma: 22666666
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
Unless you are sending them to the kindle using some means other than USB transfer via calibre, there is no reason for it to get confused. Matching is not done via title/author in such a case.
kovidgoyal is offline   Reply With Quote
Old 03-01-2024, 03:24 AM   #10
mkgtu
Zealot
mkgtu is generous with chocolatemkgtu is generous with chocolatemkgtu is generous with chocolatemkgtu is generous with chocolatemkgtu is generous with chocolatemkgtu is generous with chocolatemkgtu is generous with chocolatemkgtu is generous with chocolatemkgtu is generous with chocolatemkgtu is generous with chocolatemkgtu is generous with chocolate
 
Posts: 129
Karma: 33000
Join Date: Feb 2010
Device: Currently:Voyage, Oasis 3, Kindle mobile apps, andKindle Fire
Kovid, if I read between the lines, you seem to feel that transferring via USB to a Kindle device is the norm.
At least in my case that is far from the norm. Automated emailing is for me a priority. I want/need these items saved (via email or "send to Kindle") to the Amazon cloud so I can download them to my phone or tablet Kindle apps. That can't be done via USB.

I read your comment to imply that if you're doing anything other than USB transfer, then you may be S--t out of luck.


Sent from my SM-G986U1 using Tapatalk
mkgtu is offline   Reply With Quote
Old 03-01-2024, 03:35 AM   #11
mkgtu
Zealot
mkgtu is generous with chocolatemkgtu is generous with chocolatemkgtu is generous with chocolatemkgtu is generous with chocolatemkgtu is generous with chocolatemkgtu is generous with chocolatemkgtu is generous with chocolatemkgtu is generous with chocolatemkgtu is generous with chocolatemkgtu is generous with chocolatemkgtu is generous with chocolate
 
Posts: 129
Karma: 33000
Join Date: Feb 2010
Device: Currently:Voyage, Oasis 3, Kindle mobile apps, andKindle Fire
Quote:
Originally Posted by unkn0wn View Post
yes, output profile needs to be kindle and I think you wouldn't need compression after this update.



I also face the title-date problem, especially when calibre shows that the mobi is already present when i connect my device, but then its the yesterdays mobi i left there to read.



set profile as kindle and use this code.. the titile will include date too.

Code:
    def __init__(self, *args, **kwargs):

        BasicNewsRecipe.__init__(self, *args, **kwargs)

from datetime import date

        if self.output_profile.short_name.startswith('kindle'):

            # Reduce image sizes to get file size below amazon's email

            # sending threshold

            self.web2disk_options.compress_news_images = True

            self.web2disk_options.compress_news_images_auto_size = 5

            self.log.warn('Kindle Output profile being used, reducing image quality to keep file size below amazon email threshold')

            self.title = self.title + ' [' + date.today().strftime('%b %d, %Y') + ']'
we can add this code to all the recipes you mentioned.
So has this code been added to the stock recipe, or do we have to create a custom recipe and add the code, which would mean the recipe might stop working if you update the stock recipe.

I used to, and sometimes still do, create custom recipes just to add code to "left align" text. But I keep getting frustrated by custom recipes no longer working after changes to the underlying stock recipe. Then I'd need to re-create a custom version of the stock recipe.

Some recipes left align text quite well, others don't. And justified text created by Calibre usually leaves huge gaps between words, making articles difficult to read or scan.

Sent from my SM-G986U1 using Tapatalk
mkgtu is offline   Reply With Quote
Old 03-01-2024, 03:48 AM   #12
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 43,848
Karma: 22666666
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
Quote:
Originally Posted by mkgtu View Post
Kovid, if I read between the lines, you seem to feel that transferring via USB to a Kindle device is the norm.
At least in my case that is far from the norm. Automated emailing is for me a priority. I want/need these items saved (via email or "send to Kindle") to the Amazon cloud so I can download them to my phone or tablet Kindle apps. That can't be done via USB.

I read your comment to imply that if you're doing anything other than USB transfer, then you may be S--t out of luck.


Sent from my SM-G986U1 using Tapatalk

If you are emailing you dont need to use a Kindle profile. You arent emailing to a kindle. You are emailing to amazons conversion service.
kovidgoyal is offline   Reply With Quote
Old 03-02-2024, 03:29 PM   #13
mkgtu
Zealot
mkgtu is generous with chocolatemkgtu is generous with chocolatemkgtu is generous with chocolatemkgtu is generous with chocolatemkgtu is generous with chocolatemkgtu is generous with chocolatemkgtu is generous with chocolatemkgtu is generous with chocolatemkgtu is generous with chocolatemkgtu is generous with chocolatemkgtu is generous with chocolate
 
Posts: 129
Karma: 33000
Join Date: Feb 2010
Device: Currently:Voyage, Oasis 3, Kindle mobile apps, andKindle Fire
True. And I never do use the Kindle profile because that profile omits the dates in the title. So my quandary, or question is, it looks like the suggested code above to be added to recipes only triggers image compression if the output profile "begins with 'Kindle'". And if you have to set the profile to Kindle you can shrink images but you lose the date in the title.

Don't know how it would work in practice, but it's the SIZE of the file that's the problem. In the New Yorker case I cited, a 93mb file attachment would be rejected by almost any email service, regardless of whether it was being emailed to Amazon or my grandmother.

And while the inclusion of dates in the Kindle output may have caused problems in years gone by, is that still a problem, given that Amazon has dramatically changed the way they do things? Is it still necessary to exclude dates in the titles of news in the Kindle output? If the dates were allowed then we could still use the code above to limit file sizes without losing the dates, as the dates are really the only way to distinguish among multiple issues of the same magazine on a Kindle.

Sent from my SM-G986U1 using Tapatalk
mkgtu is offline   Reply With Quote
Old 03-02-2024, 04:55 PM   #14
mkgtu
Zealot
mkgtu is generous with chocolatemkgtu is generous with chocolatemkgtu is generous with chocolatemkgtu is generous with chocolatemkgtu is generous with chocolatemkgtu is generous with chocolatemkgtu is generous with chocolatemkgtu is generous with chocolatemkgtu is generous with chocolatemkgtu is generous with chocolatemkgtu is generous with chocolate
 
Posts: 129
Karma: 33000
Join Date: Feb 2010
Device: Currently:Voyage, Oasis 3, Kindle mobile apps, andKindle Fire
Quote:
Originally Posted by unkn0wn View Post
yes, output profile needs to be kindle and I think you wouldn't need compression after this update.



I also face the title-date problem, especially when calibre shows that the mobi is already present when i connect my device, but then its the yesterdays mobi i left there to read.



set profile as kindle and use this code.. the titile will include date too.

Code:
    def __init__(self, *args, **kwargs):

        BasicNewsRecipe.__init__(self, *args, **kwargs)

from datetime import date

        if self.output_profile.short_name.startswith('kindle'):

            # Reduce image sizes to get file size below amazon's email

            # sending threshold

            self.web2disk_options.compress_news_images = True

            self.web2disk_options.compress_news_images_auto_size = 5

            self.log.warn('Kindle Output profile being used, reducing image quality to keep file size below amazon email threshold')

            self.title = self.title + ' [' + date.today().strftime('%b %d, %Y') + ']'
we can add this code to all the recipes you mentioned.
I still have concerns about the date inclusion. This code does add the date to periodicals when using Kindle output, but obviously only with recipes that include this code.
When selecting Kindle output, that is a global option only. Recipes that include this code will add the date, but other news downloads which don't contain this code will exclude the date because they are all forced to use the Kindle output profile. I schedule about 14 periodicals for download. Most of them don't have size problems, only the 2 or 3 I mentioned, and those only occasionally. So if I switch to using the Kindle output for the sake of those 2 or 3, I either lose dates on most downloads or I have to customize all my scheduled download recipes to include this code, or at least the part that forces the inclusion of the date in the title, which means those recipes will be frequently outdated.

Seems I'd be better off not using the Kindle output and if I get an occasional file that is too big to email I can just temporarily switch to Kindle output and re-download the offending issue.

Or, another thought, those who don't email publications but instead transfer using Calibre and a device connected by USB, must also be using the mobi or azw output format. (I don't think Kindle can display epubs natively) So maintain the practice of not including dates in the mobi format since it causes problems transferring to Kindle via USB, but allow dates to always be included in epub files, even those created created using the Kindle output profile, because dates don't create problems with emailed epubs or epubs uploaded using the "send to Kindle" app. And they don't cause problems with Amazon's conversion process.

If dates could be included in epub news files, there'd be no problem switching to the Kindle output profile globally.

Sent from my SM-G986U1 using Tapatalk
mkgtu is offline   Reply With Quote
Old 03-02-2024, 09:31 PM   #15
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 43,848
Karma: 22666666
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
You dont need any special image compression. Just run the download again and see for yourslef.
kovidgoyal is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
New Yorker epub continues to fail email to KIndle mkgtu Recipes 2 12-17-2022 06:20 PM
New Yorker email epub to Kindle rejected in recent weeks mkgtu Recipes 0 11-28-2022 12:58 PM
Content How can I email a large file to my Kindle? TheFattie Amazon Kindle 3 11-30-2016 08:18 PM
Remove large amts of email Breathinlady Kindle Fire 1 04-13-2015 06:41 PM
Large stopgap ereader with large fonts renushan Which one should I buy? 2 08-16-2012 01:49 AM


All times are GMT -4. The time now is 05:00 AM.


MobileRead.com is a privately owned, operated and funded community.