Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre > Devices

Notices

Reply
 
Thread Tools Search this Thread
Old 03-30-2021, 01:25 PM   #1
Phssthpok
Age improves with wine.
Phssthpok knows how to set a laser printer to stun.Phssthpok knows how to set a laser printer to stun.Phssthpok knows how to set a laser printer to stun.Phssthpok knows how to set a laser printer to stun.Phssthpok knows how to set a laser printer to stun.Phssthpok knows how to set a laser printer to stun.Phssthpok knows how to set a laser printer to stun.Phssthpok knows how to set a laser printer to stun.Phssthpok knows how to set a laser printer to stun.Phssthpok knows how to set a laser printer to stun.Phssthpok knows how to set a laser printer to stun.
 
Posts: 558
Karma: 95229
Join Date: Nov 2014
Device: Kindle Oasis, Kobo Libra II
Autoconverting when copying to device

I have Calibre set up to autoconvert EPUBs to AZW3 when uploading to my Kindle. However, this means that each book I've selected is converted and then uploaded separately. The conversion takes a few seconds, the upload takes less than a second, and then it takes about 30 seconds to upload the metadata. If I have 20 books, it tales about 10 minutes altogether.

what I now do is to convert the books (a few seconds each) and then upload (a few seconds for 20 books, followed by about 40 seconds to upload the metadata) -- about a minute in total.

Would it be possible to somehow batch all the uploads after autoconverting so that the many lengthy metadata uploads can be combined into one?
Phssthpok is offline   Reply With Quote
Old 03-30-2021, 03:37 PM   #2
DNSB
Bibliophagist
DNSB ought to be getting tired of karma fortunes by now.DNSB ought to be getting tired of karma fortunes by now.DNSB ought to be getting tired of karma fortunes by now.DNSB ought to be getting tired of karma fortunes by now.DNSB ought to be getting tired of karma fortunes by now.DNSB ought to be getting tired of karma fortunes by now.DNSB ought to be getting tired of karma fortunes by now.DNSB ought to be getting tired of karma fortunes by now.DNSB ought to be getting tired of karma fortunes by now.DNSB ought to be getting tired of karma fortunes by now.DNSB ought to be getting tired of karma fortunes by now.
 
DNSB's Avatar
 
Posts: 34,517
Karma: 144552660
Join Date: Jul 2010
Location: Vancouver
Device: Kobo Sage, Forma, Clara HD, Lenovo M8 FHD, Paperwhite 4, Tolino epos
Quote:
Originally Posted by Phssthpok View Post
Would it be possible to somehow batch all the uploads after autoconverting so that the many lengthy metadata uploads can be combined into one?
I did a quick test here. I selected 5 books and sent them to a Kindle. The books were converted, sent to the device and the metadata uploaded in one batch after I clicked on send to device.
DNSB is offline   Reply With Quote
Advert
Old 04-05-2021, 04:30 AM   #3
Phssthpok
Age improves with wine.
Phssthpok knows how to set a laser printer to stun.Phssthpok knows how to set a laser printer to stun.Phssthpok knows how to set a laser printer to stun.Phssthpok knows how to set a laser printer to stun.Phssthpok knows how to set a laser printer to stun.Phssthpok knows how to set a laser printer to stun.Phssthpok knows how to set a laser printer to stun.Phssthpok knows how to set a laser printer to stun.Phssthpok knows how to set a laser printer to stun.Phssthpok knows how to set a laser printer to stun.Phssthpok knows how to set a laser printer to stun.
 
Posts: 558
Karma: 95229
Join Date: Nov 2014
Device: Kindle Oasis, Kobo Libra II
Quote:
Originally Posted by DNSB View Post
I did a quick test here. I selected 5 books and sent them to a Kindle. The books were converted, sent to the device and the metadata uploaded in one batch after I clicked on send to device.
Hmm, I wonder why they are done one-by-one on mine in that case? Is there a setting I need to tweak, or a tweak that I need to set?
Phssthpok is offline   Reply With Quote
Old 04-14-2021, 03:21 PM   #4
Phssthpok
Age improves with wine.
Phssthpok knows how to set a laser printer to stun.Phssthpok knows how to set a laser printer to stun.Phssthpok knows how to set a laser printer to stun.Phssthpok knows how to set a laser printer to stun.Phssthpok knows how to set a laser printer to stun.Phssthpok knows how to set a laser printer to stun.Phssthpok knows how to set a laser printer to stun.Phssthpok knows how to set a laser printer to stun.Phssthpok knows how to set a laser printer to stun.Phssthpok knows how to set a laser printer to stun.Phssthpok knows how to set a laser printer to stun.
 
Posts: 558
Karma: 95229
Join Date: Nov 2014
Device: Kindle Oasis, Kobo Libra II
Quote:
Originally Posted by DNSB View Post
I did a quick test here. I selected 5 books and sent them to a Kindle. The books were converted, sent to the device and the metadata uploaded in one batch after I clicked on send to device.
Here's two images showing what happens on mine: one shows me uploading 3 books in EPUB format, which get autoconverted to AXZW3 then uploaded one by one, with 30+ second "upload metadata" actions (plus the time it takes after the book upload before the metadat upload starts). The second one shows me converting 12 books and then uploading them after conversion: 12 books are uploaded and then a single metadata upload happens afterwards, which still only takes the same 30+ seconds as I had for each individual book when autoconverting.

So how come you don't see the same? What sins have I committed that it does them individually on my system? Please help me correct the error of my ways!
Attached Thumbnails
Click image for larger version

Name:	Capture.JPG
Views:	123
Size:	51.7 KB
ID:	186613   Click image for larger version

Name:	Capture2.JPG
Views:	120
Size:	63.0 KB
ID:	186614  
Phssthpok is offline   Reply With Quote
Old 04-14-2021, 07:53 PM   #5
DNSB
Bibliophagist
DNSB ought to be getting tired of karma fortunes by now.DNSB ought to be getting tired of karma fortunes by now.DNSB ought to be getting tired of karma fortunes by now.DNSB ought to be getting tired of karma fortunes by now.DNSB ought to be getting tired of karma fortunes by now.DNSB ought to be getting tired of karma fortunes by now.DNSB ought to be getting tired of karma fortunes by now.DNSB ought to be getting tired of karma fortunes by now.DNSB ought to be getting tired of karma fortunes by now.DNSB ought to be getting tired of karma fortunes by now.DNSB ought to be getting tired of karma fortunes by now.
 
DNSB's Avatar
 
Posts: 34,517
Karma: 144552660
Join Date: Jul 2010
Location: Vancouver
Device: Kobo Sage, Forma, Clara HD, Lenovo M8 FHD, Paperwhite 4, Tolino epos
Looking at your post, my workflow is different from yours. I use a 2 step workflow so before I send the books, I convert them to AZW3. By default, calibre leaves the converted ebook in place so it makes no real difference to storage space required. After conversion, I send the converted ebooks in one step and then delete the AZW3 ebooks.
DNSB is offline   Reply With Quote
Advert
Old 04-15-2021, 12:29 PM   #6
Phssthpok
Age improves with wine.
Phssthpok knows how to set a laser printer to stun.Phssthpok knows how to set a laser printer to stun.Phssthpok knows how to set a laser printer to stun.Phssthpok knows how to set a laser printer to stun.Phssthpok knows how to set a laser printer to stun.Phssthpok knows how to set a laser printer to stun.Phssthpok knows how to set a laser printer to stun.Phssthpok knows how to set a laser printer to stun.Phssthpok knows how to set a laser printer to stun.Phssthpok knows how to set a laser printer to stun.Phssthpok knows how to set a laser printer to stun.
 
Posts: 558
Karma: 95229
Join Date: Nov 2014
Device: Kindle Oasis, Kobo Libra II
Quote:
Originally Posted by DNSB View Post
Looking at your post, my workflow is different from yours. I use a 2 step workflow so before I send the books, I convert them to AZW3. By default, calibre leaves the converted ebook in place so it makes no real difference to storage space required. After conversion, I send the converted ebooks in one step and then delete the AZW3 ebooks.
OK, so what you're now saying is that if you manually convert and then upload the converted books, they upload as a single job -- which is what I originally said -- and not that uploading with autoconversion does the conversions followed by a single upload, which is what you said originally. In other words, you are seeing exactly what I am seeing, despite what you said originally.

None of which answers my original question: Would it be possible for Calibre to somehow batch all the uploads after autoconverting so that the many lengthy metadata uploads can be combined into one?

I can certainly do the conversion and then do the upload as a separate step, which is what I now do, but sometimes I forget and just hit "Send"... and then it takes forever. So if there were a solution, I would be very happy.

Last edited by Phssthpok; 04-15-2021 at 12:34 PM.
Phssthpok is offline   Reply With Quote
Old 04-15-2021, 12:31 PM   #7
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 43,776
Karma: 22666666
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
No, it isnt. The common case is converting takes longer than uploading not vice versa, therefore it doesnt make sense to batch.
kovidgoyal is offline   Reply With Quote
Old 04-17-2021, 10:21 AM   #8
Phssthpok
Age improves with wine.
Phssthpok knows how to set a laser printer to stun.Phssthpok knows how to set a laser printer to stun.Phssthpok knows how to set a laser printer to stun.Phssthpok knows how to set a laser printer to stun.Phssthpok knows how to set a laser printer to stun.Phssthpok knows how to set a laser printer to stun.Phssthpok knows how to set a laser printer to stun.Phssthpok knows how to set a laser printer to stun.Phssthpok knows how to set a laser printer to stun.Phssthpok knows how to set a laser printer to stun.Phssthpok knows how to set a laser printer to stun.
 
Posts: 558
Karma: 95229
Join Date: Nov 2014
Device: Kindle Oasis, Kobo Libra II
Quote:
Originally Posted by kovidgoyal View Post
No, it isnt. The common case is converting takes longer than uploading not vice versa, therefore it doesnt make sense to batch.
Agreed, but it isn't the uploading that takes the time, it's the metadata update -- about 10-15 seconds after the upload finishes while it figures out it needs to do it, plus about 30-40 seconds for the metadata update itself. It's the metadata update that I'd like to do just once at the end of the convert/upload process. Maybe just hold off the metadata update somehow if there are more uploads pending in the queue?
Phssthpok is offline   Reply With Quote
Old 04-17-2021, 10:53 AM   #9
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 43,776
Karma: 22666666
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
That's certainly doable, but a fair amount of work to handle all corner cases.
kovidgoyal is offline   Reply With Quote
Old 04-18-2021, 12:25 PM   #10
Phssthpok
Age improves with wine.
Phssthpok knows how to set a laser printer to stun.Phssthpok knows how to set a laser printer to stun.Phssthpok knows how to set a laser printer to stun.Phssthpok knows how to set a laser printer to stun.Phssthpok knows how to set a laser printer to stun.Phssthpok knows how to set a laser printer to stun.Phssthpok knows how to set a laser printer to stun.Phssthpok knows how to set a laser printer to stun.Phssthpok knows how to set a laser printer to stun.Phssthpok knows how to set a laser printer to stun.Phssthpok knows how to set a laser printer to stun.
 
Posts: 558
Karma: 95229
Join Date: Nov 2014
Device: Kindle Oasis, Kobo Libra II
Quote:
Originally Posted by kovidgoyal View Post
That's certainly doable, but a fair amount of work to handle all corner cases.
I'll leave you to think about it, then...
Phssthpok is offline   Reply With Quote
Old 04-23-2021, 11:50 PM   #11
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 43,776
Karma: 22666666
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
I did think about it and I dont think its do-able with a reasonable amount of effort. The problem is state. In order to be able to remove multiple update metadata jobs safely one has to be sure that none of the other jobs in the queue rely on the metadata being up-to-date. And the jobs system doesnt have that kind of metadata about jobs. Its fairly trivial to just delete all metadata update jobs except the latest, however, guaranteeing that is safe is far from trivial.
kovidgoyal is offline   Reply With Quote
Old 04-24-2021, 05:12 AM   #12
davidfor
Grand Sorcerer
davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.
 
Posts: 24,908
Karma: 47303748
Join Date: Jul 2011
Location: Sydney, Australia
Device: Kobo:Touch,Glo, AuraH2O, GloHD,AuraONE, ClaraHD, Libra H2O; tolinoepos
Quote:
Originally Posted by kovidgoyal View Post
I did think about it and I dont think its do-able with a reasonable amount of effort. The problem is state. In order to be able to remove multiple update metadata jobs safely one has to be sure that none of the other jobs in the queue rely on the metadata being up-to-date. And the jobs system doesnt have that kind of metadata about jobs. Its fairly trivial to just delete all metadata update jobs except the latest, however, guaranteeing that is safe is far from trivial.
Looking at it, the update metadata job is queued after a book upload or delete. And run immediately. It could be done that if the next job in the queue is a book upload or delete, the update metadata could be skipped. That way, what we currently see as:

Code:
upload book
update metadata
upload book
update metadata
upload book
update metadata
upload book
update metadata
convert
upload book
update metadata
upload book
update metadata
Would become:
Code:
upload book
upload book
upload book
upload book
update metadata
convert
upload book
upload book
update metadata
And it should be driver specific. Then the drivers that need the update metadata after each book upload could still do it. I think that for devices using the drivers based on the USBMS driver that are only sending files without updating anything else on the device it would be safe. It would be drivers that do something more that would need care. It would not be a problem for the KoboTouch driver, but might be for others that a using database or something else. It could be a flag in the driver that turns on doing this. With the default at the top of the driver class hierarchy being "off" so that there is no change in behaviour unless the driver maintainer decides it is a good idea.

I think that would work as long as you can see what is the next job and its type or something similar.
davidfor is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Copying files to device, do not change metadata Jellby Calibre 7 01-19-2014 03:25 AM
Copying device library to Micro SD card RobertJSawyer Kobo Reader 8 07-18-2013 03:45 AM
Copying bought kindle book to other device? dzigili Amazon Kindle 8 11-19-2010 08:13 AM
Copying books to Kobo device kjturner71 Kobo Reader 5 11-07-2010 04:41 PM
Copying new books to device loses my place in the book I'm currently reading? Miker2k Calibre 1 09-14-2009 01:13 PM


All times are GMT -4. The time now is 01:00 PM.


MobileRead.com is a privately owned, operated and funded community.