09-26-2016, 08:09 PM | #1 |
Member
Posts: 21
Karma: 10
Join Date: Sep 2016
Device: Note5
|
Extremely slow book import into companion
The background:
I have relatively new phone (samsung note5) with the latest (paid) version of Calibre Companion installed. I have a moderately large ebook library (7000+, epubs) stored in Calibre on a fast PC. The wifi connecting them is AC+ level meaning 200+mbits. The short issue is that no matter what I try any type of 'book import' into the companion app is insanely slow. At first I tried connecting the phone as a 'wireless device' and doing the transfer as a push from calibre. That mostly seemed to work as long as I didn't select more than a few hundred at a time to copy but it took on the order of 60+ minutes to transfer 200 books, which is pretty insane. I gave up on that and manually copied the calibre folder from the PC to the phone via USB (took about 50 minutes total) and then selected that as a 'cloud server' via the connect menu and told it to download everything (because I have to redownload files already on the phone??) It's currently running and according to its progress meter it's completing maybe one book per minute. This sucks. How do I make it faster? |
09-26-2016, 10:25 PM | #2 |
US Navy, Retired
Posts: 9,865
Karma: 13806776
Join Date: Feb 2009
Location: North Carolina
Device: Icarus Illumina XL HD, Nexus 7
|
This FAQ entry may help. http://cc_faq.multipie.co.uk/index.php?solution_id=1094
|
Advert | |
|
09-26-2016, 11:12 PM | #3 |
Member
Posts: 21
Karma: 10
Join Date: Sep 2016
Device: Note5
|
I'm actually attempting to transfer directly to the main memory of the device which I assume isn't using fat32??
|
09-27-2016, 04:56 AM | #4 |
Member
Posts: 21
Karma: 10
Join Date: Sep 2016
Device: Note5
|
Nope, changing the file name template doesn't appear to change much (although why on earth it defaults to all files in a single directory..)
|
09-27-2016, 06:53 AM | #5 | |
US Navy, Retired
Posts: 9,865
Karma: 13806776
Join Date: Feb 2009
Location: North Carolina
Device: Icarus Illumina XL HD, Nexus 7
|
Quote:
|
|
Advert | |
|
09-27-2016, 06:55 AM | #6 | |
Grand Sorcerer
Posts: 11,939
Karma: 7219261
Join Date: Jan 2010
Location: Notts, England
Device: Kobo Libra 2
|
Quote:
As for why your device is slow, I cannot say. If you changed the template, did you reload the books? If not then you might still be seeing the slowdown because the directories share the same "level" as the already-there books. Up to now we have not encountered a case where enabling the two options in Settings / File names, changing the template as discussed in the FAQ, and reloading the books has not fixed the problem. Assuming that adding the first book was quick but it slowed down as more books were added, if changing the template does not fix the problem then I am out of ideas. If we cannot solve your problems you are welcome to ask for a refund. If you want to do that then use "Contact Us" on the www.multipie.co.uk website. Technical note: The local library cloud connection simply copies the file using Android system interfaces then adds a record to its database. It does almost nothing else. If this is slow then either reading a file from the SD card is very slow, creating/writing a file in main memory (using the SAF) is very slow, or both. It could be that writing the database might be the cause of the slowdown, but given that some people have in excess of 20,000 books and aren't seeing a DB slowdown this possibility is unlikely. Hmmmm ... random thought. If you have a lot of apps that run all the time then CC could be dramatically affected. Every time CC asks Android to copy a bit of a file Android will consider swapping to some other application. If there is always such an app available then CC will lose. I consider this possibility very unlikely, but it is worth asking. |
|
09-28-2016, 02:33 AM | #7 |
Member
Posts: 21
Karma: 10
Join Date: Sep 2016
Device: Note5
|
Hey,
As I tried to explain before, there's no actual SD card involved, both Companion and the original copied library of books exist on the 'main memory' of the device. It's a note 5 so I couldn't even add a SD card if I wanted to! I'm going to try again with a clean re-install of CC and try the template change again and see if I can get slightly more precise data. For reference copying the entire folder to the device over usb in the first place took about 60 minutes. I have roughly 7.5k books in the library right now, can anyone give me some vague ballpark numbers as to how long that should take to import into CC as an on-disk 'cloud library' connection? Also, I'm sure this gets asked all the time, but if I have a complete copy of a Calibre library including metadata.db why does CC need to duplicate it file by file? |
09-28-2016, 02:53 AM | #8 |
Member
Posts: 21
Karma: 10
Join Date: Sep 2016
Device: Note5
|
Update:
This time it's going much faster so far, so I guess we can blame my incompetence for the previous result. On the other hand, I am getting a ton of messages about "Can't save {book title} to main memory", specifically when the title contains a '#' character. This seems like a minor issue somewhere.. Update the second: I left the device alone for 10 or 15 minutes and it made it down to something like "2771" books remaining and refused to move no matter how long I watched it so I disconnected, reconnected and hit 'download all' again. It told me it was going to download all 7500 books again but when I hit 'yes' it manage to only queue up the roughly 2700 remaining. Still whining about titles with # though. No idea why. Last edited by rmzg; 09-28-2016 at 03:12 AM. |
09-28-2016, 03:27 AM | #9 | ||
Grand Sorcerer
Posts: 11,939
Karma: 7219261
Join Date: Jan 2010
Location: Notts, England
Device: Kobo Libra 2
|
Quote:
Quote:
Queuing all the books is normal. CC determines that it already has the book during the queue processing phase. As for the error, it could be something in calibre or in CC. If you send me debug logs as described in our FAQ How do I get help? I might be able to work out what is going wrong. Please erase the multipie.co.uk address. EDIT: I just realized that you might have been "connecting" to the local cloud library and using its download-all function. That would mean that calibre isn't directly involved. If this is what you are doing then I am confused because I can't find any trace the message you describe. Is this the message you are seeing: "Cannot save book {title} to device memory"? Last edited by chaley; 09-28-2016 at 06:46 AM. |
||
09-29-2016, 12:52 AM | #10 |
Member
Posts: 21
Karma: 10
Join Date: Sep 2016
Device: Note5
|
I have a folder on the phone that I specified as my 'cloud connection' source that I then selected 'download all' from.
The exact error is "Cannot save book {title} to device memory" Where {title} is the actual title of the book e.g. "Joe's Mystery Case #21" |
09-29-2016, 03:35 AM | #11 | |
Grand Sorcerer
Posts: 11,939
Karma: 7219261
Join Date: Jan 2010
Location: Notts, England
Device: Kobo Libra 2
|
Quote:
Without a debug log I can't say exactly what is happening, but I can make an educated guess. The calibre database contains the text of the file path from the root of the library to a book. The stored files and folders must exactly match that path or CC will not be able to find ("download") the book. Is it possible that the method you used to transfer the library to your device "helped" you out and changed the actual paths, perhaps by changing the # character to something else? You can check this by looking at the folders/files for some failed books. If there is a # in the title but that character is missing/changed in the title folder name or the actual book file name then CC will fail. As an experiment I copied a small library from a Windows 10 machine to one of my devices using Windows MTP. One of the books has a # in the title. The folders and files were created correctly, and CC could "download" that book. |
|
09-29-2016, 07:36 AM | #12 | ||||
Member
Posts: 21
Karma: 10
Join Date: Sep 2016
Device: Note5
|
Good guess as to the issue but confusion deepens over the cause.
Some relevant sections of the debug log below. Quote:
Quote:
Quote:
I added the folder to my PHONE originally by simply opening it in the explorer window via "This PC" (Windows 10), navigating to the folder, then using ctrl-c and ctrl-v to paste the calibre library folder. When I try this again with just the individual book folder I end up with an epub file named Quote:
I've tried copying the folder and file to the phone again in various combinations and I can't get it to reproduce the file name change. I'll note that in my initial attempt to copy the Calibre Library over, something was *seriously* unhappy with the file named 'metadata.db', any attempt to copy files that included a file named that generated an error message along the lines of "The device has stopped responding or disconnected", so, uh, that's weird. I eventually changed the name of the file to something innocuous, copied it to the phone, then used the phone's file manager to rename it. |
||||
09-29-2016, 07:54 AM | #13 | ||
Grand Sorcerer
Posts: 11,939
Karma: 7219261
Join Date: Jan 2010
Location: Notts, England
Device: Kobo Libra 2
|
Quote:
It is worth noting that MTP is really flaky. I have personally seen it lose files (not show files that are on the device) and create files (show files that are not on the device). I avoid using it like the plague. Usually when I need to do a transfer I use one of the cloud tools, usually dropbox and dropsync. There are tools available like AirDroid that I haven't tried. The 10 becoming 1 could be calibre shrinking the path component. It does that when it thinks there is a possibility that the name is too long. Quote:
|
||
09-29-2016, 10:02 AM | #14 | |
Member
Posts: 21
Karma: 10
Join Date: Sep 2016
Device: Note5
|
Quote:
By the way, is there a short cut for 'clearing' a search? Like, after I searched and it displayed a subset what do I press to go back to just the default 'all items' display? |
|
09-29-2016, 11:37 AM | #15 | |
Well trained by Cats
Posts: 30,373
Karma: 58053698
Join Date: Aug 2009
Location: The Central Coast of California
Device: Kobo Libra2,Kobo Aura2v1, K4NT(Fixed: New Bat.), Galaxy Tab A
|
Quote:
There is a Clear button on the dialog |
|
|
Similar Threads | ||||
Thread | Thread Starter | Forum | Replies | Last Post |
PDF extremely slow? | Amiieey | KOReader | 1 | 03-24-2016 11:23 AM |
Extremely slow charging | dpadgham | Kobo Tablets | 0 | 06-29-2015 12:12 PM |
Another extremely SLOW Calibre | booklover81 | Calibre | 6 | 11-13-2013 10:30 PM |
Sigil 0.5.3 extremely slow to reflect changes | yotzeret | Sigil | 4 | 08-16-2012 06:10 AM |
iLiad Extremely Slow | jasem200 | iRex | 1 | 04-16-2008 05:55 PM |