Filename changed to one letter when importing from Calibre

[Demingy]

I have Calibre Companion on my first gen Kindle HD and love it! Today I went to add 174 books to the device (which would make it a total of 645 books on my device ) and everything seemed fine, there were no error messages, but there was one book in my Calibre library that didn't have a check mark in the On Device column. So I tried to copy it over again and the dialog in Calibre and Calibre Companion showed it copied over (again no errors), but once again the check mark didn't appear in Calibre and the book wasn't listed in CC. When I closed CC, the book was in my carousel. I decided to go ahead and restart my Kindle. The book was still in the carousel but not listed in CC.

I decided to delete the book from my device and try to load it again. Again no error messages (just the usual dialog), and this time the book appeared in CC and was check-marked in Calibre, but another book got uncheck-marked in Calibre and disappeared from Calibre. When I loaded it into CC the original book got unmarked and disappeared from Calibre (and this time didn't show up in my carousel or my book folder).

I have plenty of memory on my Kindle (24 GB out of 27GB available), so I was wondering if I somehow discovered the upper limit to the number of books that CC can manage. There are no error messages from either Calibre or CC, the books just seem to keep swapping.

[Difflugia]

I have 712 books in Calibre Companion on my device, so it doesn't look like it's a simple numbers thing.

[Demingy]

I was afraid of that (and glad at the same time that I haven't reached some limit). All I can think of at this point is to try restarting everything and try again.

[Purple Lady]

I have over 900 and there are people with thousands so it's not that. I did have a problem once where the template I used to create the file names created names that were too long. Whatever algorithm CC uses to shorten the name created a few books with the same filename. Check the filename on your device to see if it's the same as the other book.

[chaley]

CC can support thousands of books. I personally have 1400 on my "production" reader and I know of users with in excess of 15,000 on the device, so 600 books isn't in principle a problem. However, system memory might be a problem. The first gen kindle has only 512 MB of memory available to apps, which is not very much. In addition to having not a lot of program memory, the first gen kindle fire runs Android 2.3 (Gingerbread) that is known to have problems with memory management of images such as covers.

If you could send me a CC debug log then I might be able to tell you what the problem is. You send a debug log by going to CC's settings, scrolling to the bottom, and choosing "Send a debug log to MultiPie". It would save us a little confusion if you remove my son's email address (the multipie.co.uk one) so that he doesn't think the log is coming from the sky (as many do).

If memory is the problem then something that might help is to change CC's settings to use the smallest cover image possible. Because of Android 2.3 problems with managing memory used by images, making the images smaller can reduce memory requirements. You change the cover size by going to Settings / Covers and thumbnails / Downloaded cover size in pixels. Pick 160, then connect as a wireless device and let the sync complete.

[Demingy]

Originally Posted by Purple Lady

I have over 900 and there are people with thousands so it's not that. I did have a problem once where the template I used to create the file names created names that were too long. Whatever algorithm CC uses to shorten the name created a few books with the same filename. Check the filename on your device to see if it's the same as the other book.

Yep, I managed to figure out that this was my problem when I looked closer at the metadata info in CC - the names were shortened to one letter making them the same filename.

Unfortunately I haven't been able to fix it yet. I updated the names in Calibre, but CC keeps calling them "S" (.mobi and .azw3) so it kicks out the "duplicate". I've tried reloading the books in Calibre, updating the name, reloading in CC, and they keep getting saved as "S" (.mobi and .azw3).

I thought about changing the settings to use the book ID as a filename instead, but I started with this setting and my searches in the Fire didn't work properly for some reason. I may have to switch back to that though. Hopefully this helps someone else who runs into this problem as well, I couldn't find anything about it when I searched.

[Demingy]

Originally Posted by chaley

CC can support thousands of books. I personally have 1400 on my "production" reader and I know of users with in excess of 15,000 on the device, so 600 books isn't in principle a problem. However, system memory might be a problem. The first gen kindle has only 512 MB of memory available to apps, which is not very much. In addition to having not a lot of program memory, the first gen kindle fire runs Android 2.3 (Gingerbread) that is known to have problems with memory management of images such as covers.

If you could send me a CC debug log then I might be able to tell you what the problem is. You send a debug log by going to CC's settings, scrolling to the bottom, and choosing "Send a debug log to MultiPie". It would save us a little confusion if you remove my son's email address (the multipie.co.uk one) so that he doesn't think the log is coming from the sky (as many do).

If memory is the problem then something that might help is to change CC's settings to use the smallest cover image possible. Because of Android 2.3 problems with managing memory used by images, making the images smaller can reduce memory requirements. You change the cover size by going to Settings / Covers and thumbnails / Downloaded cover size in pixels. Pick 160, then connect as a wireless device and let the sync complete.

Thanks Chaley. As you can see in my post above, my problem actually seems to be the names getting changed to a single (same) letter so I'm getting the same filenames for different books. I've tried changing the name in Calibre but for some reason these books keep getting the same name so I may switch to the setting for the book ID as filename instead.

It's good to know about the memory and cover images though - I've been slightly curious why some of my covers wouldn't show up until I restarted my device. The memory issue is probably the reason in that case.

[chaley]

Originally Posted by Demingy

Yep, I managed to figure out that this was my problem when I looked closer at the metadata info in CC - the names were shortened to one letter making them the same filename.

Unfortunately I haven't been able to fix it yet. I updated the names in Calibre, but CC keeps calling them "S" (.mobi and .azw3) so it kicks out the "duplicate". I've tried reloading the books in Calibre, updating the name, reloading in CC, and they keep getting saved as "S" (.mobi and .azw3).

I thought about changing the settings to use the book ID as a filename instead, but I started with this setting and my searches in the Fire didn't work properly for some reason. I may have to switch back to that though. Hopefully this helps someone else who runs into this problem as well, I couldn't find anything about it when I searched.

Are you sending these books using the wireless device connection? If so, then the file name is controlled by calibre. You can see the template that calibre uses if you connect then click on the down arrow on the device icon and choose "Configure this device". Normally it is something like title - author.

Note that if you change the template in Preferences / Send to device then that template might take precedence. Check it as well.

EDIT: using the UUID as the file name will probably fix things, but it would be good to know why the templates are producing names of one letter.

[Demingy]

Originally Posted by chaley

Are you sending these books using the wireless device connection? If so, then the file name is controlled by calibre. You can see the template that calibre uses if you connect then click on the down arrow on the device icon and choose "Configure this device". Normally it is something like title - author.

Note that if you change the template in Preferences / Send to device then that template might take precedence. Check it as well.

EDIT: using the UUID as the file name will probably fix things, but it would be good to know why the templates are producing names of one letter.

Yep, using the wireless device connection. Looking at the configuration now: check sub directories is checked, save template is {title} - {authors} ({id}), replace books with same calibre ID is checked as well as use metadata cache.

When I first noticed the problem I noticed that the books had the same "S" only name in the Calibre filenames so I had the filenames updated to the same template as all my other books. I'm not sure why Calibre was still sending the wrong filename to CC because I had deleted the files from CC (using Calibre) before renaming them and even deleted the existing mobi/azw3 files and reconverted the epubs to mobi after changing the filenames and then restarted Calibre. I even double checked the filenames in the containing folder after restarting Calibre before trying to transfer them back to CC. But the filenames were still "S" in CC so Calibre was still sending the wrong info.

I changed the settings to use the UUID instead and that seems to have fixed things, but I agree that it would be nice to know why the wrong filenames persisted in the transfer.

[Purple Lady]

Originally Posted by Demingy

Yep, I managed to figure out that this was my problem when I looked closer at the metadata info in CC - the names were shortened to one letter making them the same filename.

Unfortunately I haven't been able to fix it yet. I updated the names in Calibre, but CC keeps calling them "S" (.mobi and .azw3) so it kicks out the "duplicate". I've tried reloading the books in Calibre, updating the name, reloading in CC, and they keep getting saved as "S" (.mobi and .azw3).

I thought about changing the settings to use the book ID as a filename instead, but I started with this setting and my searches in the Fire didn't work properly for some reason. I may have to switch back to that though. Hopefully this helps someone else who runs into this problem as well, I couldn't find anything about it when I searched.

My problem was for books that have multiple authors - I had author as part of the folder name and as part of the book name. I changed it to use the first author name only. I think I used this in the template - {authors:list_item(0,&)}

[chaley]

Originally Posted by Demingy

Yep, using the wireless device connection. Looking at the configuration now: check sub directories is checked, save template is {title} - {authors} ({id}), replace books with same calibre ID is checked as well as use metadata cache.

When I first noticed the problem I noticed that the books had the same "S" only name in the Calibre filenames so I had the filenames updated to the same template as all my other books. I'm not sure why Calibre was still sending the wrong filename to CC because I had deleted the files from CC (using Calibre) before renaming them and even deleted the existing mobi/azw3 files and reconverted the epubs to mobi after changing the filenames and then restarted Calibre. I even double checked the filenames in the containing folder after restarting Calibre before trying to transfer them back to CC. But the filenames were still "S" in CC so Calibre was still sending the wrong info.

I changed the settings to use the UUID instead and that seems to have fixed things, but I agree that it would be nice to know why the wrong filenames persisted in the transfer.

Try unchecking the "Replace books with same calibre id". If that is set then if a book is resent the template is ignored and the original file name is used. That means that if the original name was "S" for some reason (an incorrect template at some point?) then it is stuck with that name even if it conflicts with some other book.

[Demingy]

Originally Posted by Purple Lady

My problem was for books that have multiple authors - I had author as part of the folder name and as part of the book name. I changed it to use the first author name only. I think I used this in the template - {authors:list_item(0,&)}

Thanks for the code - I'm going to implement it in my Calibre preferences. I do have other books with filenames that are the same length in Calibre that transferred without shortening the filename, but the books in question do have a laundry list of authors and I'm not sure if the filenames in Calibre are exactly the same as what it sends to CC - I only checked the filename length as it is in the Calibre filename, I didn't check to see what the difference in length might be if I included ALL author names. Maybe Calibre truncates the name in its own naming system but feeds the full name to CC without truncation.

[Demingy]

Originally Posted by chaley

Try unchecking the "Replace books with same calibre id". If that is set then if a book is resent the template is ignored and the original file name is used. That means that if the original name was "S" for some reason (an incorrect template at some point?) then it is stuck with that name even if it conflicts with some other book.

Perfect, I'll change that as well as the Calibre naming template to only use the first author name. Those changes should definitely prevent this happening again.

Thank you so much for all your help!

[chaley]

Originally Posted by Demingy

Thanks for the code - I'm going to implement it in my Calibre preferences. I do have other books with filenames that are the same length in Calibre that transferred without shortening the filename, but the books in question do have a laundry list of authors and I'm not sure if the filenames in Calibre are exactly the same as what it sends to CC - I only checked the filename length as it is in the Calibre filename, I didn't check to see what the difference in length might be if I included ALL author names. Maybe Calibre truncates the name in its own naming system but feeds the full name to CC without truncation.

CC uses the name supplied by calibre.

Calibre truncates the name if necessary so that the total length is less that approximately 200 characters. The 'truncation' is taken from the middle of the name.

If the {id} is in the template at the end then there is almost zero probability that the file names will conflict. The only way it can happen is if the same author and title exist in two different calibre libraries and have the same calibre library id.

[Demingy]

Originally Posted by chaley

CC uses the name supplied by calibre.

Calibre truncates the name if necessary so that the total length is less that approximately 200 characters. The 'truncation' is taken from the middle of the name.

If the {id} is in the template at the end then there is almost zero probability that the file names will conflict. The only way it can happen is if the same author and title exist in two different calibre libraries and have the same calibre library id.

That is good to know. I have no idea how either Calibre or CC work in many ways (just that they work very well 99+% of the time) so when I do have odd things pop up I find myself baffled. I've been reading up as much as possible to understand both programs better.

Not sure where the same short filenames came from or why they persisted in the transfer info from Calibre even after they were changed, implementing the changes I learned from you guys helped me get them onto my Kindle so I am a very happy camper.

And I can't really complain about three oddball files when I have 4700+ books organized by Calibre - those are very good percentages.