Calibre only sending some books to device

[raenedaz]

I have no idea what is going on, but Calibre is not sending all of the books to my device.

I am trying to get my library on my prs600. I finally have everything working except this final step. I've tried multiple times to send my library to my device, but only wind up with less than half actually making it to my reader.

I have 2000 books I'm sending. Two of these I know don't go due to file type and drm. The rest are either .lrf, .rtf, or .pdf. I've kept Calibre updated to it's newest version. My reader firmware is the latest version. Calibre shows my reader and memory card fine.

When I send to device (defaulted to memory card), Calibre says it has sent all but those two books. When I open my card or reader, there are only a little over 700 books on there.

Any ideas as to why this is happening and how to fix it?

[edbro]

Wow, that's a lot of books. Can I ask why you need 2,000 books on your device? Do you know how slow it will be to browse that many?

I don't know for sure but you are might be running into a limitation of the card's file system. If it is formatted as FAT (or FAT16) then I believe you can only have 512 entries in the root folder.

Files with 13 or more characters will use 3 (three) or more directory entries, this applies to FAT16 or FAT32

[raenedaz]

Originally Posted by edbro

Wow, that's a lot of books. Can I ask why you need 2,000 books on your device? Do you know how slow it will be to browse that many?

I don't know for sure but you are might be running into a limitation of the card's file system. If it is formatted as FAT (or FAT16) then I believe you can only have 512 entries in the root folder.

Files with 13 or more characters will use 3 (three) or more directory entries, this applies to FAT16 or FAT32

I keep all of my books on my reader's sd card. 16GB, the files can be copied to it fine from the calibre library folder,so I don't see why I should be stuck somewhere finishing one book and deciding I feel like starting one I haven't got on my device. I have varying tastes in reading, depending on my mood, and frequently like to reread books.

After determining that I could just transfer my files to the card through windows explorer and have them all show on my reader, the only reason I'm still trying to get Calibre to send them there is because I prefer to find books by collection, not author or title.

I will have to research what you're saying, though. I'm a tech idiot, so what do you mean by "directory entries"? Is the file name what I've named it before sending to Calibre or what Calibre has it named as in its library folder? Yes, they are different. My files are sent to Calibre with author - series # - title. In the Calibre library, they show up as author - title. I assume the metadata folder contains the series info. Is each folder within the library also considered a file?

[edbro]

If your card is 16GB then you have to be using FAT32 or NTFS. So, the file count concern is not valid. I don't know why Calibre is not transferring them all.

I've never tried that many entries on my card but I've read others say that it will take the device hours to read that many books on startup.

[raenedaz]

Right. I'm ok with the initial opening delay. It's annoying but worth it to not have to struggle with Sony's library. The time to send the files to the device is lengthy, which is the time-consuming part of the ordeal. However, I can watch a couple of movies while making sure my computer stays awake while doing this.

I have tried this in batches of 200-400 also, and still don't get all of the books transferred.

My assumption is that I'm doing something wrong. Others on here have large libraries and have transferred their files to their cards without an issue.

When I get home, I will try using windows explorer to load the Calibre onto my card, then use Calibre to try and send to device. We'll see what happens with that. There's got to be something blocking them from being put on the card. As I said, Calibre says it sent them ok, but they don't show up.

Would my card make a difference? It's an HP SDHC. My understanding is that it shouldn't since I can send the files there through windows explorer and have them load on the reader. I just can't get them to show up if Calibre "sends" them there.

[troymc]

Originally Posted by edbro

Wow, that's a lot of books. Can I ask why you need 2,000 books on your device? Do you know how slow it will be to browse that many?

Ack...not this same old silliness again. WARNING: I'm working on putting all of Project Gutenberg's 30,000 books on my 8GB SD card to use in my 505 -- so relax, don't panic. Different people use their readers differently than you.

Originally Posted by edbro

I don't know for sure but you are might be running into a limitation of the card's file system. If it is formatted as FAT (or FAT16) then I believe you can only have 512 entries in the root folder.

Files with 13 or more characters will use 3 (three) or more directory entries, this applies to FAT16 or FAT32

Just to clarify what you said: FAT16 has a 512 directory entry limit - FAT32 does not.

Both FAT16 & FAT32 use 1 directory entry for each 8.3 file/directory name + 1 directory entry for every 13 characters beyond that.

eg.

a_long_filename.txt should use 2 directory entries
wow_this_is_long_filename.txt should use 3 directory entries

From what I understand FAT32 has a limit of 64k directory entries, although Microsoft seems to think there is no limit:

The root folder on a FAT32 drive is now an ordinary cluster chain, so it can be located anywhere on the volume. For this reason, FAT32 does not restrict the number of entries in the root folder.

Originally Posted by raenedaz

My assumption is that I'm doing something wrong. Others on here have large libraries and have transferred their files to their cards without an issue.

When I get home, I will try using windows explorer to load the Calibre onto my card, then use Calibre to try and send to device. We'll see what happens with that. There's got to be something blocking them from being put on the card. As I said, Calibre says it sent them ok, but they don't show up.

Would my card make a difference? It's an HP SDHC. My understanding is that it shouldn't since I can send the files there through windows explorer and have them load on the reader. I just can't get them to show up if Calibre "sends" them there.

I have well over 3000 books on my 2GB SD card in my 505 and have never had an issue - although I've never tried sending them all at once.

2 ideas:

1. Writes are buffered and you are ejecting/unplugging your reader before everything is flushed to disk. Calibre will say it's complete long before the OS has finished writing everything to disk. Are you ejecting your reader thru Calibre? or just unplugging it? You should always eject thru Calibre & wait until it finishes writing before unplugging it.
2. If #1 is not it, then I suggest starting calibre from the command line. This may allow you to see any error messages that may be hidden behind the gui.

Troy

[edbro]

Originally Posted by troymc

Ack...not this same old silliness again. WARNING: I'm working on putting all of Project Gutenberg's 30,000 books on my 8GB SD card to use in my 505 -- so relax, don't panic. Different people use their readers differently than you.

Dang Troymc, I guess we all can't be as cool and smart as you. Sorry for being so "silly" And don't worry, I'm no longer panicking.

[theducks]

Originally Posted by raenedaz

I have no idea what is going on, but Calibre is not sending all of the books to my device.

I am trying to get my library on my prs600. I finally have everything working except this final step. I've tried multiple times to send my library to my device, but only wind up with less than half actually making it to my reader.


When I send to device (defaulted to memory card), Calibre says it has sent all but those two books. When I open my card or reader, there are only a little over 700 books on there.

Any ideas as to why this is happening and how to fix it?

Had you disabled some formats in the Sony 600 Device Interface Plugin and forgotten ?

[raenedaz]

Originally Posted by troymc

I have well over 3000 books on my 2GB SD card in my 505 and have never had an issue - although I've never tried sending them all at once.

2 ideas:

1. Writes are buffered and you are ejecting/unplugging your reader before everything is flushed to disk. Calibre will say it's complete long before the OS has finished writing everything to disk. Are you ejecting your reader thru Calibre? or just unplugging it? You should always eject thru Calibre & wait until it finishes writing before unplugging it.
2. If #1 is not it, then I suggest starting calibre from the command line. This may allow you to see any error messages that may be hidden behind the gui.

Troy

Your first idea, yeah, just figured out last night I can eject it in Calibre, as opposed to my normal unplug. How do I know if it's finished running? The jobs list finishes.

I've read in here about running it from the command line. This sound like DOS. I have no idea how to do this.

[Stinger]

By default in Windows, any USB connected device has write-caching disabled. I've never had any problems just waiting until the Calibre jobs icons stop spinning and unplugging without using the "eject" feature.

If you open device manager and go to the properties of the Kobo, you can verity that write-caching is indeed disabled.

[troymc]

Originally Posted by raenedaz

Your first idea, yeah, just figured out last night I can eject it in Calibre, as opposed to my normal unplug. How do I know if it's finished running? The jobs list finishes.

Originally Posted by Stinger

By default in Windows, any USB connected device has write-caching disabled. I've never had any problems just waiting until the Calibre jobs icons stop spinning and unplugging without using the "eject" feature.

If you open device manager and go to the properties of the Kobo, you can verity that write-caching is indeed disabled.

Ok, a disabled write cache makes my first idea moot.

Originally Posted by raenedaz

I've read in here about running it from the command line. This sound like DOS. I have no idea how to do this.

I'm a unix guy, so I'm not completely sure myself, but it should be something like:

1. open a command prompt (start -> run -> cmd)
2. navigate to your Calibre install (cd \Program Files\???)
3. start Calibre

Then watch that command window when you try to send your books to your reader.

(didn't there use to be an option to right-click on an icon, select properties, & then tell it to run the process in a command line?)


Troy

[raenedaz]

I've tried doing the command prompt. I can't get it to show me anything while Calibre is running. As I said, I'm a tech idiot and do not know DOS at all so I'm still trying to figure out how do this correctly.

I've tried just sending 91 books. Only 63 follow through. When I try to add the books it left out, a couple at a time, it'll send one of them there but not the other. Then I go back and try to add the other one. It removes the last one I added and puts in the new one.

I've checked to make sure I have no blocks in the preferences.

Still trying. Not getting anywhere, yet.

Currently have Calibre library copied to my card through Windows Explorer and am seeing if Calibre will "send" them all to the device this way.

[DoctorOhh]

Originally Posted by raenedaz

I've tried just sending 91 books. Only 63 follow through. When I try to add the books it left out, a couple at a time, it'll send one of them there but not the other. Then I go back and try to add the other one. It removes the last one I added and puts in the new one.

Are you trying to send 2 or 3 different formats (rtf, lrf, pdf, epub) of the same book to your reader?

[raenedaz]

No. I have before, but that just sends both formats to the reader. I was talking about different books, say Stephen King's It and Robert Frost poems.
Both same format.


Would the usb driver have anything to do with this? I noticed last night that my usb cable driver is prs500. I have been looking to see if there's a cable driver for prs600, but not finding anything yet.

[kovidgoyal]

One way to get debug output is to start calibre as

calibre-debug -g
in a terminal and see if it prints out anything useful during the send.