How to stop collections from disappearing?

[June]

Originally Posted by June

Yes - thank you! I've tested adding/deleting/editing my books and everything works as expected. Except that I added 'tags' back to the device plugin, and when sending over a new book that is not on the 505 already - it shows up on the 505 without being in a collection. - - -

Originally Posted by dwanthny

- - -creating any automatic collections you have to go to Preferences - Plugins - Device Interface plugin - select your plugin and click customize.- - -

Well... It seems I spoke too soon.
I just now transferred a few new books to my 505, on the same computer and with the same settings as before (device interface plugin set to use 'Tags' to create collections). AND.... all my collections disappeared.

Calibre library shows the books as having Tags, plugin is set to create collections from tags. The books I sent to the 505 did not end up in a collection, and to top it off, all my previously created collections disappeared - my 505 now shows '258 books' and 'No collection' ---again--- I'm going to hold off catogorizing the books for now, getting tired of it

The only differences to my previous test is that before I just sent one book at a time in EPUB and PDF, now I sent three books (2+1) same formats (one EPUB and two PDF). And I'd changed the word 'tags' to 'Tags' (capitalized) to see if it would create collection when sending book, since 'tags' didn't work, as I described above.

I still have Calibre running, in case someone here knows if there is a logfile or something similar somewhere, that I could send someone, if it helps figure out what is the problem?

On a positive note: the pdf-ligatures (double L) seems to work nicely now.

[chaley]

Originally Posted by June

Well... It seems I spoke too soon. [...]

*&^". I wish this would happen to me. Even just once.

There is no long available at this point, unless you ran calibre with the command calibre-debug -g. If you are willing, you might want to do that in the future, just so the device debug log is available. Using this command does not require running from source or anything like that. Open the equivalent of a terminal (command box or whatever) and run the command. Calibre will talk to you on that screen.

Question: have you plugged in your device since collections worked? If so, have you added books and had it work? Or is this the first time for either? I have no idea what is happening, and hope that I can narrow the search a bit.

Edit: are all the books in main memory, or are some on a card?

[June]

I'll run calibre in debug mode tonight, and try stuff. I assume the log will be somewhere in the calibre directory?

I tried the collections/tags the other day adding a few books and removing some, and ejected the device via Calibre as usual. The collestions stayed put then. The 505 has been working normally.

Today I plugged it in again, I don't think it took Calibre a long time to find the device, so I *doubt* it deleted the collectiosn on plugin (but I did not check this). I sent books to the 505, which took maybe a touch longer than expected - but again, I didn't time it. I *think* the collections disappeared in connection to sending books.

All the books are in main memory, and I have about 49MB free space still, so it shouldn't be a space issue.

FWIW, this is a fairly new 505 - received it the other week after my old one "semi-died". This one has UK firmware. When I received it I first customized it with prscust and installed all my books via calibre 0.6.5x. I also have another 505 new in box (with german firmware), that is un-customized. I'll try sending books to that one too, just in case the devices behave differently.

[chaley]

Thank you for your willingness to help.

Originally Posted by June

I'll run calibre in debug mode tonight, and try stuff. I assume the log will be somewhere in the calibre directory?

Do I remember correctly that you are running some variant of Linux? Not that it matters, really, except that I don't want to keep saying things you already know.

The debug output is written to the terminal/cmd box. Most of the interesting stuff is to stdout (standard output), but some is to stderr (standard error). On all the OSes that calibre runs on, the command
calibre-debug -g >somefile 2>&1
without the quotes will send all the debug output to 'somefile'. If you want to see it, and if you are running linux, you could pipe it through 'tee'.

[June]

I'm on WinXP MCE, but can also try on Win7 later.

[chaley]

Originally Posted by June

I'm on WinXP MCE, but can also try on Win7 later.

I might have a handle on this. Did you add something to your reader without using calibre to do it? Or by using a different installation of calibre? Or delete the file .calibre.metadata in the root of your device memory?

[June]

I don't remember adding anything without calibre - but there is a *sligth* chance that I did add a book when I deleted the collections via the Sony desktop software... I'm 95% sure I didn't....

I did just now test to add books and edit tags on the 505 via another computer with Calibre 0.6.45 in order to "sinulate upgrading Calibre". It didn't make collections disappear when I added & removed books on the main computer with 0.7.4..

I haven't touched the metadata file on the 505.

[June]

OK - I just tested hooking up the 505 and opening the Sony software, which shower the 505 with my 17 collections. I then added one book to the 505 via the Sony software, ejected reader.

Plugged 505 back in and opened Calibre 0.7.4. And behold - collections are gone!

So the culprit seems to be using the sony software in this case.

I ran Calibre in debug mode, do you want me to send you the file?

It's still odd though that Calibre doesn't create Collections for a book that has a tag and no collection on the device (with the plugin set to 'Tags')

[chaley]

Originally Posted by June

OK - I just tested hooking up the 505 and opening the Sony software, which shower the 505 with my 17 collections. I then added one book to the 505 via the Sony software, ejected reader.

Plugged 505 back in and opened Calibre 0.7.4. And behold - collections are gone!

I am very happy to hear this. Your and JvdW's previous comments led me to finding a bug that accounts for it. Briefly, if a book was added or changed on the device outside of calibre, the calibre cache was rebuilt. This triggered a fairly obscure code path where collections were rebuilt before they had been read from the sony database. As a result, they all went away. I have fixed this bug and submitted it to Kovid.

So the culprit seems to be using the sony software in this case.

Yes. Copying a book onto the device or deleting a book from the device with a file manager could also trigger it.

I ran Calibre in debug mode, do you want me to send you the file?

Yes. But see below...

It's still odd though that Calibre doesn't create Collections for a book that has a tag and no collection on the device (with the plugin set to 'Tags')

Is the document a PDF? Tags aren't stored in PDFs, so calibre can't reconstruct them from the book. The bug would prevent calibre from reconstructing them from history.

If the book isn't a PDF, then yes, I would like to see the logs.

[June]

I sent at least epub and pdf to the 505, maybe an lrf too. I'll send you the entire log by email

[chaley]

Originally Posted by June

I sent at least epub and pdf to the 505, maybe an lrf too. I'll send you the entire log by email

Thank you. I see from the log that you spent nearly 2 hours testing. I very much appreciate your time.

The tags-as-collections problem is that you wrote 'Tags' instead of 'tags' in the device customization. There is no database column 'Tags'.

Of course, this isn't your mistake. Calibre should be ignoring case. I will fix it. In the meantime, change the device customization to 'tags'

[June]

You're welcome - I enjoy problem solving, and was determined to reproduce the glitch

I did have 'tags' in the plugin earlier (as posted above) but changed it to capitalized 'Tags' to see if that made a difference. I based my decision on the fact that the name of the column is capitalized in the user interface... I'll change it back again, and I'm glad that the trial brought your attention to the issue.

Let me know if you need more testing done.