Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre

Notices

Reply
 
Thread Tools Search this Thread
Old 11-30-2010, 04:11 AM   #16
Manichean
Wizard
Manichean My eyes! My eyes! The light is just too bright!Manichean My eyes! My eyes! The light is just too bright!Manichean My eyes! My eyes! The light is just too bright!Manichean My eyes! My eyes! The light is just too bright!Manichean My eyes! My eyes! The light is just too bright!Manichean My eyes! My eyes! The light is just too bright!Manichean My eyes! My eyes! The light is just too bright!Manichean My eyes! My eyes! The light is just too bright!Manichean My eyes! My eyes! The light is just too bright!Manichean My eyes! My eyes! The light is just too bright!Manichean My eyes! My eyes! The light is just too bright!
 
Manichean's Avatar
 
Posts: 3,130
Karma: 80446
Join Date: Feb 2008
Location: Germany
Device: Cybook Gen3
Create a custom column and update it with whatever device path information you like... Device sync, from what I remember, ain't gonna happen anytime soon, if at all.
As for the not-overwriting-series-stuff, finegrained metadata import control is on the todo list. That said, it's going to arrive When It's Done.
Manichean is offline   Reply With Quote
Old 11-30-2010, 10:36 AM   #17
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 25,622
Karma: 4998489
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
device sync is not going to happen. There's no way to make it work reliably, especially in software like calibre which relies on no support from the device firmware. Just look at all the problems that sync causes in the SONY software, which does have support from the device firmware.

What's the point of seeing device path while the device is not connected? And since you can connect multiple devices all of which might have the same book on them, this is not a natural fit.
kovidgoyal is offline   Reply With Quote
Old 11-30-2010, 01:08 PM   #18
mornington
Connoisseur
mornington will become famous soon enoughmornington will become famous soon enoughmornington will become famous soon enoughmornington will become famous soon enoughmornington will become famous soon enoughmornington will become famous soon enoughmornington will become famous soon enough
 
Posts: 63
Karma: 732
Join Date: Nov 2010
Device: Sony PRS-650
Quote:
Originally Posted by kovidgoyal View Post
device sync is not going to happen. There's no way to make it work reliably, especially in software like calibre which relies on no support from the device firmware. Just look at all the problems that sync causes in the SONY software, which does have support from the device firmware. .
I assumed as much, but note that what I'm talking about isn't a true device *sync*, but something rather more low-key than that: Relocating a file on the device. The logic would be:

Calibre saw some file on the device last time the device was connected but can't see it now. It *can* however see files on the device which are apparently not in the library. So before deciding that the file in the library isn't on the device it attempts to match the library file with the ones it's just found on the library. If it finds a match, it updates its record of where that file lives on the device.

This solves the issue of the user moving a file to a different location on the device outside of calibre or changing the key metadata - title or author, say - in Calibre *after* sending a file to the device. At present, calibre seems to lose track of the file (particularly evident if I change the title of a book). With this change, it wouldn't.

Quote:
Originally Posted by kovidgoyal View Post
What's the point of seeing device path while the device is not connected? And since you can connect multiple devices all of which might have the same book on them, this is not a natural fit.
This was just a suggested quick-fix for the above problem: Making the path-to-file-on-device a visible and editable text field would allow the user to update that info in calibre "manually" if, for example, the name of the folder the file lives in on the device is changed for whatever reason (eg to create Kindle collections from folders using the KCM tool).

At present, that requires me to backup the kindle metadata files for each book, remove the books from the device in calibre, send them back to the device in calibre (to the new location) and then restore the kindle metadata files to the correct location and filename on my kindle.

With the matching code described above I'd just change the folder name and calibre would find the files automatically in their new location. Or with the visible-and-editable path-to-file-on-device field I could just do a search and replace on that field to change the old folder name to the new one. Less convenient than the matching approach, but much more convenient than what I have to do now.
mornington is offline   Reply With Quote
Old 11-30-2010, 01:18 PM   #19
mornington
Connoisseur
mornington will become famous soon enoughmornington will become famous soon enoughmornington will become famous soon enoughmornington will become famous soon enoughmornington will become famous soon enoughmornington will become famous soon enoughmornington will become famous soon enough
 
Posts: 63
Karma: 732
Join Date: Nov 2010
Device: Sony PRS-650
Actually, come to think of it, the file-matching process can have even simpler logic:

Assume that Calibre keeps a note that a given file was on a given device last time it was connected to that device. Now, that device is connected and that file is still in calibre's library but isn't seen on the device (due to a different title/author/file path or name or whatever calibre keys on). That device also contains one or more files which are not in calibre's library.

Match the first group of files (in library, not on device) against the second ground (on device, not in library) using actual file data (CRC, say). If a match is found, link the library entry to the device entry.

That should do it.
mornington is offline   Reply With Quote
Old 11-30-2010, 02:01 PM   #20
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 25,622
Karma: 4998489
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
That is conceivable, but I am going to defer to chaley, who wrote and maintains the book on device matching code.
kovidgoyal is offline   Reply With Quote
Old 11-30-2010, 03:23 PM   #21
chaley
"chaley", not "charley"
chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.
 
Posts: 5,193
Karma: 821512
Join Date: Jan 2010
Location: France
Device: Many android devices
I confess to confusion, because I think that what you are asking for already happens in many cases.

Calibre stores information about the books on a device in a file that is stored on the device. When you connect, it retrieves this information and compares it with the book files actually existing on the device. Books no longer there are removed. Books that are added are scanned to get their metadata.

The next thing that calibre does is attempt to match the books actually on the device with those in the library. A book matches if it has the same title/author (found in the book's metadata if calibre options permit that) or if it has the same unique ID (EPUB supports this, but I don't know if other formats do). Thus, a moved book will (or should) still be matched with the book in calibre. It is even possible that the same book could be stored multiple times on the device, and calibre will find these.

Clearly calibre can't arbitrarily decide that some random book on the device matches a book in the library. There must be some information in the book that calibre can use, and I think that calibre currently uses that information.

How does what calibre does today differ from what you are suggesting?
chaley is offline   Reply With Quote
Old 11-30-2010, 05:00 PM   #22
Worldwalker
Curmudgeon
Worldwalker ought to be getting tired of karma fortunes by now.Worldwalker ought to be getting tired of karma fortunes by now.Worldwalker ought to be getting tired of karma fortunes by now.Worldwalker ought to be getting tired of karma fortunes by now.Worldwalker ought to be getting tired of karma fortunes by now.Worldwalker ought to be getting tired of karma fortunes by now.Worldwalker ought to be getting tired of karma fortunes by now.Worldwalker ought to be getting tired of karma fortunes by now.Worldwalker ought to be getting tired of karma fortunes by now.Worldwalker ought to be getting tired of karma fortunes by now.Worldwalker ought to be getting tired of karma fortunes by now.
 
Posts: 3,087
Karma: 722357
Join Date: Feb 2010
Device: PRS-505
Remember too that calibre not only has to be able to support every device out there, but it has to support multiple devices, often multiple unrelated devices, per user. There are users with several devices -- a pocket reader and a tablet, for instance, with different books on them -- and there are families who all use the same computer and copy of calibre, with at least one device each. The location of a book on one person's Aluratek Libre is not going to be the same as the location of that book on his wife's Sony Reader, and trying to correlate the two would probably break calibre's wee little silicon brain. When you add in devices which allow their users to rearrange their files, it goes from merely hairy to covered in dense fur and chasing you through the woods; even two outwardly identical devices might have totally different book organizations.

It's one of those things that would be nice, but probably can't be made to work with the userbase calibre has to support.
Worldwalker is offline   Reply With Quote
Old 12-01-2010, 12:47 AM   #23
mornington
Connoisseur
mornington will become famous soon enoughmornington will become famous soon enoughmornington will become famous soon enoughmornington will become famous soon enoughmornington will become famous soon enoughmornington will become famous soon enoughmornington will become famous soon enough
 
Posts: 63
Karma: 732
Join Date: Nov 2010
Device: Sony PRS-650
Quote:
Originally Posted by chaley View Post
I confess to confusion, because I think that what you are asking for already happens in many cases.

Calibre stores information about the books on a device in a file that is stored on the device. When you connect, it retrieves this information and compares it with the book files actually existing on the device. Books no longer there are removed. Books that are added are scanned to get their metadata.
[...]
How does what calibre does today differ from what you are suggesting?
There's the key part: Matching is done by metadata. So if I change the title, say, of the book in calibre then it loses that connection between the device-side book and the calibre-side book.

What I'm proposing is that when that happens (in the scenario I gave earlier) Calibre would take one further step and attempt to match by file CRCs to reconcile those "orphaned" items.
mornington is offline   Reply With Quote
Old 12-01-2010, 01:34 AM   #24
mornington
Connoisseur
mornington will become famous soon enoughmornington will become famous soon enoughmornington will become famous soon enoughmornington will become famous soon enoughmornington will become famous soon enoughmornington will become famous soon enoughmornington will become famous soon enough
 
Posts: 63
Karma: 732
Join Date: Nov 2010
Device: Sony PRS-650
To take an example:

1) Send to device some book, putting it in folder "foo"
2) Change the folder name (in windows explorer) of folder "foo" to "bar"
3) Change the title of the book in calibre
4) Disconnect and reconnect the device

As far as I can tell, if I do steps (2) OR (3) then calibre seems to be able to detect the book on the device and link it to the library record ok.

BUT if I do steps (2) AND (3) then I end up with a calibre record which thinks the book isn't on the device *and* a book on the device which thinks it's not in calibre's library. Comparing the two files (the one in calibre and the one on the device) in this circumstance should verify that they're the same book.

(using kindle 3/firmware 3.0.3 with vista x64, incidentally, in case that's important)
mornington is offline   Reply With Quote
Old 12-01-2010, 01:46 AM   #25
DoctorOhh
US Navy, Retired
DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.
 
DoctorOhh's Avatar
 
Posts: 8,800
Karma: 12528001
Join Date: Feb 2009
Location: North Carolina
Device: Nexus 7
Quote:
Originally Posted by mornington View Post
What I'm proposing is that when that happens (in the scenario I gave earlier) Calibre would take one further step and attempt to match by file CRCs to reconcile those "orphaned" items.
Unless I'm mistaken the CRC of the file on the device is often going to be different then the CRC of the file in calibre's library. Upon using send to device the file being sent is updated by adding/changing metadata and sometimes a cover to the file prior to sending the file. This means that the CRC of the on device file and the in library file will be different.
DoctorOhh is offline   Reply With Quote
Old 12-01-2010, 02:49 AM   #26
chaley
"chaley", not "charley"
chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.
 
Posts: 5,193
Karma: 821512
Join Date: Jan 2010
Location: France
Device: Many android devices
Quote:
Originally Posted by dwanthny View Post
Unless I'm mistaken the CRC of the file on the device is often going to be different then the CRC of the file in calibre's library. Upon using send to device the file being sent is updated by adding/changing metadata and sometimes a cover to the file prior to sending the file. This means that the CRC of the on device file and the in library file will be different.
You are correct. When a book is sent to the device, various metadata fields are changed. It is practically a certainty that the books will not be bit-wise identical.

Looking at the code, I see that .mobi does not support storing the UUID in the book, or at least that calibre does not store it. That removes the other option, matching by calibre's id.

Bottom line: unless the UUID can be put into the book somewhere, or unless you can use EPUB, there isn't much I can do.
chaley is offline   Reply With Quote
Old 12-01-2010, 04:17 AM   #27
mornington
Connoisseur
mornington will become famous soon enoughmornington will become famous soon enoughmornington will become famous soon enoughmornington will become famous soon enoughmornington will become famous soon enoughmornington will become famous soon enoughmornington will become famous soon enough
 
Posts: 63
Karma: 732
Join Date: Nov 2010
Device: Sony PRS-650
Quote:
Originally Posted by chaley View Post
You are correct. When a book is sent to the device, various metadata fields are changed. It is practically a certainty that the books will not be bit-wise identical.
Is there anything to prevent obtaining a file signature (CRC or otherwise) when the file is transferred to the device? i.e. send to device then obtain CRC/whatever from the file as it now is on the device.
mornington is offline   Reply With Quote
Old 12-01-2010, 04:29 AM   #28
chaley
"chaley", not "charley"
chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.
 
Posts: 5,193
Karma: 821512
Join Date: Jan 2010
Location: France
Device: Many android devices
Quote:
Originally Posted by mornington View Post
Is there anything to prevent obtaining a file signature (CRC or otherwise) when the file is transferred to the device? i.e. send to device then obtain CRC/whatever from the file as it now is on the device.
There are several issues:

1. The same calibre book can be sent to the device multiple times. Each will have a different signature. Which one is stored?
2. Computing the signature takes time.
3. Many people have multiple devices. Which one wins?

I suppose that one could address concern #3 (and possibly #1) by computing a signature and storing it in the calibre metadata cache file on the device. One would then check new files against information in the cache, correcting the path information if a signature match is found. Reliability and performance would be a concern. However, doing this work is of no interest to me. I don't have a kindle, and my sonys and smartphones use EPUB, which already does it via the UUID.

Perhaps you could do the work, or perhaps someone else with a kindle might take it on.
chaley is offline   Reply With Quote
Old 12-01-2010, 04:37 AM   #29
Lady Fitzgerald
Wizard
Lady Fitzgerald ought to be getting tired of karma fortunes by now.Lady Fitzgerald ought to be getting tired of karma fortunes by now.Lady Fitzgerald ought to be getting tired of karma fortunes by now.Lady Fitzgerald ought to be getting tired of karma fortunes by now.Lady Fitzgerald ought to be getting tired of karma fortunes by now.Lady Fitzgerald ought to be getting tired of karma fortunes by now.Lady Fitzgerald ought to be getting tired of karma fortunes by now.Lady Fitzgerald ought to be getting tired of karma fortunes by now.Lady Fitzgerald ought to be getting tired of karma fortunes by now.Lady Fitzgerald ought to be getting tired of karma fortunes by now.Lady Fitzgerald ought to be getting tired of karma fortunes by now.
 
Lady Fitzgerald's Avatar
 
Posts: 2,013
Karma: 251649
Join Date: Apr 2010
Location: Tempe, AZ, USA, Earth
Device: JetBook Lite (away from home) + 1 spare, 32" TV (at home)
Quote:
Originally Posted by Worldwalker View Post
...When you add in devices which allow their users to rearrange their files, it goes from merely hairy to covered in dense fur and chasing you through the woods...
Snort...choke...gasp... I was already primed when I saw the foo and bar examples so this had me laughing hysterically. Thanks for the laugh! You realize I'm going to steal that line?
Lady Fitzgerald is offline   Reply With Quote
Old 12-01-2010, 06:15 AM   #30
mornington
Connoisseur
mornington will become famous soon enoughmornington will become famous soon enoughmornington will become famous soon enoughmornington will become famous soon enoughmornington will become famous soon enoughmornington will become famous soon enoughmornington will become famous soon enough
 
Posts: 63
Karma: 732
Join Date: Nov 2010
Device: Sony PRS-650
Quote:
Originally Posted by chaley View Post
There are several issues:

1. The same calibre book can be sent to the device multiple times. Each will have a different signature. Which one is stored?
2. Computing the signature takes time.
3. Many people have multiple devices. Which one wins?
Actually, I was surprised that Calibre doesn't store information about where each edition of a book is kept. So that you could, for example, ask calibre for a list of the books on each device you own without having to plug in the devices in turn.

If it did then that would be the natural place to keep this data - i.e a table with device-file associations - but since it doesn't then, yeah, this might be more work than it's worth.

Thankfully, I'm still at early stages with my shiny new kindle, so it's not *too* much trouble to experiment with different ways of organising the books and then just level the whole thing and rebuild the entire kindle content in my finally-decided form at the end.

In fact, that's my current approach, and will remain doable right up until the point that I have files in multiple collections (as that's one thing it's apparently not possible to automate as yet).
mornington is offline   Reply With Quote
Reply

Tags
add, calibre, columns, feature request, import

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Feature request / Good idea for future versions of Calibre rahulm Calibre 9 07-18-2010 06:35 PM
[Old Thread] Calibre 'feature request' thread Waba Calibre 2 02-10-2010 07:52 PM
Additional Columns dvs0826 Calibre 3 08-22-2009 06:43 PM
Feature request AprilHare Calibre 0 11-22-2008 05:42 AM
Calibre Feature request: broader search of ISBNDB edembowski Calibre 4 07-11-2008 06:44 PM


All times are GMT -4. The time now is 04:17 PM.


MobileRead.com is a privately owned, operated and funded community.