Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre

Notices

Reply
 
Thread Tools Search this Thread
Old 10-03-2008, 03:45 AM   #1
slushie
Junior Member
slushie began at the beginning.
 
Posts: 7
Karma: 10
Join Date: Sep 2008
Device: Sony PRS-505
Question lost metadata on pdfs

first, i have to say what a great suite of tools you've created with calibre! it is a wonderful testament to the power of open source software not to mention your endless patience. thanks!

i just got my prs505 a few days ago so i am still learning the ropes, so please bear with me if i misunderstand something...

in calibre i have a few books loaded and i've configured the automatic download of metadata via the gui. in the 'library' section this all looks perfect! however, when i send my library to the reader i am losing all my configured metadata... the reader sees whatever is A. inside the pdf's title/author slots (in acroread File>Properties dialog), or B. just the filename as title, with all spaces and non-alphanumeric chars turning into underscores and no author.

this problem only seems to happen with PDF's, and only when i transfer from my library to the hardware... if i click 'Card' and double-click an item in that list i can fix the metadata manually.

i'm running linux, just updated to the latest version of calibre (0.4.89). let me know if you need any more information, i'm happy to help you help me any way i can!

thank you!
slushie is offline   Reply With Quote
Old 10-03-2008, 11:16 AM   #2
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 25,442
Karma: 4961459
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
There was some change in the latest firmware for the 505 that causes it to always rescan books and update the metadata in the cache (the file media.xml) for books in the main memory. This doesn't matter for LRF since calibre sets the metadata correctly in LRF. It doesn't do that for PDF because there isn't a python library that can do that and writing one is non-trivial
kovidgoyal is offline   Reply With Quote
Old 10-03-2008, 02:17 PM   #3
slushie
Junior Member
slushie began at the beginning.
 
Posts: 7
Karma: 10
Join Date: Sep 2008
Device: Sony PRS-505
thanks for your quick response.

i see how updating the pdf would be the ideal solution, though nontrivial of course... still, would it be possible for calibre to set the metadata manually after the transfer -- i should say, emulating the action of me double-clicking each field and editing it by hand -- because that seems to update the information perfectly well for files on the card. i have been doing this myself but i have many pdfs and the reader loads pdf's *much* faster than lrf's, so i'd rather not convert them all.

also, i am not a python developer but am a programmer by trade. i would gladly submit a patch if you could give me a hint to where to find what i'm looking for in the source

thanks!
slushie is offline   Reply With Quote
Old 10-03-2008, 02:21 PM   #4
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 25,442
Karma: 4961459
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
calibre does set the metadata manually after an upload, it's just that ever since the latest firmware update, the reader overwrites the metadata set by calibre with metadata read from the file. This only happens for books in the main memory.

If you care enough, you can try to figure out why it happens. metadata is stored on the device in the file media.xml in the main memory and cache.xml on the storage card.

Transfer a book using calibre. Dont disconnect the reader. See media.xml. Now disconnect. Let it rescan the collection. Reconnect and compare the new media.xml to the old one.

If you want to submit a patch for editing PDF metadata, you should submit one to the pypdf project (it's what calibre uses for PDF metadata handling)
kovidgoyal is offline   Reply With Quote
Old 10-03-2008, 02:23 PM   #5
pilotbob
Grand Sorcerer
pilotbob ought to be getting tired of karma fortunes by now.pilotbob ought to be getting tired of karma fortunes by now.pilotbob ought to be getting tired of karma fortunes by now.pilotbob ought to be getting tired of karma fortunes by now.pilotbob ought to be getting tired of karma fortunes by now.pilotbob ought to be getting tired of karma fortunes by now.pilotbob ought to be getting tired of karma fortunes by now.pilotbob ought to be getting tired of karma fortunes by now.pilotbob ought to be getting tired of karma fortunes by now.pilotbob ought to be getting tired of karma fortunes by now.pilotbob ought to be getting tired of karma fortunes by now.
 
pilotbob's Avatar
 
Posts: 19,493
Karma: 11248282
Join Date: Jan 2007
Location: Tampa, FL USA
Device: Kindle Touch
Quote:
Originally Posted by kovidgoyal View Post
If you care enough, you can try to figure out why it happens. metadata is stored on the device in the file media.xml in the main memory and cache.xml on the storage card.
Can you flag the file as read-only somehow?

BOb
pilotbob is offline   Reply With Quote
Old 10-03-2008, 02:28 PM   #6
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 25,442
Karma: 4961459
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
Quote:
Originally Posted by pilotbob View Post
Can you flag the file as read-only somehow?

BOb
No, the reader needs to be able to edit the file for storing bookmarks, history pagination information etc
kovidgoyal is offline   Reply With Quote
Old 10-03-2008, 03:26 PM   #7
slushie
Junior Member
slushie began at the beginning.
 
Posts: 7
Karma: 10
Join Date: Sep 2008
Device: Sony PRS-505
Quote:
Originally Posted by kovidgoyal View Post
This only happens for books in the main memory.
i'm definitely seeing this issue on the SD card as well...

Quote:
Originally Posted by kovidgoyal View Post
If you care enough, you can try to figure out why it happens. metadata is stored on the device in the file media.xml in the main memory and cache.xml on the storage card.
i'm going to look into this today and see if i can begin to get a grasp of it.

thanks for the info!
slushie is offline   Reply With Quote
Old 10-03-2008, 06:32 PM   #8
=X=
Wizard
=X= ought to be getting tired of karma fortunes by now.=X= ought to be getting tired of karma fortunes by now.=X= ought to be getting tired of karma fortunes by now.=X= ought to be getting tired of karma fortunes by now.=X= ought to be getting tired of karma fortunes by now.=X= ought to be getting tired of karma fortunes by now.=X= ought to be getting tired of karma fortunes by now.=X= ought to be getting tired of karma fortunes by now.=X= ought to be getting tired of karma fortunes by now.=X= ought to be getting tired of karma fortunes by now.=X= ought to be getting tired of karma fortunes by now.
 
=X='s Avatar
 
Posts: 3,672
Karma: 12205348
Join Date: Mar 2008
Device: Galaxy S, Nook w/CM7
Quote:
Originally Posted by slushie View Post
also, i am not a python developer but am a programmer by trade. i would gladly submit a patch if you could give me a hint to where to find what i'm looking for in the source

thanks!
There is a Perl library called PDF::API2 that allows you to create and edit (including metatdata) PDF from a perl script.

You might want to try it out.

=X=

Last edited by =X=; 10-03-2008 at 06:37 PM.
=X= is offline   Reply With Quote
Old 10-03-2008, 08:04 PM   #9
pilotbob
Grand Sorcerer
pilotbob ought to be getting tired of karma fortunes by now.pilotbob ought to be getting tired of karma fortunes by now.pilotbob ought to be getting tired of karma fortunes by now.pilotbob ought to be getting tired of karma fortunes by now.pilotbob ought to be getting tired of karma fortunes by now.pilotbob ought to be getting tired of karma fortunes by now.pilotbob ought to be getting tired of karma fortunes by now.pilotbob ought to be getting tired of karma fortunes by now.pilotbob ought to be getting tired of karma fortunes by now.pilotbob ought to be getting tired of karma fortunes by now.pilotbob ought to be getting tired of karma fortunes by now.
 
pilotbob's Avatar
 
Posts: 19,493
Karma: 11248282
Join Date: Jan 2007
Location: Tampa, FL USA
Device: Kindle Touch
Quote:
Originally Posted by =X= View Post
There is a Perl library called PDF::API2 that allows you to create and edit (including metatdata) PDF from a perl script.

You might want to try it out.

=X=
Heh... just what calibre needs, another dependeny.

The page for it doesn't say anything about its license. Can I assume it is open source since it is hosted on sourceforge?

BOb
pilotbob is offline   Reply With Quote
Old 10-03-2008, 08:32 PM   #10
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 25,442
Karma: 4961459
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
Quote:
Originally Posted by pilotbob View Post
Heh... just what calibre needs, another dependeny.

The page for it doesn't say anything about its license. Can I assume it is open source since it is hosted on sourceforge?

BOb
calibre is *not* going to embed a perl/java/ruby interpreter.
kovidgoyal is offline   Reply With Quote
Old 10-03-2008, 11:55 PM   #11
slushie
Junior Member
slushie began at the beginning.
 
Posts: 7
Karma: 10
Join Date: Sep 2008
Device: Sony PRS-505
im a perl developer and the first thing i did was check CPAN to see if someone had a solution... and yes, PDF::API2 would seem to be a very complete pdf solution... but its rather huge, as a matter of fact. i think rather than try to put all of that alongside a big python app it would be better either to port the relevant parts of the library to python, or try to install sony's software in windows someplace and do some reversing to fix this issue... both seem to be interesting, but neither are not gonna be finished tonight ;]

edit: oh btw, that perl library is gpl2
slushie is offline   Reply With Quote
Old 10-04-2008, 12:31 AM   #12
pilotbob
Grand Sorcerer
pilotbob ought to be getting tired of karma fortunes by now.pilotbob ought to be getting tired of karma fortunes by now.pilotbob ought to be getting tired of karma fortunes by now.pilotbob ought to be getting tired of karma fortunes by now.pilotbob ought to be getting tired of karma fortunes by now.pilotbob ought to be getting tired of karma fortunes by now.pilotbob ought to be getting tired of karma fortunes by now.pilotbob ought to be getting tired of karma fortunes by now.pilotbob ought to be getting tired of karma fortunes by now.pilotbob ought to be getting tired of karma fortunes by now.pilotbob ought to be getting tired of karma fortunes by now.
 
pilotbob's Avatar
 
Posts: 19,493
Karma: 11248282
Join Date: Jan 2007
Location: Tampa, FL USA
Device: Kindle Touch
Quote:
Originally Posted by kovidgoyal View Post
calibre is *not* going to embed a perl/java/ruby interpreter.
heh... I didn't think you would. Of course, Mac and Linux already have perl on them.

BOb
pilotbob is offline   Reply With Quote
Old 10-04-2008, 12:42 AM   #13
=X=
Wizard
=X= ought to be getting tired of karma fortunes by now.=X= ought to be getting tired of karma fortunes by now.=X= ought to be getting tired of karma fortunes by now.=X= ought to be getting tired of karma fortunes by now.=X= ought to be getting tired of karma fortunes by now.=X= ought to be getting tired of karma fortunes by now.=X= ought to be getting tired of karma fortunes by now.=X= ought to be getting tired of karma fortunes by now.=X= ought to be getting tired of karma fortunes by now.=X= ought to be getting tired of karma fortunes by now.=X= ought to be getting tired of karma fortunes by now.
 
=X='s Avatar
 
Posts: 3,672
Karma: 12205348
Join Date: Mar 2008
Device: Galaxy S, Nook w/CM7
Quote:
Originally Posted by pilotbob View Post
.... Can I assume it is open source since it is hosted on sourceforge?
Yes it is GNU CopyLeft. I downloaded it from CPAN


Quote:
Originally Posted by kovidgoyal View Post
calibre is *not* going to embed a perl/java/ruby interpreter.
Well the post was really more intended for the developer who, had an itch to resolve his PDF problem.

But this does raise the opportunity to ask the question about calibre. As calibre continues to grow in popularity more and more are going to want to contribute to calibre. It would be nice if there was a plugin architecture to the tool where individuals could develop their component independently to the tool and have it seamlessly integrate to calibre.

Just a wish list I suppose

=X=
=X= is offline   Reply With Quote
Old 10-04-2008, 01:25 AM   #14
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 25,442
Karma: 4961459
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
Quote:
Originally Posted by =X= View Post
Yes it is GNU CopyLeft. I downloaded it from CPAN



Well the post was really more intended for the developer who, had an itch to resolve his PDF problem.

But this does raise the opportunity to ask the question about calibre. As calibre continues to grow in popularity more and more are going to want to contribute to calibre. It would be nice if there was a plugin architecture to the tool where individuals could develop their component independently to the tool and have it seamlessly integrate to calibre.

Just a wish list I suppose

=X=
calibre does have a plugin architecture. For a new input format you need to write a plugin to convert format -> html for a new output format you need to write opf -> format. Once you do that, integrating it into calibre is trivial.
kovidgoyal is offline   Reply With Quote
Old 10-04-2008, 11:57 PM   #15
=X=
Wizard
=X= ought to be getting tired of karma fortunes by now.=X= ought to be getting tired of karma fortunes by now.=X= ought to be getting tired of karma fortunes by now.=X= ought to be getting tired of karma fortunes by now.=X= ought to be getting tired of karma fortunes by now.=X= ought to be getting tired of karma fortunes by now.=X= ought to be getting tired of karma fortunes by now.=X= ought to be getting tired of karma fortunes by now.=X= ought to be getting tired of karma fortunes by now.=X= ought to be getting tired of karma fortunes by now.=X= ought to be getting tired of karma fortunes by now.
 
=X='s Avatar
 
Posts: 3,672
Karma: 12205348
Join Date: Mar 2008
Device: Galaxy S, Nook w/CM7
Quote:
Originally Posted by kovidgoyal View Post
calibre does have a plugin architecture. For a new input format you need to write a plugin to convert format -> html for a new output format you need to write opf -> format. Once you do that, integrating it into calibre is trivial.
Well not really, calibre has a modular design which makes it easy to add features. However this requires the 3rd party tool to get released with calibre. A plugin architecture allows indep developers to enhance the feature of a product without requiring a release or and update to include that plugin. A devloper can just install the plugin and integrate with the product.

Good examples is WordRMR or OOo FBTool that plugs into OpenOffice.

=X=
=X= is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Metadata and PDFs mdroberts Calibre 0 12-27-2009 06:54 AM
KDX: Unable to search PDFs from main screen... PDFs not indexed? unrequited Amazon Kindle 3 06-22-2009 07:59 PM
PDFs, metadata and conversion on a Mac iain_benson Sony Reader 2 01-19-2009 03:51 AM
Searching for a way to batch-update PDFs metadata Pulp PDF 2 01-13-2009 06:40 AM
View PDFs with Metadata? Sonist iRex 2 12-13-2008 05:02 PM


All times are GMT -4. The time now is 05:38 AM.


MobileRead.com is a privately owned, operated and funded community.