No Metadate from nonDRM eReader PDB file?

[macr0t0r]

I just want to confirm if this is a missing feature or a bug. I have quite a few files that I've generated using DropBook 1.6. The resulting PDB has the correct metadata (title, author, etc) when I load it in eReader on the desktop. However, if I pull it into Calibre, it just puts the filename into Title and leaves the rest blank. In the past, I had a script that converted it to HTML that I would then import into Calibre. It would always pull the MetaData successfully with that one. (No...not THAT shady script...the html in that one is SCARY).

I know how the metadata is stored in an eReader file. If this is a missing feature, I'll get to work on a pluguin. If this is a bug, I'll start debugging. So, which is it?

Ah yes, I'm running Calibre 0.6.24 on OS X, Leopard (no snow).

- Jim

[macr0t0r]

Aaaaaand once again I post too soon!

I searched again and found Ticket #3743: PML Metadata Missing. This references PML OUTPUT rather than input, but part of John's comment pretty much tells the whole story:
I have no interest in writing a PML metadata reader or writer. Meaning unless someone else contributes it I won't get around to doing anything with it in the near future.

Alrighty! That answers that question. I've done some in-house Calibre pre-processing plugins, but Metadata is a new one for me. Time to do some reading of the Calibre manual.

- Jim

[macr0t0r]

Now hang on a minute! I grabbed the source and looked up the metadata folder for an example, and there IS an ereader Metadata class for nonDRM Dropbook files!

Grrrrr.... let's see how many times I can second-guess myself. Alrighty, I guess I should run Calibre in debug mode? I notice that there is also code to set the Metadata, which makes me wonder if that ticket should be closed and I should be treating this as a bug.

Crimony, is this a feature or not?

- Jim

[kovidgoyal]

The latest calibre release has a PML metadata reader. But that works wil pml and pmlz files, not pdb files.

There is also a PDB metadata reader, but user_none will have to update you on the status of that.

[user_none]

Open a ticket and attach the file that the metadata is not being read from. I'll look into why it's not working correctly. It is one of three things. 1) regression in the metadata reader. 2) Dropbook has changed how it sets the metadata. 3) There is no metadata in the PDB itself and eReader desktop is reading it from the PML inside the file (calibre does not support this).

PML metadata reading has been added. WayneD contributed the core part that I didn't want to deal with. A set of regex's to read it from the PML. I took that, turned it into a metadata plugin, and added support for extracting the content from the a pmlz archive.

eReader PDB files should have the meatadata read from them. However, eReader PDB files will only have the metadata read from the metadata section of eReader PDB file not from the PML within the file. I'm toying with the idea of having the metadata read from the PML within the file if it cannot be read from the metadata section. However, I'm hesitant to implement this because it would require the entire file to be decompressed which could be slow. Though it shouldn't be much if any slower than reading from pmlz... What are your thoughts on this macr0t0r?

[kovidgoyal]

@user_none: If you do implement a decompress based metadata reader, make sure it doesn't happen when quick metadata reading is asked for (see for example the epub metadata reader).

[macr0t0r]

Hmmmm...I don't now about reading the PML text for Metadata. I suppose you could read just the first text section since it's always in the first few lines, but that would take a lot of extra coding.

You can get the title by just sucking down the first 24 bytes of the file itself (the header of the PDB wrapper) and be happy with that. As a bonus, that works for both DRM and nonDRM files. Unless the person is using some sort of homebrew eReader creator, it's nearly a guarantee that if the metadata was specified within the /v blocks, then it will be loaded in the MetaInfo section.

I'm setting up my Calibre development environment (bzr is chugging along). I'm going to add a few print statements and see what's going on with my file within the "get_metadata" function of ereader.py.

- Jim

[macr0t0r]

Well, that was useful!

I got a ton of error messages. Turns out I had corrupted preferences. I'm not sure how Calibre kept running at all. After clearing out the preferences and re-starting, it all works.

Well...I guess I'll have to put my skills to use elsewhere. Thanks guys!

- Jim

[user_none]

Originally Posted by macr0t0r

Hmmmm...I don't now about reading the PML text for Metadata. I suppose you could read just the first text section since it's always in the first few lines ...

It should be in the first few lines. Since this would only be used as a fall back I was going to have it read the fist section and if it's not there continue to the next. Once all have been read then look at the entire document and see if possibly it was broken into multiple sections.

Originally Posted by macr0t0r

... but that would take a lot of extra coding.

It would take very little work to make this happen. All that would be required is to check the eReader section 0 header type. Open the stream with the appropriate reader and use a simple for loop to extract each section. Something like

for i in section_count:
    read_metadata(section(i))

Originally Posted by macr0t0r

You can get the title by just sucking down the first 24 bytes of the file itself (the header of the PDB wrapper) and be happy with that. As a bonus, that works for both DRM and nonDRM files.

This is the current fall back.

Originally Posted by macr0t0r

Unless the person is using some sort of homebrew eReader creator, it's nearly a guarantee that if the metadata was specified within the /v blocks, then it will be loaded in the MetaInfo section.

So basically reading the /v block as a fall back is waste of time. the MetaInfo section is the first place metadata is read.