Plugin ideas... add yours here

[Divingduck]

Originally Posted by ddjohn

I find a need to catalog design files that are not traditional document file formats. These files are for machine embroidery designs. Common file format extensions are .VP3, .VIP, .HUS, .PEC, .PES, .PCS, .xxx, .SEW, .JEF, .EXP, .10*, .DST, .SHV, .DHV

The problem with this formats is, that not all formats have metadata you can simply extract. In addition there are different versions of theses formats with different capabilities. Today you need to have additional external programs to extract theses data like Wilcom TrueSizer, Bernina Artlink, Embird Iconizer SandsComputing SewIconz etc.

[compurandom]

What would I need to do (to write a plugin) to extract metadata not currently supported into custom columns for book formats that are already supported by calibre?

[davidfor]

Originally Posted by compurandom

What would I need to do (to write a plugin) to extract metadata not currently supported into custom columns for book formats that are already supported by calibre?

The metadata source plugins can only update certain columns. That means you have to write all the metadata update and page fetching code yourself. There are some sample that you can look at. The Goodreads frating plugin fetches the rating from Goodreads. The Goodreads Sync also fetches some information, but that uses the API for some of the function.

[compurandom]

Originally Posted by davidfor

The metadata source plugins can only update certain columns. That means you have to write all the metadata update and page fetching code yourself. There are some sample that you can look at. The Goodreads frating plugin fetches the rating from Goodreads. The Goodreads Sync also fetches some information, but that uses the API for some of the function.

Actually, I'm looking to add metadata that is in pdf documents that isn't currently supported. Specifically, I want to know if the document has a TOC or not, so that I can look at my library and know which documents I might need to add a TOC.

I already know how to extract this information, at least by hand and from command line.

[kovidgoyal]

You can create a PDF metadata reader plugin that simply adds a tag "has_toc" to the list of tags returned by the builtin pdf metadata reading code.

[theducks]

Calibre support embedding fonts, but this seems redundant (and a space hog) for many devices that have the provision to use Device (system) fonts

(only provide support for CSS, Font declarations)

When sending to device:
1) Query CSS for @Font callouts
2) Add the FONT (if needed) to Device Font folder <move from the book?>
3) adjust the CSS font SRC as needed
4) remove the embedded font
send skinnier book

[davidfor]

Originally Posted by theducks

Calibre support embedding fonts, but this seems redundant (and a space hog) for many devices that have the provision to use Device (system) fonts

(only provide support for CSS, Font declarations)

When sending to device:
1) Query CSS for @Font callouts
2) Add the FONT (if needed) to Device Font folder <move from the book?>
3) adjust the CSS font SRC as needed
4) remove the embedded font
send skinnier book

I could see something like this working in a few different ways. One is within the library in the way Polish Books or convert work. Or as part of the Modify ePub plugin. That already has font removal, but I don't know if it fixes references.

An alternative is during send-to-device. The books are already being modified to update the metadata and covers. Stripping the fonts could be done there. It would need support in the device drivers to report the fonts on the device and if anything special is needed to fix references. The advantage of this is that it doesn't change the in-library copy, so that if you use multiple devices you get a copy that is appropriate for each.

Another way would be a plugin that wraps send-to-device. That could prepare the book and then call send-to-device to send the modified version rather than the in-library copy.

Lastly, a driver could be written that subclasses the existing drivers. The KoboTouchExtended driver already does this and it would be "easy" to add this function to that driver. The disadvantage of this is that an extra driver needs to be written for each device that needs the support.

I know the first and last would be doable. I'm not sure about the other two. The second is the way I'd prefer to do it, but is probably the most work.

Oh, one thing I probably wouldn't do is to automatically send the fonts to the device. I would probably write it to check the font is on the device, and if it is strip it from the book. If the font isn't there, I'd leave it alone. A danger of sending the font is if subsetting has been done. You might end up with a font on the device that doesn't support all the characters in all the books.

[theducks]

Originally Posted by davidfor

I could see something like this working in a few different ways. One is within the library in the way Polish Books or convert work. Or as part of the Modify ePub plugin. That already has font removal, but I don't know if it fixes references.

An alternative is during send-to-device. The books are already being modified to update the metadata and covers. Stripping the fonts could be done there. It would need support in the device drivers to report the fonts on the device and if anything special is needed to fix references. The advantage of this is that it doesn't change the in-library copy, so that if you use multiple devices you get a copy that is appropriate for each.

Another way would be a plugin that wraps send-to-device. That could prepare the book and then call send-to-device to send the modified version rather than the in-library copy.

Lastly, a driver could be written that subclasses the existing drivers. The KoboTouchExtended driver already does this and it would be "easy" to add this function to that driver. The disadvantage of this is that an extra driver needs to be written for each device that needs the support.

I know the first and last would be doable. I'm not sure about the other two. The second is the way I'd prefer to do it, but is probably the most work.

Oh, one thing I probably wouldn't do is to automatically send the fonts to the device. I would probably write it to check the font is on the device, and if it is strip it from the book. If the font isn't there, I'd leave it alone. A danger of sending the font is if subsetting has been done. You might end up with a font on the device that doesn't support all the characters in all the books.

Not as simple, when you get down to it.

Great point about Subsetting ( I don't do it...but!)
Maybe get from the same place Calibre would use (system folders)
I don't know how far Remove Embedded goes either. I only remove all references to Charis (manually). I like display fonts for limited use

Maybe a sideways use of The Quality check PI (selected books) to find a list of fonts called for,
then collect those from the system and
send them to the device (\fonts\ in my RMSDK case) if needed, then cleanup those books

[ddjohn]

BetterRed, thanks for this information. I'm still on an older version of Calibre, would I need to update to a more recent version to be able to add non-document files? The last time I tried, many years ago, 'add books' did not see the files I was hoping to add.

[ddjohn]

Originally Posted by Divingduck

The problem with this formats is, that not all formats have metadata you can simply extract. In addition there are different versions of theses formats with different capabilities. Today you need to have additional external programs to extract theses data like Wilcom TrueSizer, Bernina Artlink, Embird Iconizer SandsComputing SewIconz etc.

DivingDuck, I'm impressed that you are familiar with these files and potential softwares. I currently have HV Premier+ (just their complimentary package) and SandS Computing SWP. The HV Premier+ provides similar funtionality to the Iconizer, giving meta data regarding dimensions, stitch counts, thread brand and color, even how much thread is required per color. It would be incredible to pull any of that meta data into Calibre. I'll focus on what Better Read has covered, before leaping ahead to the metadata possabilities.

Thank you to all of you for your help.

[BetterRed]

Originally Posted by ddjohn

I'll focus on what Better Read has covered, before leaping ahead to the metadata possibilities.

Good thinking, that will enable you to explore the custom columns you want/need to have before taking the next step of automating the metadata loading via a Metadata Source plugin.

I also suggest you have a look at the Import List PI which can be used to update metadata for existing books and/or add new books. A metadata input PI would probably be better, but if the data is available as a CSV then the Import List PI may get you off the ground.

BR

[eschwartz]

Add Books opens a standard Windows file dialog, with a default filter to only show known ebook types... but you can switch the filter to All Files.

You can also use the Add Empty Book option to create the metadata entry and then add the files by dragging-and-dropping them onto the Book Details panel.

[BetterRed]

Originally Posted by eschwartz

Add Books opens a standard Windows file dialog, with a default filter to only show known ebook types... but you can switch the filter to All Files.

You can also use the Add Empty Book option to create the metadata entry and then add the files by dragging-and-dropping them onto the Book Details panel.

Originally Posted by ddjohn

BetterRed, thanks for this information. I'm still on an older version of Calibre, would I need to update to a more recent version to be able to add non-document files? The last time I tried, many years ago, 'add books' did not see the files I was hoping to add.

The ability to drag and drop any file type into Book details is a fairly recent enhancement, previously it was limited to ebook types.

The main reason I use Windows Send To was to obviate the need to spin the filter in Add Books. Having calibre in Send To suits me very well; given that my primary mode of using the PC is from my file manager via the context menu.

BR

[petzi]

At the moment metadata of German books are often poor or are missing totally with the available Calibre plugins. The best catch so far is amazon.de, but even there it mostly works with bestsellers – the metadata quality for science books are poor or they are missing completely.

I suggest therefore a metadata source plugin for the German National Library (if that is feasible and allowed).
http://dnb.d-nb.de/ or http://www.dnb.de/EN/Home/home_node.html

I think the German National Library is cataloguing *all* published German books and the quality of ther metadata is excellent.

[BetterRed]

Originally Posted by petzi

At the moment metadata of German books are often poor or are missing totally with the available Calibre plugins. The best catch so far is amazon.de, but even there it mostly works with bestsellers – the metadata quality for science books are poor or they are missing completely.

I suggest therefore a metadata source plugin for the German National Library (if that is feasible and allowed).
http://dnb.d-nb.de/ or http://www.dnb.de/EN/Home/home_node.html

I think the German National Library is cataloguing *all* published German books and the quality of ther metadata is excellent.

Did you check the list of optional metadata source plugins, IIRC there are one or two for German sites, but I can't attest to their 'quality'.

BR