Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre > Plugins

Notices

Reply
 
Thread Tools Search this Thread
Old 08-09-2012, 10:46 PM   #1
oblib__
Junior Member
oblib__ began at the beginning.
 
Posts: 6
Karma: 10
Join Date: Aug 2012
Device: Smartphone
Help with content modifying plugin

I hope this isn't the wrong place to ask, or to simple/demanding of a question, but I'd like to make a Calibre plugin that can do mass replacements of strings within the text of a book, independent of incoming or outgoing format. I've got the heart of the conversion written in python already, in that if I pass my python script a text file (or even just a string), it does the replacements and prints out or saves the result.

What I'm missing is how to do the plugin. Could someone either point me to a similar plugin that I could copy and learn from, or even better, provide me with the skeleton outline of what I need to do this? The "write your own plugin" page gives me the basics, and I would think that I could take the HelloWorld plugin and make it do what I want with just a few changes.

So here's the heart of HelloWorld. How do I make it provide me with the contents of the book (as in all of the words in the book) as a string, and then let me save a modified string as the new contents?
Code:
 def run(self, path_to_ebook):
        from calibre.ebooks.metadata.meta import get_metadata, set_metadata
        file = open(path_to_ebook, 'r+b')
        ext  = os.path.splitext(path_to_ebook)[-1][1:].lower()
        mi = get_metadata(file, ext)
        mi.publisher = 'Hello World'
        set_metadata(file, mi, ext)
        return path_to_ebook
Thanks in advance!
oblib__ is offline   Reply With Quote
Old 08-09-2012, 10:53 PM   #2
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 24,823
Karma: 4369673
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
Why aren't you using the builtin search and replace options in the conversion pipeline?
kovidgoyal is online now   Reply With Quote
Old 08-09-2012, 11:08 PM   #3
oblib__
Junior Member
oblib__ began at the beginning.
 
Posts: 6
Karma: 10
Join Date: Aug 2012
Device: Smartphone
Not enough entries through the GUI at least. I didn't try researching command line options.

My script has a lots of replacements (182), including some conditional ones (if I see this text in the book, do this, otherwise do that). Could I do 10s to 100s of replacements using the builtin replace feature?

Last edited by oblib__; 08-09-2012 at 11:15 PM.
oblib__ is offline   Reply With Quote
Old 08-09-2012, 11:43 PM   #4
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 24,823
Karma: 4369673
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
you can do as many as you want.
kovidgoyal is online now   Reply With Quote
Old 08-10-2012, 12:12 AM   #5
oblib__
Junior Member
oblib__ began at the beginning.
 
Posts: 6
Karma: 10
Join Date: Aug 2012
Device: Smartphone
But I can't do the conditionals, right?

I see the parameter --search-replace for ebook-convert, which I'm guessing you are recommending. Is there some way to add my list of regex's so that it executes when I run a convert through the GUI?

See next post

Last edited by oblib__; 08-10-2012 at 12:27 AM. Reason: strike it
oblib__ is offline   Reply With Quote
Old 08-10-2012, 12:26 AM   #6
oblib__
Junior Member
oblib__ began at the beginning.
 
Posts: 6
Karma: 10
Join Date: Aug 2012
Device: Smartphone
Actually, looking at my code (I actually wrote the heart of this over a year ago), I've got custom processes for every replacement. I probably just don't know how to use regex's properly, but I wanted to be able to either preserve case, or sometimes lower case, etc.

For example if I wanted to replace all instances of 'hello' with 'howdy', I want Hello to become Howdy, HELLo to become HOWDy, and HELLO to become HOWDY. However with some replacements I wanted to change only subsequent case, so that Hello became Howdy, and HELLO became Howdy. I couldn't see a way to do that in regular expressions without a processing function.

Bottom line, is that now that I look at my code, I can't easily change it into a list of regular expressions.
oblib__ is offline   Reply With Quote
Old 08-10-2012, 12:49 AM   #7
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 24,823
Karma: 4369673
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
Then dont create a plugin and instead run calibre from source and simply modify the preprocess part (wherethe regexes are applied). Much easier than creatinf a plugin.
kovidgoyal is online now   Reply With Quote
Old 12-26-2012, 04:56 PM   #8
oblib__
Junior Member
oblib__ began at the beginning.
 
Posts: 6
Karma: 10
Join Date: Aug 2012
Device: Smartphone
So I've been working on this off and on for a while now, and could use some more help. First off, it would be nice if a new type of plugin were created maybe called mid-processing or something. It would be called in the preprocess class (in preprocess.py) just before the final 'return html' in the __call__ function. The plugin would take in the html text, and return a modified html text. I would try and add it myself and submit my changes, but I'm having trouble running from source right now.

As a work around for now, I'm trying to write my plugin to do all the needed opening, modifying, and saving of the book contents, independent of type input type or output type. What is the easiest way to do that? I've tried grabbing relevant code from the plumber.py file, and can successfully create the oeb in a temp directory, but I decided rather than keep hacking at it once every couple of months, maybe it was time to ask for help.

So bottom line is that I'd like to write a plugin that can take any book format in, modify the contents of that book with my python script, and then return the modified contents for final creation by the rest of calibre processing. Thanks in advance!

Oh, and the reason I don't want to just modify and run from source -- I'd like to be able to do this on all my installations (both Linux and Windows) and keep on doing it though future versions without having to redownload and modify code. Basically I want Calibre to always be able to do this work for me, and if it's useful, share it with others.
oblib__ is offline   Reply With Quote
Old 12-26-2012, 09:52 PM   #9
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 24,823
Karma: 4369673
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
Modifying source code and having your changes preserved through updates is easy. Just use bzr merge to update the source and it will preserve your changes, unless there is a conflicting change (which is unlikely since this part of the calibre code is pretty stable).

There is no shortcut to magically explode to html and repack to arbitrary format. You have to run a full conversion pipeline for that. You can do it for particular formats, see the tweak book function in calibre or the modify epub plugin, but in the general case a full pipeline run is required, as some output plugins depend on the normalization that occurs during a pipeline run.
kovidgoyal is online now   Reply With Quote
Old 12-28-2012, 04:39 PM   #10
oblib__
Junior Member
oblib__ began at the beginning.
 
Posts: 6
Karma: 10
Join Date: Aug 2012
Device: Smartphone
That helps -- the tweak file helps get epubs going at least, and I can try converting everything to epub.

What do you think about creating a new text-modifying plugin type?
oblib__ is offline   Reply With Quote
Old 12-28-2012, 09:42 PM   #11
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 24,823
Karma: 4369673
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
In principle, i have no objections to a new plugin type, but, I am not very motivated to implement it as I have other priorities and you;re the only person that has asked for it so far. So unless more people ask, or you submit a patch, it is not likely to happen.
kovidgoyal is online now   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Sort plugin or update content.opf velde046 Plugins 1 06-17-2012 05:08 AM
Plugin not customizable: Plugin: HTML Output does not need customization flyingfoxlee Conversion 2 02-24-2012 02:24 AM
Modifying B&N content Colerson Barnes & Noble NOOK 3 02-17-2012 08:57 AM
[GUI Plugin] Plugin Updater **Deprecated** kiwidude Plugins 159 06-19-2011 12:27 PM
New Plugin Type Idea: Library Plugin cgranade Plugins 3 09-15-2010 12:11 PM


All times are GMT -4. The time now is 10:51 AM.


MobileRead.com is a privately owned, operated and funded community.