Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre > Plugins

Notices

Reply
 
Thread Tools Search this Thread
Old 09-16-2014, 03:08 PM   #1
ClashTheBunny
Junior Member
ClashTheBunny began at the beginning.
 
Posts: 8
Karma: 10
Join Date: Sep 2014
Device: Kindle Paperwhite 2
[FileType Plugin] YVES Bible Plugin

YouVersion allows you to download Bibles offline. They are just simple html snippets of each chapter of the Bible encoded in a strange fashion. Using this plugin, you can "add book" for the "manifest.yves" for each Bible on your Android's SD Card. It can then be converted to any format that Calibre knows about.

For example, in /sdcard/.youversion/bibles/1/ is manifest.yves for the King James version of the Bible. Just copy the whole '1' directory over to your computer somewhere and "add" the manifest.yves from it. Or, if your android phone shows up like a drive on your computer, just open the /sdcard/.youversion/bibles/1/manifest.yves file straight from there. It will import a htmlz formatted book into your library that you can then convert to epub or mobi.

It imports everything, so for example if the Bible has introductions or something, it will be added to the book that it is a part of, even if it's not "canonical".

It is a very simple concatenation of all the files with a simple chapter header put between each chapter. Don't expect anything beautiful. It's just easier on the eyes than an Android phone. It's also searchable offline. There are so many times that the Bible is just useless to me if I can't find a specific phrase.

Either way, it's here on github: https://github.com/ClashTheBunny/calibre-yves-input

This is clearly fair use of the files on SD Card because they are not encrypted and there is no reason you would distribute any of these, you already are able to download them for free. Don't ask me for any of the data that this is built from. That would be VERY wrong. Just get you own Bible app and download for offline usage.

Changelog:

Version 0.0.7:
This version is compatible back to 1.25: the version that comes on Ubuntu 14.04. Since it's LTS, there may be many people who will use that version for the years to come, so it's helpful to have it supported. It goes back on sub-classing HTML2ZIP and instead just instantiates an instance of it and calls it. There is a bug of some sort in that version that doesn't overwrite the .name property on a plugin that has been imported at the top of the file. This should not change other functionality.

Version 0.0.6:
Make the yves file read in with binary, so that it is compatible with Windows. Windows has different ways of treating binary and text files, which can remove information.

Version 0.0.5 (unreleased):
Refactored thing out of __init__.py so that I could test them easier from the command line.

Version 0.0.4:
Cleans up some general things and adds support for Right-to-Left Languages and fixes languages that don't have a two letter code.

Todo:

Footnote parsing
Better chapter and book separation
Option to exclude extraneous text, like introductions and such
Add more documentation on converting to EPub or Mobi, like book and chapter xpath statements.
Attached Files
File Type: zip YVES to ZIP.zip (41.4 KB, 11 views)

Last edited by ClashTheBunny; 09-23-2014 at 09:14 AM. Reason: Update version to 0.0.7
ClashTheBunny is offline   Reply With Quote
Old 09-16-2014, 03:41 PM   #2
eschwartz
Irrational Optimist
eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.
 
Posts: 6,456
Karma: 10683802
Join Date: Nov 2012
Location: The Beaten Path, USA, Roundworld, This Side of Infinity
Device: Kindle Touch (Wifi only)
Usually, they do what you did.

Plugins are distributed through their MobileRead thread, and the canonical location is via the attachment you put here. the calibre website indexes the plugin threads and caches the attached plugin for the plugin updater.

You could also just move the "src/calibre_plugins/yves_input/" folder into the root of the repo If you plan on making multiple plugins, then for organization purposes I would use a separate repo for each. Probably no big deal, though.

For quick packaging, you can use calibre-customize.exe. It allows you to update the plugin in calibre, and then you just upload the working plugin from your configuration folder.

on the plugin and Welcome to MobileRead!
eschwartz is offline   Reply With Quote
 
Enthusiast
Old 09-16-2014, 04:02 PM   #3
eschwartz
Irrational Optimist
eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.
 
Posts: 6,456
Karma: 10683802
Join Date: Nov 2012
Location: The Beaten Path, USA, Roundworld, This Side of Infinity
Device: Kindle Touch (Wifi only)
Quote:
Originally Posted by ClashTheBunny View Post
The last thing that is still not working is that I don't know how to add yves to the list of files that you can "open". I just always have to choose "all files" before it will let me select yves, even thought the plugin says that yves is the right extension.

Maybe you need to extend the BOOK_EXTENSIONS?

https://github.com/kovidgoyal/calibr..._init__.py#L29
eschwartz is offline   Reply With Quote
Old 09-17-2014, 03:18 AM   #4
ClashTheBunny
Junior Member
ClashTheBunny began at the beginning.
 
Posts: 8
Karma: 10
Join Date: Sep 2014
Device: Kindle Paperwhite 2
Unhappy

Quote:
Originally Posted by eschwartz View Post
Maybe you need to extend the BOOK_EXTENSIONS?

https://github.com/kovidgoyal/calibr..._init__.py#L29
I looked into this and I don't see a place in the API documentation where this would be extendable. It seems that this is built into a plist for osx in the setup section of the source, which makes me think that maybe a humble plugin would not have much effect on the app.

I also don't see many FileTypePlugin plugins out there, maybe this is a fairly non-standard thing to try to add file types to the list. For example, the plugin that I started with, calibre-doc-input, adds support for the .doc extension, but it's already in the BOOK_EXTENSION list. It seems that even if it's not supported in calibre, it's in the BOOK_EXTENSION list to allow calibre to manage the type of document.
ClashTheBunny is offline   Reply With Quote
Old 09-17-2014, 03:22 AM   #5
ClashTheBunny
Junior Member
ClashTheBunny began at the beginning.
 
Posts: 8
Karma: 10
Join Date: Sep 2014
Device: Kindle Paperwhite 2
Thumbs up

Quote:
Originally Posted by eschwartz View Post
You could also just move the "src/calibre_plugins/yves_input/" folder into the root of the repo If you plan on making multiple plugins, then for organization purposes I would use a separate repo for each. Probably no big deal, though.
I had copied another plugin called calibre_doc_input, so I took the directory structure from there, hoping that it would be convenient. Looks like I'm going to just move it around so I can just upload the github zip. This forum allows uploading via URL, so it should make development pretty easy with the shift around in source.

Quote:
Originally Posted by eschwartz View Post
on the plugin and Welcome to MobileRead!
Thanks! It's good to be here!
ClashTheBunny is offline   Reply With Quote
Old 09-17-2014, 03:44 AM   #6
ClashTheBunny
Junior Member
ClashTheBunny began at the beginning.
 
Posts: 8
Karma: 10
Join Date: Sep 2014
Device: Kindle Paperwhite 2
Quote:
Originally Posted by ClashTheBunny View Post
Looks like I'm going to just move it around so I can just upload the github zip. This forum allows uploading via URL, so it should make development pretty easy with the shift around in source.
Github creates the zip with a parent directory and calibre isn't smart enough to look in the directory. I guess I can add a to src/calibre/customize/zipplugin.py" _locate_code to look one directory deep if there is nothing but a directory in the zip.
ClashTheBunny is offline   Reply With Quote
Old 09-17-2014, 06:18 AM   #7
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 26,109
Karma: 5101571
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
BOOK_EXTENSIONS controls what filetypes calibre recognizes as being ebooks. This list is not meant to be affected by file input plugins, which influence only what files calibre can convert.

As for creating the zip file, dont use github export, use calibre-customize. If you must use github export then simply move the files into the root of the repository.

And you need to have both __init__.py and the .txt file with the plugin import name in the root of the folder, for calibre to recognize the plugin, that is not going to change.
kovidgoyal is offline   Reply With Quote
Old 09-17-2014, 10:58 AM   #8
eschwartz
Irrational Optimist
eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.
 
Posts: 6,456
Karma: 10683802
Join Date: Nov 2012
Location: The Beaten Path, USA, Roundworld, This Side of Infinity
Device: Kindle Touch (Wifi only)
Yep, plugin folder should be the repo, not be in the repo.
eschwartz is offline   Reply With Quote
Old 09-17-2014, 11:06 AM   #9
ClashTheBunny
Junior Member
ClashTheBunny began at the beginning.
 
Posts: 8
Karma: 10
Join Date: Sep 2014
Device: Kindle Paperwhite 2
Yep, tried that, and github still doesn't work with their way of trying to prevent tarbombs. They will not let you download a zip file without the name of the repo and the name of the branch as the root directory.

It seems that tarbombs are the policy though, so no releases from github are possible.
ClashTheBunny is offline   Reply With Quote
Old 09-17-2014, 11:29 AM   #10
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 26,109
Karma: 5101571
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
Dont use github's zip export then. In any case you want the plugin zip file to not be compressed for performance while loading the plugin code, calibre-customize takes care of that for you automatically.
kovidgoyal is offline   Reply With Quote
Old 09-23-2014, 04:32 AM   #11
ichbindasauge
Connoisseur
ichbindasauge has memorized the entire works of Homer, Shakespeare, and Jane Austenichbindasauge has memorized the entire works of Homer, Shakespeare, and Jane Austenichbindasauge has memorized the entire works of Homer, Shakespeare, and Jane Austenichbindasauge has memorized the entire works of Homer, Shakespeare, and Jane Austenichbindasauge has memorized the entire works of Homer, Shakespeare, and Jane Austenichbindasauge has memorized the entire works of Homer, Shakespeare, and Jane Austenichbindasauge has memorized the entire works of Homer, Shakespeare, and Jane Austenichbindasauge has memorized the entire works of Homer, Shakespeare, and Jane Austenichbindasauge has memorized the entire works of Homer, Shakespeare, and Jane Austenichbindasauge has memorized the entire works of Homer, Shakespeare, and Jane Austenichbindasauge has memorized the entire works of Homer, Shakespeare, and Jane Austen
 
Posts: 67
Karma: 23456
Join Date: Oct 2009
Device: Kindle 2 US, iPad, Samsung Win 8 Tablet
The plugin doesn't seem to work for me. I have installed it and added the manifest.yves file to the library but calibre just adds the single file, it will not convert it to zip or anything else. Anything else I have to do?
ichbindasauge is offline   Reply With Quote
Old 09-23-2014, 04:38 AM   #12
ClashTheBunny
Junior Member
ClashTheBunny began at the beginning.
 
Posts: 8
Karma: 10
Join Date: Sep 2014
Device: Kindle Paperwhite 2
Thanks so much for trying this!

What platform are you on?

Did you keep the manifest.yves in the same location as all of the directories containing zips? AKA, manifest.yves is in the same directory as GEN directory, which contains 1.yves, 2.yves, etc?

If that doesn't work, could you do the "restart calibre in debug mode" and then post the text file output of the session after you try the import again? That will tell me where the script is failing.
ClashTheBunny is offline   Reply With Quote
Old 09-23-2014, 06:38 AM   #13
ichbindasauge
Connoisseur
ichbindasauge has memorized the entire works of Homer, Shakespeare, and Jane Austenichbindasauge has memorized the entire works of Homer, Shakespeare, and Jane Austenichbindasauge has memorized the entire works of Homer, Shakespeare, and Jane Austenichbindasauge has memorized the entire works of Homer, Shakespeare, and Jane Austenichbindasauge has memorized the entire works of Homer, Shakespeare, and Jane Austenichbindasauge has memorized the entire works of Homer, Shakespeare, and Jane Austenichbindasauge has memorized the entire works of Homer, Shakespeare, and Jane Austenichbindasauge has memorized the entire works of Homer, Shakespeare, and Jane Austenichbindasauge has memorized the entire works of Homer, Shakespeare, and Jane Austenichbindasauge has memorized the entire works of Homer, Shakespeare, and Jane Austenichbindasauge has memorized the entire works of Homer, Shakespeare, and Jane Austen
 
Posts: 67
Karma: 23456
Join Date: Oct 2009
Device: Kindle 2 US, iPad, Samsung Win 8 Tablet
Windows 8.1 64 bit

This is what I get from the debug mode:
Spoiler:

Running file type plugin YVES to ZIP failed with traceback:
Traceback (most recent call last):
File "site-packages\calibre\customize\ui.py", line 154, in _run_filetype_plugins
File "calibre_plugins.yves_input.__init__", line 33, in run
File "json\__init__.py", line 338, in loads
File "json\decoder.py", line 366, in decode
File "json\decoder.py", line 382, in raw_decode
ValueError: Unterminated string starting at: line 1 column 56320 (char 56319)

Added manifest to db in: 0.447000026703 seconds

Last edited by ichbindasauge; 09-23-2014 at 10:24 AM.
ichbindasauge is offline   Reply With Quote
Old 09-23-2014, 06:58 AM   #14
ClashTheBunny
Junior Member
ClashTheBunny began at the beginning.
 
Posts: 8
Karma: 10
Join Date: Sep 2014
Device: Kindle Paperwhite 2
Okay I think I found the problem, could you try 0.0.6? I made a rookie mistake on not reading the yves files in binary mode, which works on Linux and OSX, but isn't too happy on Windows.
ClashTheBunny is offline   Reply With Quote
Old 09-23-2014, 07:13 AM   #15
ClashTheBunny
Junior Member
ClashTheBunny began at the beginning.
 
Posts: 8
Karma: 10
Join Date: Sep 2014
Device: Kindle Paperwhite 2
Also, this was the part of the traceback that I needed. I'll copy it here so that you can delete the rest of the traceback if you worry about privacy issues, but so that I can still see it if I need it later.

Quote:
Originally Posted by ichbindasauge View Post
Running file type plugin YVES to ZIP failed with traceback:
Traceback (most recent call last):
File "site-packages\calibre\customize\ui.py", line 154, in _run_filetype_plugins
File "calibre_plugins.yves_input.__init__", line 33, in run
File "json\__init__.py", line 338, in loads
File "json\decoder.py", line 366, in decode
File "json\decoder.py", line 382, in raw_decode
ValueError: Unterminated string starting at: line 1 column 56320 (char 56319)
ClashTheBunny is offline   Reply With Quote
Reply

Tags
bible, calibre, input format

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
[GUI Plugin] KindleUnpack - The Plugin DiapDealer Plugins 263 10-14-2014 01:08 PM
Plugin not customizable: Plugin: HTML Output does not need customization flyingfoxlee Conversion 2 02-24-2012 02:24 AM
[GUI Plugin] Plugin Updater **Deprecated** kiwidude Plugins 159 06-19-2011 12:27 PM
Zip Filetype Plugin? rsingley Plugins 7 02-11-2011 05:11 PM
New Plugin Type Idea: Library Plugin cgranade Plugins 3 09-15-2010 12:11 PM


All times are GMT -4. The time now is 01:16 PM.


MobileRead.com is a privately owned, operated and funded community.