Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre > Plugins

Notices

Reply
 
Thread Tools Search this Thread
Old 04-01-2019, 03:59 PM   #1
karl1c
Connoisseur
karl1c has become a pillar of the MobileRead communitykarl1c has become a pillar of the MobileRead communitykarl1c has become a pillar of the MobileRead communitykarl1c has become a pillar of the MobileRead communitykarl1c has become a pillar of the MobileRead communitykarl1c has become a pillar of the MobileRead communitykarl1c has become a pillar of the MobileRead communitykarl1c has become a pillar of the MobileRead communitykarl1c has become a pillar of the MobileRead communitykarl1c has become a pillar of the MobileRead communitykarl1c has become a pillar of the MobileRead community
 
Posts: 54
Karma: 15568
Join Date: Jul 2017
Device: iPad
[GUI Plugin] KyBook3 Sync

Corrected error when a book had no language assigned

If you're using this plugin, please leave a comment to say whether it works for you. Thanks.

Does anyone with some Python experience want to help with developing this plugin? If so, I'd be happy to make it available. PM me.

Now ONLY syncs those file formats in the comma-separated list in settings.

This plugin provides master-slave synchronisation of books and metadata from Calibre to KyBook3 (http://kybook-reader.com).

KyBook3 provides support for almost every book and audiobook format and can read metadata from them. However, many files have incorrect metadata and/or cover. Even when KyBook3 downloads metadata and covers from Calibre's Content Server some metadata and cover errors occur. This plugin is an attempt to overcome these shortcomings.

Main Features:
  • Upload selected books to KyBook3 (skips files with the same MD5 already in KyBook3)
  • Sync title, authors, publishers, subjects, series, publication date, language, comments, ratings, (some) identifiers, & cover to KyBook3 with the following mapping:
    Calibre Tags -> KyBook3 Subjects
    Calibre Comments -> KyBook3 Annotations
  • Backup of KyBook3's metadata

Special Notes:
  • The plugin assumes that Calibre's metadata is more correct than KyBook3's
  • Neither books nor metadata in Calibre are changed.
  • No book files in KyBook3 will be changed or overwritten.
  • Metadata in KyBook3 is overwritten by that in Calibre
  • You should probably ensure that Calibre's Tags are single words with an initial capital
  • Calibre's Comments can (optionally) have HTML stripped
  • There is currently no way to sync from KyBook3 to Calibre (although this is planned)
  • Sync takes place over Wi-Fi, so may take a long time. I suggest you sync in small batches. Alternatively, you can add all the books you want in KyBook3 from iTunes and then run KyBook3 Sync to sync just the metadata

Testimonials:
  • "... syncing calibre with an actual reader app is a dream come true, thank you."
Installation Steps:
  1. Download the attached zip file and install the plugin/restart Calibre/add to context menu or toolbar as described in the Introduction to plugins thread
  2. Click on the KyBook3 Sync icon to see KyBook3's menu
  3. Click on "Customize this plugin" in KyBook3's menu and set the Link, Username, & Password used by KyBook3's Content Server
  4. You should now be able to select some books in Calibre and sync them to KyBook3

Debugging:
If you have problems with the plugin:
  1. Be patient, syncing over WiFi can be slow and if your first book is a 500MB PDF, progress will be "stuck" at 1% for a long time!
  2. In Calibre click Preferences | Restart in debug mode
  3. After the restart, try running the plugin again
  4. When it fails, or it appears to get stuck, close Calibre
  5. A text file will appear with debug info, copy the contents and paste into a post here between spoiler tags (the icon with an eye and a red x) or save the text file to Dropbox and paste a link to it here.

Paypal Donations:
  • If you find this plugin useful please feel free to show your appreciation. I have spent many unpaid hours in its development and support so any encouragement for me to continue is appreciated!

Version History:
Spoiler:

Version 1.0.11 - 18 April 2019
Corrected error when a book had no language assigned

Version 1.0.10 - 10 April 2019
Added syncing of series number (thanks to nimapourkarimi)
Corrected error when no files of the required formats were found
Removed setting ebookid fields to NULL in authors & sequences tables

Version 1.0.9 - 08 April 2019
Corrected error where /$User/covers was not created correctly from Windows (thanks to ilovejedd)

Version 1.0.8 - 08 April 2019
Now ONLY syncs formats specified in settings

Version 1.0.7 - 07 April 2019
Added syncing of rating
Added syncing of some identifiers (isbn, amazon, asin, oclc)
Corrected error in presentation of SQL in debug output

Version 1.0.6 - 05 April 2019
Added automatic logging when running as a plugin
Improved robustness of file uploads

Version 1.0.5 - 04 April 2019
Added progress bar for uploading DB file
Tidied up failure when Content Server on KyBook3 has not been started

Version 1.0.4 - 03 April 2019
Corrected creation of zip file (thanks to ilovejedd)
Added dynamic setting of Calibre's library path (thanks to ilovejedd)
Corrected error where some subjects appeared more than once in KyBook3

Version 1.0.3 - 02 April 2019
Added support for Windows paths
Corrected error in syncing series (thanks to nimapourkarimi)

Version 1.0.2 - 02 April 2019
Added support for Calibre's GUI Debug mode
Corrected filling lookups with NULL values

Version 1.0.1 - 01 April 2019
Changed port number and password for client/listener.

Version 1.0.0 - 01 April 2019
Initial release of KyBook3 Sync plugin


Future Development:
I plan to add the following features:
  • syncing of identifiers (isbn, issn, etc.) [in progress]
  • use Calibre's uuid to avoid calculating md5 and speed up processing
  • adding to Calibre books found in KyBook3, but not already in Calibre
  • syncing metadata back from KyBook3 to Calibre
  • syncing of collections (does anyone actually use collections?)
Attached Files
File Type: zip KyBook3 Sync.zip (79.4 KB, 61 views)

Last edited by karl1c; 04-18-2019 at 06:26 PM.
karl1c is offline   Reply With Quote
Old 04-01-2019, 05:19 PM   #2
nimapourkarimi
Connoisseur
nimapourkarimi is a rising star in the heavensnimapourkarimi is a rising star in the heavensnimapourkarimi is a rising star in the heavensnimapourkarimi is a rising star in the heavensnimapourkarimi is a rising star in the heavensnimapourkarimi is a rising star in the heavensnimapourkarimi is a rising star in the heavensnimapourkarimi is a rising star in the heavensnimapourkarimi is a rising star in the heavensnimapourkarimi is a rising star in the heavensnimapourkarimi is a rising star in the heavens
 
Posts: 56
Karma: 13922
Join Date: Jul 2017
Device: iPad mini 3, Kindle PW2
Thanks for this plugin, I can successfully sync some books but randomly encounter this error:
calibre, version 3.40.1 (darwin, embedded-python: True)
Failed to sync with KyBook3: Failed: Sync 3 books. Did you start KyBook3's Content Server?

Starting job: Sync 3 books
Sync complete, with 0 failures
Job: "Sync 3 books" failed with error:
Traceback (most recent call last):
File "site-packages/calibre/gui2/threaded_jobs.py", line 84, in start_work
File "calibre_plugins.kybook3_sync.jobs", line 109, in sync_threaded
EOFError

Called with args: ([30455, 30454, 30453], <calibre.db.cache.Cache object at 0x1192c18d0>) {u'log': <calibre.utils.logging.GUILog object at 0x127615b50>, u'abort': <threading._Event object at 0x127615ad0>, u'notifications': <Queue.Queue instance at 0x1220dccf8>}


...and I'm sure Kybook3's content server is on.
nimapourkarimi is offline   Reply With Quote
Advert
Old 04-01-2019, 05:41 PM   #3
nimapourkarimi
Connoisseur
nimapourkarimi is a rising star in the heavensnimapourkarimi is a rising star in the heavensnimapourkarimi is a rising star in the heavensnimapourkarimi is a rising star in the heavensnimapourkarimi is a rising star in the heavensnimapourkarimi is a rising star in the heavensnimapourkarimi is a rising star in the heavensnimapourkarimi is a rising star in the heavensnimapourkarimi is a rising star in the heavensnimapourkarimi is a rising star in the heavensnimapourkarimi is a rising star in the heavens
 
Posts: 56
Karma: 13922
Join Date: Jul 2017
Device: iPad mini 3, Kindle PW2
I found out that this problem occurs with some of my PDF books, some of PDF files and all of my epub files sync correctly.

Edit: Nope, it happens with epubs too.

Last edited by nimapourkarimi; 04-01-2019 at 05:47 PM.
nimapourkarimi is offline   Reply With Quote
Old 04-01-2019, 06:33 PM   #4
karl1c
Connoisseur
karl1c has become a pillar of the MobileRead communitykarl1c has become a pillar of the MobileRead communitykarl1c has become a pillar of the MobileRead communitykarl1c has become a pillar of the MobileRead communitykarl1c has become a pillar of the MobileRead communitykarl1c has become a pillar of the MobileRead communitykarl1c has become a pillar of the MobileRead communitykarl1c has become a pillar of the MobileRead communitykarl1c has become a pillar of the MobileRead communitykarl1c has become a pillar of the MobileRead communitykarl1c has become a pillar of the MobileRead community
 
Posts: 54
Karma: 15568
Join Date: Jul 2017
Device: iPad
@nimapourkarimi: Well that wasn't a good start was it? :-(

I've uploaded an amended version. Give that a go and see whether it is any better.
karl1c is offline   Reply With Quote
Old 04-01-2019, 06:48 PM   #5
nimapourkarimi
Connoisseur
nimapourkarimi is a rising star in the heavensnimapourkarimi is a rising star in the heavensnimapourkarimi is a rising star in the heavensnimapourkarimi is a rising star in the heavensnimapourkarimi is a rising star in the heavensnimapourkarimi is a rising star in the heavensnimapourkarimi is a rising star in the heavensnimapourkarimi is a rising star in the heavensnimapourkarimi is a rising star in the heavensnimapourkarimi is a rising star in the heavensnimapourkarimi is a rising star in the heavens
 
Posts: 56
Karma: 13922
Join Date: Jul 2017
Device: iPad mini 3, Kindle PW2
Well no, it does that again with certain books.
nimapourkarimi is offline   Reply With Quote
Advert
Old 04-01-2019, 06:55 PM   #6
karl1c
Connoisseur
karl1c has become a pillar of the MobileRead communitykarl1c has become a pillar of the MobileRead communitykarl1c has become a pillar of the MobileRead communitykarl1c has become a pillar of the MobileRead communitykarl1c has become a pillar of the MobileRead communitykarl1c has become a pillar of the MobileRead communitykarl1c has become a pillar of the MobileRead communitykarl1c has become a pillar of the MobileRead communitykarl1c has become a pillar of the MobileRead communitykarl1c has become a pillar of the MobileRead communitykarl1c has become a pillar of the MobileRead community
 
Posts: 54
Karma: 15568
Join Date: Jul 2017
Device: iPad
1. What OS are you using?
2. Are the same books failing each time?
3. Is there any commonality in the books that fail? (Very large, specific format, etc.)
karl1c is offline   Reply With Quote
Old 04-01-2019, 06:59 PM   #7
nimapourkarimi
Connoisseur
nimapourkarimi is a rising star in the heavensnimapourkarimi is a rising star in the heavensnimapourkarimi is a rising star in the heavensnimapourkarimi is a rising star in the heavensnimapourkarimi is a rising star in the heavensnimapourkarimi is a rising star in the heavensnimapourkarimi is a rising star in the heavensnimapourkarimi is a rising star in the heavensnimapourkarimi is a rising star in the heavensnimapourkarimi is a rising star in the heavensnimapourkarimi is a rising star in the heavens
 
Posts: 56
Karma: 13922
Join Date: Jul 2017
Device: iPad mini 3, Kindle PW2
1. macOS Mojave 10.14.4 running calibre 3.40.1
2. Yes
3. No, I can sync a 250mb PDF file but a certain 2mb pdf file won't sync, that applies to epubs too, the file copies to kybook successfully but the process fails while syncing metadata to kybook.

Last edited by nimapourkarimi; 04-01-2019 at 07:02 PM.
nimapourkarimi is offline   Reply With Quote
Old 04-01-2019, 07:11 PM   #8
karl1c
Connoisseur
karl1c has become a pillar of the MobileRead communitykarl1c has become a pillar of the MobileRead communitykarl1c has become a pillar of the MobileRead communitykarl1c has become a pillar of the MobileRead communitykarl1c has become a pillar of the MobileRead communitykarl1c has become a pillar of the MobileRead communitykarl1c has become a pillar of the MobileRead communitykarl1c has become a pillar of the MobileRead communitykarl1c has become a pillar of the MobileRead communitykarl1c has become a pillar of the MobileRead communitykarl1c has become a pillar of the MobileRead community
 
Posts: 54
Karma: 15568
Join Date: Jul 2017
Device: iPad
I've uploaded another version, give that a go.
karl1c is offline   Reply With Quote
Old 04-01-2019, 08:26 PM   #9
nimapourkarimi
Connoisseur
nimapourkarimi is a rising star in the heavensnimapourkarimi is a rising star in the heavensnimapourkarimi is a rising star in the heavensnimapourkarimi is a rising star in the heavensnimapourkarimi is a rising star in the heavensnimapourkarimi is a rising star in the heavensnimapourkarimi is a rising star in the heavensnimapourkarimi is a rising star in the heavensnimapourkarimi is a rising star in the heavensnimapourkarimi is a rising star in the heavensnimapourkarimi is a rising star in the heavens
 
Posts: 56
Karma: 13922
Join Date: Jul 2017
Device: iPad mini 3, Kindle PW2
There is no error anymore but it gets stuck at metadata sync the moment it reaches those certain books and it remains in that position. I gave it some time but nothing happened and had to kill the job eventually.
nimapourkarimi is offline   Reply With Quote
Old 04-01-2019, 10:03 PM   #10
ilovejedd
hopeless n00b
ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.
 
ilovejedd's Avatar
 
Posts: 3,694
Karma: 9002933
Join Date: Jan 2009
Location: in the middle of nowhere
Device: PW4, PW3, iPad 9.7, iPad 12.9
Tried the plugin. Job progress is just stuck at 1%.

Tested KyBook3 content server link is accessible via Chrome.
  1. What OS are you using?
    Windows 10 Pro 64-bit Build 17134
  2. Are the same books failing each time?
    all books fail to sync
  3. Is there any commonality in the books that fail? (Very large, specific format, etc.)
    all books fail to sync (epub, 15KB - 1MB)

P.S.
Is there a way to disable the mini top menu bar and clock in KyBook3?

Last edited by ilovejedd; 04-01-2019 at 10:09 PM.
ilovejedd is offline   Reply With Quote
Old 04-02-2019, 08:06 AM   #11
karl1c
Connoisseur
karl1c has become a pillar of the MobileRead communitykarl1c has become a pillar of the MobileRead communitykarl1c has become a pillar of the MobileRead communitykarl1c has become a pillar of the MobileRead communitykarl1c has become a pillar of the MobileRead communitykarl1c has become a pillar of the MobileRead communitykarl1c has become a pillar of the MobileRead communitykarl1c has become a pillar of the MobileRead communitykarl1c has become a pillar of the MobileRead communitykarl1c has become a pillar of the MobileRead communitykarl1c has become a pillar of the MobileRead community
 
Posts: 54
Karma: 15568
Join Date: Jul 2017
Device: iPad
@ilovejedd I suspect this was a pathing issue. Unfortunately, I don't have Windows to test it.

I've uploaded a new version that I hope will work. If not, and you're happy to help with some debugging, PM we and I'll try and help you get it working.
karl1c is offline   Reply With Quote
Old 04-02-2019, 11:04 AM   #12
ilovejedd
hopeless n00b
ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.
 
ilovejedd's Avatar
 
Posts: 3,694
Karma: 9002933
Join Date: Jan 2009
Location: in the middle of nowhere
Device: PW4, PW3, iPad 9.7, iPad 12.9
Had a look at the source code and am I correct in guessing that you hard coded the "Calibre Library" folder name and that the plugin assumes it's located in the home directory?

I'm not familiar with python nor programming Calibre plugins but I believe there must be a different method of getting the library path(s).


Not sure what the relevant pieces of code are but I think this is probably a good example.

Job Spy/visualize_metadata_dialog.py
Code:
import os,sys,apsw
from PyQt5.Qt import (Qt, QDialog, QGridLayout, QVBoxLayout, QHBoxLayout, QFont,
                                        QWidget, QSize, QPushButton, QComboBox, QGroupBox, QSplitter )  # for visualizing
from PyQt5.Qt import QFileDialog,QObject,QApplication  # for exporting a .csv

from calibre import isbytestring
from calibre.constants import filesystem_encoding, DEBUG
from calibre.gui2 import gprefs, question_dialog, info_dialog

from calibre_plugins.job_spy.qpainter_charts import Chart, PieChart, ViewerLegend, ViewerPieChart, DataTable
from calibre_plugins.job_spy.config import prefs

#-----------------------------------------------------------------------------------------
    def apsw_connect_to_library(self):

        my_db = self.gui.library_view.model().db

        path = my_db.library_path
        if isbytestring(path):
            path = path.decode(filesystem_encoding)
        path = path.replace(os.sep, '/')
        path = os.path.join(path, 'metadata.db')
        path = path.replace(os.sep, '/')

        if isbytestring(path):
            path = path.decode(filesystem_encoding)

        if path.endswith("/"):
            path = path[0:-1]

        try:
            my_db =apsw.Connection(path)
            is_valid = True
        except Exception as e:
            if DEBUG: print("path to metadata.db is: ", path)
            if DEBUG: print("error: ", str(e))
            is_valid = False
            return None,None,is_valid

        my_cursor = my_db.cursor()

        mysql = "PRAGMA main.busy_timeout = 15000;"      #PRAGMA busy_timeout = milliseconds;
        my_cursor.execute(mysql)

        return my_db,my_cursor,is_valid

P.S.
Forgot to mention earlier that I had to repackage the zip in order to load the plugin (got rid of the top level KyBook3 Sync directory). Calibre was complaining about __init__.py not being in the correct location. I'd be happy to help debug but please note I use Calibre Portable and I dunno how to run calibre-debug with that since the paths are all different.

Last edited by ilovejedd; 04-02-2019 at 11:07 AM.
ilovejedd is offline   Reply With Quote
Old 04-03-2019, 12:20 AM   #13
ilovejedd
hopeless n00b
ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.
 
ilovejedd's Avatar
 
Posts: 3,694
Karma: 9002933
Join Date: Jan 2009
Location: in the middle of nowhere
Device: PW4, PW3, iPad 9.7, iPad 12.9
Thanks! Tested it and it works great if an entry only has one format. Seems like it only sends thumbnails, though.

Plugin hangs at Metadata Sync when there are multiple formats after the ebooks have been sent to the device. I'll PM you debug logs tomorrow.
ilovejedd is offline   Reply With Quote
Old 04-07-2019, 02:37 PM   #14
karl1c
Connoisseur
karl1c has become a pillar of the MobileRead communitykarl1c has become a pillar of the MobileRead communitykarl1c has become a pillar of the MobileRead communitykarl1c has become a pillar of the MobileRead communitykarl1c has become a pillar of the MobileRead communitykarl1c has become a pillar of the MobileRead communitykarl1c has become a pillar of the MobileRead communitykarl1c has become a pillar of the MobileRead communitykarl1c has become a pillar of the MobileRead communitykarl1c has become a pillar of the MobileRead communitykarl1c has become a pillar of the MobileRead community
 
Posts: 54
Karma: 15568
Join Date: Jul 2017
Device: iPad
I've added the syncing of ratings and some identifiers (notably isbn, oclc, and asin). If your identifiers turn up in KyBook3 as 'id' or not at all, let me know and I'll amend the code to allow for them.
karl1c is offline   Reply With Quote
Old 04-08-2019, 04:22 AM   #15
theengine
Member
theengine began at the beginning.
 
Posts: 15
Karma: 10
Join Date: May 2011
Device: Kindle
Hi,
Thanks very much for this plugin. I've been testing it since the first version.
I've noticed one issue.
I currently use the Calibre content server to transfer a book to Kybook3. When I do this it appears in the download section of the Kybook3 library. When I shut down my iPhone and restart it Kybook3 remembers my place in the book.
However, when I use the plugin to transfer the book to my iPhone, it is just in the Library and when I shutdown and restart my iPhone Kybook3 has forgotten my place in the book.
Is it possible to fix this issue?

Thanks

Last edited by theengine; 04-09-2019 at 04:00 AM.
theengine is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
[GUI Plugin] Goodreads Sync kiwidude Plugins 1192 04-08-2019 08:09 PM
[GUI Plugin] BookFusion Cloud Sync skillachie Plugins 10 02-05-2019 11:28 AM
[Gui Plugin] Cloud Sync kerimlcr Plugins 13 03-06-2018 05:13 AM
[GUI Plugin] Save Virtual Libraries To Column (GUI) chaley Plugins 8 09-24-2017 09:01 PM
[GUI Plugin] Book Sync **Deprecated** kiwidude Plugins 111 06-07-2011 07:47 PM


All times are GMT -4. The time now is 12:23 AM.


MobileRead.com is a privately owned, operated and funded community.