[Metadata Source Plugin] Kobo Books

[Aerandir14]

Hi,
I can't get the metadatas from Kobo Books.
Here is the log:

calibre, version 3.27.1
ERROR: No matches found: <p>Failed to find any books that match your search. Try making the search <b>less specific</b>. For example, use only the author's last name and a single distinctive word from the title.<p>To see the full log, click "Show details".

Running identify query with parameters:
{u'title': u'\xdcbel Blatt, Vol. 1', u'authors': [u'Etorouji Shiono'], u'identifiers': {}, u'timeout': 30}
Using plugins: Kobo Books (1, 3, 1)
The log from individual plugins is below

****************************** Kobo Books (1, 3, 1) ******************************
Found 0 results
Downloading from Kobo Books took 7.10299992561
identify - title: "Ãœbel Blatt, Vol. 1" authors= "[u'Etorouji Shiono']"
create_query - title: "Ãœbel Blatt, Vol. 1"
create_query - after decode title: "Ubel Blatt, Vol. 1"
create_query - title_tokens: "[u'Ubel', u'Blatt', u'Vol', u'1']"
Querying: http://store.kobobooks.com/Search?Qu...o&fcmedia=Book
failed using three divs
Have close enough match - title='50 Chefs-D'Å“uvre Que Vous Devez Lire Avant De Mourir : Vol 1 (Golden Deer Classics)', id='50-chefs-d-uvre-que-vous-devez-lire-avant-de-mourir-vol-1-golden-deer-classics'
failed using three divs
Have close enough match - title='Captive-Moi (Vol. 1)', id='captive-moi-vol-1'
failed using three divs
Have close enough match - title='Mine Again - Vol. 1', id='mine-again-vol-1'
failed using three divs
Have close enough match - title='Angel after dark Vol.1', id='angel-after-dark-vol-1'
failed using three divs
Have close enough match - title='Plaisirs Partagés (Roman Érotique, Vol. 1)', id='plaisirs-partages-roman-erotique-vol-1'
KoboBooks url: u'http://store.kobobooks.com/ebook/50-chefs-d-uvre-que-vous-devez-lire-avant-de-mourir-vol-1-golden-deer-classics'
KoboBooks url: u'http://store.kobobooks.com/ebook/captive-moi-vol-1'
KoboBooks url: u'http://store.kobobooks.com/ebook/mine-again-vol-1'
KoboBooks url: u'http://store.kobobooks.com/ebook/angel-after-dark-vol-1'
KoboBooks url: u'http://store.kobobooks.com/ebook/plaisirs-partages-roman-erotique-vol-1'
parse_authors - author: "Kahlen Aymes"
Rejecting authors as not a close match: Kahlen Aymes
Could not find title/authors/KoboBooks id for u'http://store.kobobooks.com/ebook/angel-after-dark-vol-1'
Kobo Books: u'angel-after-dark-vol-1' Title: 'Angel after dark Vol.1' Authors: None
parse_authors - author: "Christina Ross"
Rejecting authors as not a close match: Christina Ross
Could not find title/authors/KoboBooks id for u'http://store.kobobooks.com/ebook/captive-moi-vol-1'
Kobo Books: u'captive-moi-vol-1' Title: 'Captive-Moi (Vol. 1)' Authors: None
parse_authors - author: "Louise Valmont"
Rejecting authors as not a close match: Louise Valmont
Could not find title/authors/KoboBooks id for u'http://store.kobobooks.com/ebook/mine-again-vol-1'
Kobo Books: u'mine-again-vol-1' Title: 'Mine Again - Vol. 1' Authors: None
parse_authors - author: "Mark Twain,Stendhal,Edgar Allan Poe,Victor Hugo,Gustave Flaubert,Alexandre Dumas,Arthur Conan Doyle,René Descartes,Lewis Carroll,Charles Baudelaire,Guillaume Apollinaire,Golden Deer Classics"
Rejecting authors as not a close match: Mark Twain,Stendhal,Edgar Allan Poe,Victor Hugo,Gustave Flaubert,Alexandre Dumas,Arthur Conan Doyle,René Descartes,Lewis Carroll,Charles Baudelaire,Guillaume Apollinaire,Golden Deer Classics
Could not find title/authors/KoboBooks id for u'http://store.kobobooks.com/ebook/50-chefs-d-uvre-que-vous-devez-lire-avant-de-mourir-vol-1-golden-deer-classics'
Kobo Books: u'50-chefs-d-uvre-que-vous-devez-lire-avant-de-mourir-vol-1-golden-deer-classics' Title: u"50 Chefs-D'\u0153uvre Que Vous Devez Lire Avant De Mourir : Vol 1 (Golden Deer Classics)" Authors: None
parse_authors - author: "Analia Noir"
Rejecting authors as not a close match: Analia Noir
Could not find title/authors/KoboBooks id for u'http://store.kobobooks.com/ebook/plaisirs-partages-roman-erotique-vol-1'
Kobo Books: u'plaisirs-partages-roman-erotique-vol-1' Title: u'Plaisirs Partag\xe9s (Roman \xc9rotique, Vol. 1)' Authors: None

************************************************** ******************************
The identify phase took 7.24 seconds
The longest time (7.103000) was taken by: Kobo Books
Merging results from different sources and finding earliest publication dates from the worldcat.org service
We have 0 merged results, merging took: 0.00 seconds

However, the book does exist on kobo store, with the exact same title : https://www.kobo.com/fr/en/ebook/ubel-blatt-vol-1-2

Does anybody have an idea about how to solve this issue? I've already tried with the 1.3.1 version

Thanks!

EDIT: It seams like it has something to do with the language. I think it searches in French written books only.

EDIT2 : Adding "&fclanguages=all" at the end of the query seems to have solved the issue. I don't know if it would be possible to get the language from the metadata, and search only for that language, or for all languages otherwise.

[davidfor]

Attached is a long delayed beta. The changes are:

• Fix: Wasn't handling case where there were no tags/genres.
• Fix: Missed fetching ISBN if there was an "Imprint".
• New: Add support for fetching languages and rating and get high resolution cover. Based on code by MR member @qsxwdc.
• Change: Try for better support of other languages in search. Based on suggestion by MR member @Aerandir14.

When I looked at the way the covers are handled by Kobo, I discovered you could set any size and it would generate a cover to fit that. I have changed this to use the size specified in the Tweak "maximum_cover_size". This will default to 1650x2200.

For the other languages, the suggestion by @Aerandir14 seems to work. But, I haven't tested many non-English books, and all my testing comes from an Australia IP address. The IP address does seem to have an affect on the result. If this change doesn't help, I might have to use a country or language specific search. That would need a configuration option and I'd prefer not to do that.

Please test and report any problems, or success.

[davidfor]

I have just released version 1.4.0 of the plugin. The changes in this release are:

• Fix: Wasn't handling case where there were no tags/genres.
• Fix: Missed fetching ISBN if there was an "Imprint".
• New: Add support for fetching languages and rating and get high resolution cover. Based on code by MR member @qsxwdc
• Change: Try for better support of other languages in search. Based on suggestion by MR member @Aerandir14
• New: Add id_from_url to return the identifier.

Most of the changes were in the last beta.

The last item is for a change I made to the metadata editor that has been released in calibre version 3.34. When you press the button next to the identifiers field, if there is a URL in the clipboard, then it will attempt to get an identifier from the URL. That function needs support from the metadata source plugins.

As always, calibre will show update in the next hour or so. And if there are any problems, please report them here.

[jackie_w]

@davidfor,

I just updated this plugin today. I am now getting this error message when I use Edit metadata > Download cover

calibre, version 3.34.0
ERROR: Download failed: Failed to download any covers, click "Show details" for details.

Traceback (most recent call last):
  File "site-packages\calibre\utils\ipc\simple_worker.py", line 289, in main
  File "site-packages\calibre\ebooks\metadata\sources\worker.py", line 132, in single_covers
WindowsError: [Error 183] Cannot create a file when that file already exists: u'D:\\sysfiles\\calibre\\caltemp\\calibre_95odki\\rjfrcf_single_metadata_download\\Kobo Books,,1368,,2100,,jpeg.cover.done'

Despite the error message, many covers are fetched - including one labelled Kobo Books - and double-clicking one of them pulls it into calibre in the normal way.

[davidfor]

Originally Posted by jackie_w

@davidfor,

I just updated this plugin today. I am now getting this error message when I use Edit metadata > Download cover

calibre, version 3.34.0
ERROR: Download failed: Failed to download any covers, click "Show details" for details.

Traceback (most recent call last):
  File "site-packages\calibre\utils\ipc\simple_worker.py", line 289, in main
  File "site-packages\calibre\ebooks\metadata\sources\worker.py", line 132, in single_covers
WindowsError: [Error 183] Cannot create a file when that file already exists: u'D:\\sysfiles\\calibre\\caltemp\\calibre_95odki\\rjfrcf_single_metadata_download\\Kobo Books,,1368,,2100,,jpeg.cover.done'

Despite the error message, many covers are fetched - including one labelled Kobo Books - and double-clicking one of them pulls it into calibre in the normal way.

I don't know what is happening here. That error appears to be in calibre code controlling the download of covers. The file look to be recording what has been done. It should disappear after the downloads are finished. The only way I can see for that error to happen is if the plugin returned two cover images. And I can't think of how that could happen.

Is there anything in the log for the cover download?

[jackie_w]

Originally Posted by davidfor

I don't know what is happening here. That error appears to be in calibre code controlling the download of covers. The file look to be recording what has been done. It should disappear after the downloads are finished. The only way I can see for that error to happen is if the plugin returned two cover images. And I can't think of how that could happen.

Is there anything in the log for the cover download?

There was no extra info in the debug log. In fact the on-screen error message didn't even appear in the debug log

I decided to completely uninstall 'Kobo Books then start over with a new installation.

The good news is that I'm no longer getting an error after the fresh install. I suppose there must have been a problem during my update from 1.3.2 (??? I think) to 1.4.0. The pre-update version was one of your betas.

[davidfor]

Originally Posted by jackie_w

There was no extra info in the debug log. In fact the on-screen error message didn't even appear in the debug log

I should have thought about what I wanted to look for. I wasn't really expecting any extra errors, but, if something was causing multiple covers to be retrieved, then that should have been shown.

I decided to completely uninstall 'Kobo Books then start over with a new installation.

The good news is that I'm no longer getting an error after the fresh install. I suppose there must have been a problem during my update from 1.3.2 (??? I think) to 1.4.0. The pre-update version was one of your betas.

This plus the restart probably cleaned things up. The issue was with a temporary file created in a temporary directory. If something prevented the cleanup, then there might have been a clash.

[davidfor]

I have just updated the plugin to version 1.5.0. The changes are:

• Change: Handle country specific URLs in id_from_url.
• Change: Use HTTPS instead of HTTP.

The first change is for when you use the paste button in the metadata editor when a books URL is in the clipboard. It was only handling the non-country specific URL.

As usual, calibre will announce the update in the next hour or so. And if you see any problems, please report them here.

[Harzmounty]

Hi David,

sorry if I am posting this a second time, just bumped in this thread.
Is the following eror message probably generated from your pluging?
-------------
calibre, version 3.37.0
FEHLER: Fehler: Fehler beim Kommunizieren mit dem Gerät

float argument required, not NoneType

Traceback (most recent call last):
File "site-packages\calibre\gui2\device.py", line 87, in run
File "site-packages\calibre\gui2\device.py", line 540, in _sync_booklists
File "calibre_plugins.kobotouch_extended.device.driver" , line 476, in sync_booklists
File "site-packages\calibre\devices\kobo\driver.py", line 968, in sync_booklists
File "site-packages\calibre\devices\kobo\driver.py", line 2513, in update_device_database_collections
File "site-packages\calibre\devices\kobo\driver.py", line 3046, in set_core_metadata
TypeError: float argument required, not NoneType
---------------

I am getting this since today when transferring books to the reader. Last transferes were made without error probably a week ago.

The books are copied OK, but they aren't listed as "On the reader" in Calibre.

Reader: Kobo Aura One
PC: Windows 10 / 1809

Plugins: KoboBooks V 1.5.0
KoboTouchExtended V 3.02

Any idea, suggestion, help?

Thank you
Uwe

[davidfor]

@Harzmounty: Answered in your original thread. And this is a completely inappropriate place for the post. This plugin has absolutely nothing to do with the Kobo devices. Add to that the forum doesn't like multi-posting like this or not giving enough time for a response.

[davidfor]

I have just released version 1.6.0 of the plugin. This fixes problems caused by changes to the "eBook details" section of the book pages. This meant there was an error getting the publishing details and the ISBN.

As usual, calibre will announce the update in the next hour or so. And if you see any problems, please report them here.

[Kintano]

Hello davidfor,
Thanks a lot for this plugin. It's the best metadata source so far and it helped me a lot improving my library.

That being said, I have a couple of trouble with it but I am not sure if it comes from the plugin itself.

My eReader is a Kobo Aura H2O and I am using Kobo books ver 1.6.0.
My library contains both french and english books.

First, when it retrieves metadata for French Editors, KoboBooks is messing a lot with special characters.
e.g.: an Editor "Les Éditions Bouquin" appears like this http://prntscr.com/nkb3k8 in Calibre.
The problem only appears with the collumn "Editor". Everything else looks good, special characters included or not.


Second, Synopsis are correctly dl/shown in Calibre but when I add ebooks on my Kobo, it becomes a bit weird with some random line breaks.

e.g:
From this in Calibre:
A narrative is presented that dwells on the life of miners during the Industrial revolution of France. The book gives the view of the revolution from outside. It explains not only the hideous living conditions of the labourers but also the frustration that lead to the revolution. The book beautifully balances between the individual of society and society itself.

To this in Kobo:
A narrative is presented that dwells on the life of miners
during
the Industrial revolution of France. The book gives the view of the revolution from
outside.
It explains not only the hideous living conditions of the labourers but also the frustration that lead to the revolution. The book beautifully balances between
the
individual of society and society itself.

Once again, thank you for putting so much effort and your time into this plugin.

[davidfor]

Originally Posted by Kintano

Hello davidfor,
Thanks a lot for this plugin. It's the best metadata source so far and it helped me a lot improving my library.

That being said, I have a couple of trouble with it but I am not sure if it comes from the plugin itself.

My eReader is a Kobo Aura H2O and I am using Kobo books ver 1.6.0.
My library contains both french and english books.

First, when it retrieves metadata for French Editors, KoboBooks is messing a lot with special characters.
e.g.: an Editor "Les Éditions Bouquin" appears like this http://prntscr.com/nkb3k8 in Calibre.
The problem only appears with the collumn "Editor". Everything else looks good, special characters included or not.

What column is "Editor"? "Publisher" is the only standard column I can think of that might match. Can you give me some example books? It is probably in how the values are encoded, but, I need examples to see what Kobo is doing and what I'm not.

Second, Synopsis are correctly dl/shown in Calibre but when I add ebooks on my Kobo, it becomes a bit weird with some random line breaks.

e.g:
From this in Calibre:
A narrative is presented that dwells on the life of miners during the Industrial revolution of France. The book gives the view of the revolution from outside. It explains not only the hideous living conditions of the labourers but also the frustration that lead to the revolution. The book beautifully balances between the individual of society and society itself.

To this in Kobo:
A narrative is presented that dwells on the life of miners
during
the Industrial revolution of France. The book gives the view of the revolution from
outside.
It explains not only the hideous living conditions of the labourers but also the frustration that lead to the revolution. The book beautifully balances between
the
individual of society and society itself.

I haven't seen any problems like this. What does the HTML code look like for comment? If you look at the HTML version of the comment in the metadata editor, are there line breaks at the same places? Is the Kobo version from the database, or are you retyping it to match.

Also, are you using a metadata plugboard when sending the books to the device? If so, that might have an error and not producing exactly what you want.

And again, if this is coming from the Kobo store, I need some examples to look at.

[Kintano]

It's the "Publisher" column yes, you are right. My bad, messed a bit with my french there.

So here is a few examples of books with wrong characters in Calibre.
Please ignore the "extra space" I put to prevent parsed text.

https://www.kobo.com/fr/fr/ebook/silo-6
From : Actes Sud Littérature on kobo.com
To : Actes Sud Litt& #233;rature in Calibre

https://www.kobo.com/fr/fr/ebook/la-defense-lincoln
From : Calmann-Lévy
To : Calmann-L& #233;vy

https://www.kobo.com/fr/fr/ebook/le-...-de-l-anneau-3
From : Christian Bourgois éditeur
To : Christian Bourgois & #233;

https://www.kobo.com/fr/fr/ebook/m-le-bord-de-l-abime
From : XO éditions
To : XO & #233;ditions



Concerning the Synopsis case, there are line breaks too at the exact position in the HTML view too... I have no idea what I did wrong the first times I retrieved metadata with Kobo Books but I did it again this morning and there are no line breaks anymore and it looks good on both Calibre and my Kobo so nevermind.

It probably came from me messing a lot with all the plugins/options of Calibre. I am beginning with it and the beast is not that easy to use to be frank.
Sorry to bother you with this one.


Thank you for being that responsive by the way.

[Kintano]

Huh, ok I take back what I just said about the synopsis. It's now perfect (no line breaks) on eBooks with "simple" sysnopsis like this one https://www.kobo.com/fr/fr/ebook/har...er-d-azkaban-1

But it's a complete mess when it comes with more "elaborated" ones (some bold text, many paragraphs etc) like this one :
https://www.kobo.com/fr/fr/ebook/un-...rt-que-la-peur

So I tried to know why the plugin gave me extra line breaks yesterday and not today and it has something to do with the Caliber version I use. Because I went from 3.41.3 to 3.42.0 this morning.


So I started fresh with a new Caliber 3.41.3 installation, deleting calibre folders, cache, libraries. Just Caliber 3.41.3 and the Kobo Books plugin. I didn't mess with any other option (just disabling the other metadata sources obviously). Then I added my previous ebook (https://www.kobo.com/fr/fr/ebook/un-...rt-que-la-peur) and here is the result : https://cdn.discordapp.com/attachmen...373/3.41.3.png

As you can see, the "Normal view" gives me something very close to the kobo.com synopsis. But once I put it in my Kobo, it somehow breaks (red parts on my pic).


Then I did the same process again but this time with Caliber 3.42.0 + Kobo Books.
New result : https://cdn.discordapp.com/attachmen...122/3.42.0.png
This time, yay no line breaks but the "Normal view" and Kobo one is far different from the kobo.com website (and I just realize that my screens are all in french, I hope it doesn't bother you too much!).


I hope it helps a bit to see more clear.