[Metadata Source Plugin] Kobo Books - Page 3

davidfor · 05-05-2019, 11:43 PM

@Kintano: It looks like somewhere some paragraph tags are being stripped. That hasn't happened in the past, so it must be a change in calibre 3.42. I can see another thread with issues where there were two nr tags in a row that has been fixed in calibre 3.42. I'll have to see if that is related.

How exactly are you producing the image from the device? Is it a compilation of screenshots of the book details page? Or something else? It does seem to be doing a new line where there is in the HTML. I'm pretty sure that has changed recently in the firmware, but I'll have to do some testing.

Kintano · 05-06-2019, 02:13 AM

Quote:

Originally Posted by davidfor

How exactly are you producing the image from the device? Is it a compilation of screenshots of the book details page? Or something else?

Since I have no idea how to take a screenshot from an eReader (is it even possible?), I just edited/photshoped my Calibre "normal view" so it matches with what I see on my Kobo. It's a bit barbaric but trust me, it's 100% accurate.

I gave it a new try today, same process, same book but this time with Calibre 3.40.1 and with this version, it FINALLY gives me the wanted outcome. So there is definitely something in Calibre 3.41.3 and 3.42.0 who prevents Kobo Books from behaving like it should.

I checked the recent patchnotes and have found this part in 3.42.0 :
"Fix a regression in the previous release (3.41.3) that caused the conversion of HTML to text in comments when [...] converting downloaded metadata to behave slightly differently."

I don't know if this is related or if it matters. You judge.
Welp, I think I did all I could here.

DNSB · 05-07-2019, 03:52 PM

Quote:

Originally Posted by Kintano

Since I have no idea how to take a screenshot from an eReader (is it even possible?), I just edited/photshoped my Calibre "normal view" so it matches with what I see on my Kobo. It's a bit barbaric but trust me, it's 100% accurate.

In the Kobo eReader.conf file in the .kobo\Kobo directory, look for a header:

Code:

[FeatureSettings]

if the header exists, add a line reading:

Code:

Screenshots=true

under it.

If the FeatureSettings header does not exist, add the following two lines:

Code:

[FeatureSettings]
Screenshots=true

Press the power button to take a screenshot and change the true to false when you are done.

Under Windows, use a text editor such as Notepad++ that respects the Unix/Linux EOL.

DNSB · 05-10-2019, 12:28 AM

I've been noticing an on again off again issue with downloading metadata and cover images from Kobo.

The most recent was Larry Correia & Kacey Ezell's Noir Fatale anthology from Baen. It is on the Kobo site, I added the kobo link to the Ids and still no joy.

Kintano · 05-10-2019, 08:38 AM

Thanks for the tip DNSB!

About your problem, you are talking about this book right ? I manage to get a match only if I don't fill the field "author(s)" (so just with its title and ids) before starting a search.

Probably because there isn't any on the kobo site. For whatever reason, this space is left blank : https://prnt.sc/nmt0xu

DNSB · 05-10-2019, 12:03 PM

Quote:

Originally Posted by Kintano

Thanks for the tip DNSB!

About your problem, you are talking about this book right ? I manage to get a match only if I don't fill the field "author(s)" (so just with its title and ids) before starting a search.

Probably because there isn't any on the kobo site. For whatever reason, this space is left blank : https://prnt.sc/nmt0xu

That's the critter.

davidfor · 05-11-2019, 05:50 AM

Quote:

Originally Posted by DNSB

I've been noticing an on again off again issue with downloading metadata and cover images from Kobo.

The most recent was Larry Correia & Kacey Ezell's Noir Fatale anthology from Baen. It is on the Kobo site, I added the kobo link to the Ids and still no joy.

It worked for me. I set the title to "Noir Fatale" and nothing in title, it found https://www.kobo.com/ebook/noir-fatale. The metadata and cover downloaded with no problems. It also worked when I set the id. It sets the author to "Unknown".

Next time it happens, can post the log?

davidfor · 05-11-2019, 07:13 AM

Quote:

Originally Posted by Kintano

It's the "Publisher" column yes, you are right. My bad, messed a bit with my french there.

So here is a few examples of books with wrong characters in Calibre.
Please ignore the "extra space" I put to prevent parsed text.

https://www.kobo.com/fr/fr/ebook/silo-6
From : Actes Sud Littérature on kobo.com
To : Actes Sud Litt& #233;rature in Calibre

https://www.kobo.com/fr/fr/ebook/la-defense-lincoln
From : Calmann-Lévy
To : Calmann-L& #233;vy

https://www.kobo.com/fr/fr/ebook/le-...-de-l-anneau-3
From : Christian Bourgois éditeur
To : Christian Bourgois & #233;

https://www.kobo.com/fr/fr/ebook/m-le-bord-de-l-abime
From : XO éditions
To : XO & #233;ditions

I can't reproduce this here. Each of those get displayed as accented characters here when I fetch the metadata. It is probably something in the encoding of the characters, but I can't think of a reason it would be different. I'l have to think about it.

davidfor · 05-11-2019, 07:49 AM

Quote:

Originally Posted by Kintano

Huh, ok I take back what I just said about the synopsis. It's now perfect (no line breaks) on eBooks with "simple" sysnopsis like this one https://www.kobo.com/fr/fr/ebook/har...er-d-azkaban-1

But it's a complete mess when it comes with more "elaborated" ones (some bold text, many paragraphs etc) like this one :
https://www.kobo.com/fr/fr/ebook/un-...rt-que-la-peur

So I tried to know why the plugin gave me extra line breaks yesterday and not today and it has something to do with the Caliber version I use. Because I went from 3.41.3 to 3.42.0 this morning.

So I started fresh with a new Caliber 3.41.3 installation, deleting calibre folders, cache, libraries. Just Caliber 3.41.3 and the Kobo Books plugin. I didn't mess with any other option (just disabling the other metadata sources obviously). Then I added my previous ebook (https://www.kobo.com/fr/fr/ebook/un-...rt-que-la-peur) and here is the result : https://cdn.discordapp.com/attachmen...373/3.41.3.png

As you can see, the "Normal view" gives me something very close to the kobo.com synopsis. But once I put it in my Kobo, it somehow breaks (red parts on my pic).

Then I did the same process again but this time with Caliber 3.42.0 + Kobo Books.
New result : https://cdn.discordapp.com/attachmen...122/3.42.0.png
This time, yay no line breaks but the "Normal view" and Kobo one is far different from the kobo.com website (and I just realize that my screens are all in french, I hope it doesn't bother you too much!).

I hope it helps a bit to see more clear.

This is due to a change in calibre in some code the plugin uses to clean the HTML. It is used in a few places, so I am surprised it hasn't broken them as well. I have attached a beta that doesn't use this. It works better for your example above. Not sure if it will cause other problems.

DNSB · 05-11-2019, 09:53 AM

Quote:

Originally Posted by davidfor

It worked for me. I set the title to "Noir Fatale" and nothing in title, it found https://www.kobo.com/ebook/noir-fatale. The metadata and cover downloaded with no problems. It also worked when I set the id. It sets the author to "Unknown".

Next time it happens, can post the log?

Here's the log with the (ed) after the editors names:

Spoiler:

Removed the (ed) after the editors names in the author field:

Spoiler:

If I leave the author field blank in calibre, it works and finds the book. Which makes sense since the returned author field is "". Possibly return a match on title only if no author information is returned?

davidfor · 05-11-2019, 10:44 AM

Quote:

Originally Posted by DNSB

Here's the log with the (ed) after the editors names:

Spoiler:

Removed the (ed) after the editors names in the author field:

Spoiler:

If I leave the author field blank in calibre, it works and finds the book. Which makes sense since the returned author field is "". Possibly return a match on title only if no author information is returned?

I hadn't set the author, so it worked. And at the place the book is being rejected, allowing no authors is a valid thing. The attached beta should do it.

DNSB · 05-12-2019, 12:23 AM

Quote:

Originally Posted by davidfor

I hadn't set the author, so it worked. And at the place the book is being rejected, allowing no authors is a valid thing. The attached beta should do it.

Worked fine to fetch metadata and covers on the three books I remembered having issues with and left the current authors in place.

davidfor · 07-13-2019, 10:21 AM

I have just released a new version of the plugin with the following changes:

Fix: Remove "sanitize_comments_html" from comments to workaround change in this in calibre 3.42.0.
Fix: If Kobo has no authors, do not reject.
Fix: Page layout change.

This should be announced by calibre in the next hour or so. If there are any problem, please report them here.

gldnbrwncouches · 01-03-2020, 12:44 PM

I'm having an issue installing this plugin in Calibre 4.5 (installed from Fedora 31 distro repo). After downloading the plugin I made the following changes in lines 10-13 in __init__.py to make it compatible with import statements for Python 3 libraries.

import time, re
from html.parser import HTMLParser
from urllib.parse import quote, unquote
from queue import Queue, Empty

After those changes and re-zipping the files, it seems to install and work just fine for me. Has anyone else had this issue? Should it have worked fine in my version of Calibre without the changes I made? Is there a repo where I can submit a pull request (maybe one that can somehow import according to python version)?

Thanks @davidfor for this excellent plugin!

davidfor · 01-03-2020, 07:06 PM

Quote:

Originally Posted by gldnbrwncouches

I'm having an issue installing this plugin in Calibre 4.5 (installed from Fedora 31 distro repo). After downloading the plugin I made the following changes in lines 10-13 in __init__.py to make it compatible with import statements for Python 3 libraries.

import time, re
from html.parser import HTMLParser
from urllib.parse import quote, unquote
from queue import Queue, Empty

After those changes and re-zipping the files, it seems to install and work just fine for me. Has anyone else had this issue? Should it have worked fine in my version of Calibre without the changes I made? Is there a repo where I can submit a pull request (maybe one that can somehow import according to python version)?

Thanks @davidfor for this excellent plugin!

Those are roughly the changes I have for the Python 3, but with checks to be backwardly compatible. I haven't been any hurry to release it. I will in the next week if people are using Python 3 already.

07-13-2019, 10:21 AM	#43
davidfor Grand Sorcerer Posts: 24,907 Karma: 47303748 Join Date: Jul 2011 Location: Sydney, Australia Device: Kobo:Touch,Glo, AuraH2O, GloHD,AuraONE, ClaraHD, Libra H2O; tolinoepos	Update to version 1.7.0 I have just released a new version of the plugin with the following changes: Fix: Remove "sanitize_comments_html" from comments to workaround change in this in calibre 3.42.0. Fix: If Kobo has no authors, do not reject. Fix: Page layout change. This should be announced by calibre in the next hour or so. If there are any problem, please report them here.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
[Metadata Source Plugin] Goodreads	kiwidude	Plugins	730	03-23-2024 10:24 PM
[Metadata Source Plugin] BOL.com (DE)	Pr.BarnArt	Plugins	52	06-01-2023 04:50 PM
[Metadata Source Plugin] Comicvine	chewt0y	Plugins	88	07-11-2022 12:00 PM
[Metadata Source Plugin] INMONDADORI	Pr.BarnArt	Plugins	7	12-03-2021 12:56 PM

05-05-2019, 11:43 PM	#31
davidfor Grand Sorcerer Posts: 24,907 Karma: 47303748 Join Date: Jul 2011 Location: Sydney, Australia Device: Kobo:Touch,Glo, AuraH2O, GloHD,AuraONE, ClaraHD, Libra H2O; tolinoepos	@Kintano: It looks like somewhere some paragraph tags are being stripped. That hasn't happened in the past, so it must be a change in calibre 3.42. I can see another thread with issues where there were two nr tags in a row that has been fixed in calibre 3.42. I'll have to see if that is related. How exactly are you producing the image from the device? Is it a compilation of screenshots of the book details page? Or something else? It does seem to be doing a new line where there is in the HTML. I'm pretty sure that has changed recently in the firmware, but I'll have to do some testing.

05-10-2019, 12:28 AM	#34
DNSB Bibliophagist Posts: 35,428 Karma: 145525534 Join Date: Jul 2010 Location: Vancouver Device: Kobo Sage, Forma, Clara HD, Lenovo M8 FHD, Paperwhite 4, Tolino epos	I've been noticing an on again off again issue with downloading metadata and cover images from Kobo. The most recent was Larry Correia & Kacey Ezell's Noir Fatale anthology from Baen. It is on the Kobo site, I added the kobo link to the Ids and still no joy.

05-10-2019, 08:38 AM	#35
Kintano Junior Member Posts: 6 Karma: 10 Join Date: May 2019 Device: Kobo Aura H2O	Thanks for the tip DNSB! About your problem, you are talking about this book right ? I manage to get a match only if I don't fill the field "author(s)" (so just with its title and ids) before starting a search. Probably because there isn't any on the kobo site. For whatever reason, this space is left blank : https://prnt.sc/nmt0xu

01-03-2020, 12:44 PM	#44
gldnbrwncouches Enthusiast Posts: 40 Karma: 132 Join Date: Mar 2019 Device: Kobo Forma, Libra H2O & Clara HD, Nook ST w/GL, GL+ (2015), Kindle PW4	I'm having an issue installing this plugin in Calibre 4.5 (installed from Fedora 31 distro repo). After downloading the plugin I made the following changes in lines 10-13 in __init__.py to make it compatible with import statements for Python 3 libraries. import time, re from html.parser import HTMLParser from urllib.parse import quote, unquote from queue import Queue, Empty After those changes and re-zipping the files, it seems to install and work just fine for me. Has anyone else had this issue? Should it have worked fine in my version of Calibre without the changes I made? Is there a repo where I can submit a pull request (maybe one that can somehow import according to python version)? Thanks @davidfor for this excellent plugin!