[Metadata Source Plugin] Kobo Books

[davidfor]

Originally Posted by JSWolf

I can modify the code as long as I know where to put the new function. Where does it go?

Honestly, @13xforever has given enough instructions to do it. If you do not understand what was posted, then you probably shouldn't try.

But, attached is a beta that is working here. It hasn't been done in exactly the way that @13xforever did it, as I'm still thinking and experimenting. And gzip is specified elsewhere. I'm loathe to release it as using such a specific user agent is probably going to fail soon.

[Shpounz]

Thank you, this beta works well.
A small detail : tags were previously downloaded in my language / language of the book (French), they are now in English

[JSWolf]

Originally Posted by davidfor

Honestly, @13xforever has given enough instructions to do it. If you do not understand what was posted, then you probably shouldn't try.

But, attached is a beta that is working here. It hasn't been done in exactly the way that @13xforever did it, as I'm still thinking and experimenting. And gzip is specified elsewhere. I'm loathe to release it as using such a specific user agent is probably going to fail soon.

Thanks. It's working.

As for the other code, I just needed to know where to place it and I would have been OK.

[DNSB]

Originally Posted by JSWolf

Thanks. It's working.

As for the other code, I just needed to know where to place it and I would have been OK.

Enough information to do that was in the original post.

[RobertJSawyer]

@Davidfor, this is terrific! Thank you! I installed your beta version from message #121 and ran a test using Calibre 4.23. With this beta version and Kobo selected as the only metadata source, Calibre successfully downloaded metadata and covers for almost all of the last 20 books I bought from Kobo.

Thank you again!

[gguigui]

It works again! Thank you very much @Davidfor

[Uncle Robin]

Adding my voice of thanks for the fix - it's great to have my primary source of books back 'online' in Calibre, THANK YOU!

[trowgundam]

Thanks for the beta update above. It's working well for me. One thing I have noticed though is it seems Kobo is presenting Series differently and it seems to have broken how the plugin retrieves volume numbers. It'll get the series name, but everything is showing as Volume 1, although I can look on Kobo and see that it has the proper volume number listed. This seems like a fairly recent change as when I first used the beta for a few books it found the volume number just fine, but when I went to add some today it wasn't working. Its been a couple weeks between, so I'm not sure when it started occurring exactly.

[davidfor]

Originally Posted by trowgundam

Thanks for the beta update above. It's working well for me. One thing I have noticed though is it seems Kobo is presenting Series differently and it seems to have broken how the plugin retrieves volume numbers. It'll get the series name, but everything is showing as Volume 1, although I can look on Kobo and see that it has the proper volume number listed. This seems like a fairly recent change as when I first used the beta for a few books it found the volume number just fine, but when I went to add some today it wasn't working. Its been a couple weeks between, so I'm not sure when it started occurring exactly.

Can you give me some examples? I haven't noticed any issues, but, I also haven't added a new book with a series for a couple of weeks.

[Uncle Robin]

I can confirm the "everything is series number (1)" bug. It has become the norm for books I've bought in series, the most recent was a book I bought 2 days ago. The 1st screenshot shows the book in the Kobo store, the second the listing for that version's ISBN in the metadata import plugin and the third shows the listing for other editions.

It has become so routine that I tend to mix and match the metadata download - I prefer the books to have the covers match the copy I bought from Kobo, but I also prefer not having to continually change the series number.

[DNSB]

Originally Posted by davidfor

Can you give me some examples? I haven't noticed any issues, but, I also haven't added a new book with a series for a couple of weeks.

On the series number bug, I've found the same where every book in a series does not parse the series index. From what I saw, it appears the books are showing the series name and then a two digit series index while the plugin seems to be look for a #number as the series index. I copied the log for one download in the spoiler tags.

Spoiler:

Running identify query with parameters: 
{'title': 'Demi-Diety', 'authors': ['Michael Anderle'], 'identifiers': {'isbn': '9781649711823', 'kobo': 'demi-diety'}, 'timeout': 30} 
Using plugins: Goodreads (1, 5, 3), Kobo Books (1, 8, 4), Smashwords Metadata (1, 0, 2), Amazon.com (1, 2, 24) 
The log from individual plugins is below 

****************************** Goodreads (1, 5, 3) ****************************** 
Found 0 results 
Downloading from Goodreads took 1.9682505130767822 
identify - start. title=Demi-Diety, authors=['Michael Anderle'], identifiers={'isbn': '9781649711823', 'kobo': 'demi-diety'}
get_goodreads_id_from_identifiers - isbn=9781649711823
get_goodreads_id_using_api - identifiers=9781649711823
Querying using autocomplete API: https://www.goodreads.com/book/auto_complete?format=json&q=9781649711823
JSON Result: b'[]'
Result using autocomplete API: None
Querying: https://www.goodreads.com/search?search_type=books&search[query]=Demi-Diety+Michael+Anderle
no isbn
response: <Element html at 0x281a2e66ef0> 

******************************************************************************** 

****************************** Kobo Books (1, 8, 4) ****************************** 
Found 1 results 
Downloading from Kobo Books took 1.7568421363830566 


--- 
Title               : Demi-Diety
Author(s)           : Michael Anderle
Publisher           : LMBPN Publishing
Tags                : Science Fiction & Fantasy, Science Fiction, Adventure Sci Fi
Series              : Chronicles Of KieraFreya #1
Rating              : 
Published           : 2020-09-24T11:00:00+00:00
Identifiers         : kobo:demi-diety, isbn:9781649711823
Comments            : b'<div class="synopsis-description" translate="no"><p><strong>Everyone wants you to fail…CHECK!</strong></p>\n<p><em>“Screw Them!”</em></p>\n<p><strong>What’s a billionaire heiress supposed to do when the world seems hell-bent on stopping her from completing her mission?</strong></p>\n<p>Chloe is on the cusp of finishing her quest, but forces inside and outside the game are colluding to stop her. What is it going to take?</p>\n<p><strong>Grab your copy today to find out if Chloe has what it takes to help KieraFreya.</strong></p>\n<p><strong>And Herself.</strong></p>\n</div> \r\n    ' 
identify - title: "Demi-Diety" authors= "['Michael Anderle']"
KoboBooks url: 'https://www.kobo.com/ebook/demi-diety'
parse_details - kobobooks_id: "demi-diety" 
parse_details - title: "Demi-Diety" 
parse_authors - author: "Michael Anderle"
parse_authors - authors: "['Michael Anderle']"
parse_details - authors: "['Michael Anderle']" 
Script page_metadata= {'@context': 'http://schema.org', '@type': 'Product', 'name': 'Demi-Diety', 'description': 'Read "Demi-Diety Chronicles Of KieraFreya Book 05" by Michael Anderle available from Rakuten Kobo. Everyone wants you to fail…CHECK!\n“Screw Them!”\nWhat’s a billionaire heiress supposed to do when the world seems hell-be...', 'image': 'https://kbimages1-a.akamaihd.net/79dff0d7-6f0c-48d4-95f6-66a6a1b56c24/180/1000/False/demi-diety.jpg', 'releasedate': '2020-09-24T04:00:00', 'sku': '9781649711823', 'gtin13': '9781649711823', 'offers': {'@type': 'Offer', 'price': 5.99, 'priceCurrency': 'CAD', 'availability': 'http://schema.org/InStock', 'url': 'https://www.kobo.com/ca/en/ebook/demi-diety'}, 'brand': 'LMBPN Publishing'}
pubdate from JSON: 2020-09-24 04:00:00
parse_series - series_text: "Chronicles Of KieraFreya" 
parse_series - no # in series name
parse_series - Cannot split series info - series_text=Chronicles Of KieraFreya
parse_series - series_name=Chronicles Of KieraFreya, series_index=None
parse_rating - rating: "0" 

******************************************************************************** 

****************************** Smashwords Metadata (1, 0, 2) ****************************** 
Found 0 results 
Downloading from Smashwords Metadata took 2.344212532043457 
Querying: https://www.smashwords.com/books/search?query=9781649711823
No matches found with identifiers, retrying using title and authors. Query: 'https://www.smashwords.com/books/search?query=9781649711823'
Querying: https://www.smashwords.com/books/search?query=Demi-Diety
No Full Search section. Has the page changed?
Building match list from found_ids: []
Matches: []
No matches found with query: 'https://www.smashwords.com/books/search?query=Demi-Diety' 

******************************************************************************** 

****************************** Amazon.com (1, 2, 24) ****************************** 
Found 0 results 
Downloading from Amazon.com took 2.768747091293335 
User-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4389.114 Safari/537.36
Server: amazon
No matches found with identifiers, retrying using only title and authors. Query: 'https://www.amazon.ca/s/?search-alias=aps&unfiltered=1&sort=relevancerank&field-isbn=9781649711823'
User-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4389.114 Safari/537.36
Server: amazon
No matches found with query: 'https://www.amazon.ca/s/?search-alias=stripbooks&unfiltered=1&sort=relevancerank&field-title=Demi-Diety&field-author=Michael+Anderle' 

******************************************************************************** 
The identify phase took 2.87 seconds 
The longest time (2.768747) was taken by: Amazon.com 
Merging results from different sources 
We have 1 merged results, merging took: 0.00 seconds

[ownedbycats]

I also had the series number bug happen a little while ago, but I forgot what book it was in order to replicate.

[Uncle Robin]

Originally Posted by ownedbycats

I also had the series number bug happen a little while ago, but I forgot what book it was in order to replicate.

I just checked the last 10 Kobo books that were part of a series I imported to Calibre, and two of them listed the series position correctly - the two that were ACTUALLY first in their series.

[ownedbycats]

Unchecking all other metadata sources I am able to replicate it:





("Traditional British"? It takes place in Mississippi...)

[davidfor]

I have just updated the plugin to version 1.9.0. This is to fix the problem with series info and handle the other site changes.

The changes in this release are:

• Fix: Site change for Series info
• Fix: Set some request headers to one that the site seems to find acceptable.

Calibre will announce the release in the next hour or so. As usual, if there are any problems or suggestions, please post them here.