Thread: Downloading metadata: Blank description overriding?
View Single Post
Old 12-23-2022, 09:02 PM   #5
kiwidude
Calibre Plugins Developer
kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.
 
Posts: 4,720
Karma: 2197770
Join Date: Oct 2010
Location: Australia
Device: Kindle Oasis
Quote:
Originally Posted by kovidgoyal View Post
When merging comments during metadata download calibre picks the longest. The goodreads plugin is probably setting the comments to a string with spaces or similar. It should instead set it to None or the empty string, or just not set it at all when no comments are present.

See the merge() function in sources/identify.py
I have checked the Goodreads plugin code. It was returning an empty string in this circumstance. I have tried changing the Goodreads code to ensure it returns None, and I am *still* seeing the same problem reported above from the merge that takes place with the Amazon result.

To replicate, create this empty book:
Title: Lost Galaxy: The Complete Series
Author: Daniel Young
Identifiers: amazon:B0B8G1287N, goodreads:61991150

You can use the attached version of the Goodreads plugin that ensures it always leaves mi.comments as None rather than empty string.

Spoiler:

Running identify query with parameters:
{'title': 'Lost Galaxy: The Complete Series', 'authors': ['Young, Daniel'], 'identifiers': {'amazon': 'B0B8G1287N', 'goodreads': '61991150'}, 'timeout': 30}
Using plugins: Goodreads (1, 7, 2), Amazon.com (1, 3, 3)
The log from individual plugins is below

****************************** Goodreads (1, 7, 2) ******************************
Found 1 results
Downloading from Goodreads took 2.950592517852783


---
Title : Lost Galaxy: The Complete Series (Books 1-6) (Complete Series Box Sets)
Author(s) : Daniel Young
Rating : 2.2
Published : 2022-08-21T00:00:00+00:00
Identifiers : goodreads:61991150, amazon:B0B8G1287N
identify - start. title=Lost Galaxy: The Complete Series, authors=['Young, Daniel'], identifiers={'amazon': 'B0B8G1287N', 'goodreads': '61991150'}
Goodreads book url: 'https://www.goodreads.com/book/show/61991150'
Trying to parse book json for 2022 web page format
Page is legacy html format as NO Json found
parse_title: title_text='Lost Galaxy: The Complete Series (Books 1-6) (Complete Series Box Sets)'
parse_authors_legacy - author: Daniel Young
parse_authors_legacy - author: Daniel Young
parse_details - goodreads_id: 61991150, mi: Title : Lost Galaxy: The Complete Series (Books 1-6) (Complete Series Box Sets)
Author(s) : Daniel Young
parse_rating_legacy: 4.4
parse_comments: None
parse_publisher_and_date_legacy: Publisher: None
parse_publisher_and_date_legacy: 2022-08-21 00:00:00+00:00 (Edition)

************************************************** ******************************

****************************** Amazon.com (1, 3, 3) ******************************
Found 1 results
Downloading from Amazon.com took 3.9161298274993896


---
Title : Lost Galaxy: The Complete Series
Author(s) : Daniel Young
Identifiers : amazon:B0B8G1287N
Comments : <p>AN INVASION. A CONSPIRACY. ONE SHOT AT REDEMPTION.</p>
<p><strong>The</strong> Lost Galaxy <strong>series</strong> follows the adventures of a lone lawman, his aging and possibly insane spaceship, and a trio of mercenaries, new recruits, and hardened criminals who are all swept together behind an alien invasion that could wipe out humanity.</p>
<p>As Provincial Sergeant Jack Bowman and his ragtag group race to save themselves and warn humanity of the invasion force, they learn that there is more at stake than anyone—human or alien—could have guessed.</p>
<p>A conspiracy could bring down everything in the galaxy...</p>
<p><em>Lost Galaxy: The Complete Series</em> is old-fashioned space opera at its best, perfect for fans of thrilling pulp sci-fi of yesteryear. <strong>If you're interested in hard science fiction look elsewhere. But if you're looking for nonstop adventures on strange worlds with even stranger aliens, this series is for you!</strong></p>
<p>Includes all 6 books in the Lost Galaxy series:</p>
<ul>
<li><em>The Last Enemy</em></li>
<li><em>Beyond Revolt</em></li>
<li><em>Shadowland</em></li>
<li><em>The Last Deceit</em></li>
<li><em>Swept Away</em></li>
<li><em>On Redemption</em></li>
</ul>
<p><strong>Five star reviews for the first 3 books in the series:</strong></p>
<p><em>★★★★★ "Lots of great space battles and character development with plenty of background."</em></p>
<p><em>★★★★★ "I can honestly say that I didn't even have to get 10 pages into this book set and I was hooked."</em></p>
<p><em>★★★★★ "Quite a gripping and action packed adrenaline fest."</em></p>
<p><em>★★★★★ "Another start of an amazing series that I got sucked into and could not put down."</em></p>
<p><em>★★★★★ "Full of action, suspense, intrigue and deceit throughout the series."</em></p>
User-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.87 Safari/537.36
Server: auto
Making google query: https://www.google.com/search?q=B0B8...www.amazon.com
Ignoring 'Customer reviews: Lost Galaxy: The Complete Series (Books ...https://www.amazon.com › B0B8G1287N › pd_rd_i=B0B...' as it has no cached page
Ignoring 'Customer reviews: Lost Galaxy: The Complete Series (Books ...https://www.amazon.com › B0B8G1287N › pd_rd_i=B0B...' as it has no cached page
Ignoring 'Customer reviews: Lost Galaxy: The Complete Series (Books ...https://www.amazon.com › B0B8G1287N › pd_rd_i=B0B...' as it has no cached page
Ignoring 'Customer reviews: Lost Galaxy: The Complete Series (Books ...https://www.amazon.com › B0B8G1287N › pd_rd_i=B0B...' as it has no cached page
Ignoring 'Customer reviews: Lost Galaxy: The Complete Series (Books ...https://www.amazon.com › B0B8G1287N › pd_rd_i=B0B...' as it has no cached page
Skipping non-book result: Result(url='https://www.amazon.com/product-reviews/B0B8G1287N/ref=zg_bsnr_9059886011_cr_sccl_7/000-0000000-0000000?pd_rd_i=B0B8G1287N', title='Amazon.com: Customer reviews: Lost Galaxyhttps://www.amazon.com › B0B8G1287N › pd_rd_i=B...', cached_url='https://webcache.googleusercontent.com/search?q=cache:66HjyMxDkjQJ:https://www.amazon.com/product-revie...&ct=clnk&gl=au')
Skipping non-book result: Result(url='https://www.amazon.com/product-reviews/B0B8G1287N/ref=zg_bsnr_9059886011_cr_sccl_7/000-0000000-0000000?pd_rd_i=B0B8G1287N', title='Amazon.com: Customer reviews: Lost Galaxyhttps://www.amazon.com › B0B8G1287N › pd_rd_i=B...', cached_url='https://webcache.googleusercontent.com/search?q=cache:66HjyMxDkjQJ:https://www.amazon.com/product-revie...&ct=clnk&gl=au')
Skipping non-book result: Result(url='https://www.amazon.com/product-reviews/B0B8G1287N/ref=zg_bsnr_9059886011_cr_sccl_7/000-0000000-0000000?pd_rd_i=B0B8G1287N', title='Amazon.com: Customer reviews: Lost Galaxyhttps://www.amazon.com › B0B8G1287N › pd_rd_i=B...', cached_url='https://webcache.googleusercontent.com/search?q=cache:66HjyMxDkjQJ:https://www.amazon.com/product-revie...&ct=clnk&gl=au')
Skipping non-book result: Result(url='https://www.amazon.com/product-reviews/B0B8G1287N/ref=zg_bsnr_9059886011_cr_sccl_7/000-0000000-0000000?pd_rd_i=B0B8G1287N', title='Amazon.com: Customer reviews: Lost Galaxyhttps://www.amazon.com › B0B8G1287N › pd_rd_i=B...', cached_url='https://webcache.googleusercontent.com/search?q=cache:66HjyMxDkjQJ:https://www.amazon.com/product-revie...&ct=clnk&gl=au')
Skipping non-book result: Result(url='https://www.amazon.com/product-reviews/B0B8G1287N/ref=zg_bsnr_9059886011_cr_sccl_7/000-0000000-0000000?pd_rd_i=B0B8G1287N', title='Amazon.com: Customer reviews: Lost Galaxyhttps://www.amazon.com › B0B8G1287N › pd_rd_i=B...', cached_url='https://webcache.googleusercontent.com/search?q=cache:66HjyMxDkjQJ:https://www.amazon.com/product-revie...&ct=clnk&gl=au')
Skipping non-book result: Result(url='https://www.amazon.com/product-reviews/B0B8G1287N/ref=dbs_r_series', title='Customer reviews: Lost Galaxy - Amazon.comhttps://www.amazon.com › B0B8G1287N', cached_url='https://webcache.googleusercontent.com/search?q=cache:rMX8jXAYOuEJ:https://www.amazon.com/product-revie...&ct=clnk&gl=au')
Skipping non-book result: Result(url='https://www.amazon.com/product-reviews/B0B8G1287N/ref=dbs_r_series', title='Customer reviews: Lost Galaxy - Amazon.comhttps://www.amazon.com › B0B8G1287N', cached_url='https://webcache.googleusercontent.com/search?q=cache:rMX8jXAYOuEJ:https://www.amazon.com/product-revie...&ct=clnk&gl=au')
Skipping non-book result: Result(url='https://www.amazon.com/product-reviews/B0B8G1287N/ref=dbs_r_series', title='Customer reviews: Lost Galaxy - Amazon.comhttps://www.amazon.com › B0B8G1287N', cached_url='https://webcache.googleusercontent.com/search?q=cache:rMX8jXAYOuEJ:https://www.amazon.com/product-revie...&ct=clnk&gl=au')
Skipping non-book result: Result(url='https://www.amazon.com/product-reviews/B0B8G1287N/ref=dbs_r_series', title='Customer reviews: Lost Galaxy - Amazon.comhttps://www.amazon.com › B0B8G1287N', cached_url='https://webcache.googleusercontent.com/search?q=cache:rMX8jXAYOuEJ:https://www.amazon.com/product-revie...&ct=clnk&gl=au')
Skipping non-book result: Result(url='https://www.amazon.com/product-reviews/B0B8G1287N/ref=dbs_r_series', title='Customer reviews: Lost Galaxy - Amazon.comhttps://www.amazon.com › B0B8G1287N', cached_url='https://webcache.googleusercontent.com/search?q=cache:rMX8jXAYOuEJ:https://www.amazon.com/product-revie...&ct=clnk&gl=au')
Skipping non-book result: Result(url='https://www.amazon.com/Daniel-Young/e/B07Y7KJQN5/ref=d_pd_sim_bl_sccl_2_6/000-0000000-0000000?pd_rd_w=nyX19&content-id=amzn1.sym.cea269b5-2b99-44d4-a309-214a6674c0a9&pf_rd_p=cea269b5-2b99-44d4-a309-214a6674c0a9&pf_rd_r=YGPJ6GJE0YBQD26BB3WD&pd_rd_wg =Xfqnr&pd_rd_r=98fc447b-e2e2-4374-acbe-09d4c0ee6f00&pd_rd_i=B0B8G1287N', title='Untitledhttps://www.amazon.com › Daniel-Young › pd_rd_i=B...', cached_url='https://webcache.googleusercontent.com/search?q=cache:Ix-LBEMWiYcJ:https://www.amazon.com/Daniel-Young/...&ct=clnk&gl=au')
Skipping non-book result: Result(url='https://www.amazon.com/Daniel-Young/e/B07Y7KJQN5/ref=d_pd_sim_bl_sccl_2_6/000-0000000-0000000?pd_rd_w=nyX19&content-id=amzn1.sym.cea269b5-2b99-44d4-a309-214a6674c0a9&pf_rd_p=cea269b5-2b99-44d4-a309-214a6674c0a9&pf_rd_r=YGPJ6GJE0YBQD26BB3WD&pd_rd_wg =Xfqnr&pd_rd_r=98fc447b-e2e2-4374-acbe-09d4c0ee6f00&pd_rd_i=B0B8G1287N', title='Untitledhttps://www.amazon.com › Daniel-Young › pd_rd_i=B...', cached_url='https://webcache.googleusercontent.com/search?q=cache:Ix-LBEMWiYcJ:https://www.amazon.com/Daniel-Young/...&ct=clnk&gl=au')
Skipping non-book result: Result(url='https://www.amazon.com/Daniel-Young/e/B07Y7KJQN5/ref=d_pd_sim_bl_sccl_2_6/000-0000000-0000000?pd_rd_w=nyX19&content-id=amzn1.sym.cea269b5-2b99-44d4-a309-214a6674c0a9&pf_rd_p=cea269b5-2b99-44d4-a309-214a6674c0a9&pf_rd_r=YGPJ6GJE0YBQD26BB3WD&pd_rd_wg =Xfqnr&pd_rd_r=98fc447b-e2e2-4374-acbe-09d4c0ee6f00&pd_rd_i=B0B8G1287N', title='Untitledhttps://www.amazon.com › Daniel-Young › pd_rd_i=B...', cached_url='https://webcache.googleusercontent.com/search?q=cache:Ix-LBEMWiYcJ:https://www.amazon.com/Daniel-Young/...&ct=clnk&gl=au')
Skipping non-book result: Result(url='https://www.amazon.com/Daniel-Young/e/B07Y7KJQN5/ref=d_pd_sim_bl_sccl_2_6/000-0000000-0000000?pd_rd_w=nyX19&content-id=amzn1.sym.cea269b5-2b99-44d4-a309-214a6674c0a9&pf_rd_p=cea269b5-2b99-44d4-a309-214a6674c0a9&pf_rd_r=YGPJ6GJE0YBQD26BB3WD&pd_rd_wg =Xfqnr&pd_rd_r=98fc447b-e2e2-4374-acbe-09d4c0ee6f00&pd_rd_i=B0B8G1287N', title='Untitledhttps://www.amazon.com › Daniel-Young › pd_rd_i=B...', cached_url='https://webcache.googleusercontent.com/search?q=cache:Ix-LBEMWiYcJ:https://www.amazon.com/Daniel-Young/...&ct=clnk&gl=au')
Skipping non-book result: Result(url='https://www.amazon.com/Daniel-Young/e/B07Y7KJQN5/ref=d_pd_sim_bl_sccl_2_6/000-0000000-0000000?pd_rd_w=nyX19&content-id=amzn1.sym.cea269b5-2b99-44d4-a309-214a6674c0a9&pf_rd_p=cea269b5-2b99-44d4-a309-214a6674c0a9&pf_rd_r=YGPJ6GJE0YBQD26BB3WD&pd_rd_wg =Xfqnr&pd_rd_r=98fc447b-e2e2-4374-acbe-09d4c0ee6f00&pd_rd_i=B0B8G1287N', title='Untitledhttps://www.amazon.com › Daniel-Young › pd_rd_i=B...', cached_url='https://webcache.googleusercontent.com/search?q=cache:Ix-LBEMWiYcJ:https://www.amazon.com/Daniel-Young/...&ct=clnk&gl=au')
Skipping non-book result: Result(url='https://www.amazon.com/Daniel-Young/e/B07Y7KJQN5/ref=d_pd_sim_bl_sccl_2_6/000-0000000-0000000?pd_rd_w=nyX19&content-id=amzn1.sym.cea269b5-2b99-44d4-a309-214a6674c0a9&pf_rd_p=cea269b5-2b99-44d4-a309-214a6674c0a9&pf_rd_r=YGPJ6GJE0YBQD26BB3WD&pd_rd_wg =Xfqnr&pd_rd_r=98fc447b-e2e2-4374-acbe-09d4c0ee6f00&pd_rd_i=B0B8G1287N', title='Untitledhttps://www.amazon.com › Daniel-Young › pd_rd_i=B...', cached_url='https://webcache.googleusercontent.com/search?q=cache:Ix-LBEMWiYcJ:https://www.amazon.com/Daniel-Young/...&ct=clnk&gl=au')
Downloading details from: https://webcache.googleusercontent.c...&ct=clnk&gl=au
Failed to find product description for url: 'https://webcache.googleusercontent.com/search?q=cache:6t34fmla0xoJ:https://www.amazon.com/Lost-Galaxy-C...&ct=clnk&gl=au'

************************************************** ******************************
The identify phase took 4.10 seconds
The longest time (3.916130) was taken by: Amazon.com
Merging results from different sources
We have 2 merged results, merging took: 0.00 seconds


So if it is a Goodreads plugin problem I'm not quite sure where the problem could be?
Attached Files
File Type: zip Goodreads.zip (119.4 KB, 109 views)
kiwidude is offline   Reply With Quote