[Metadata Source Plugin] BOL.com (NL) - Page 25

bigpapaw · 01-30-2018, 06:07 PM

Hello,

I have trouble scraping metadata with bol_NL. I get no results from Bol_NL. Each time there is the nothing found error. Even for books i know there is an entrance in bol.

As example: The book "Obsessie" from Tatiana de Rosnay.
In calibre i get error message:
calibre, version 3.16.0
ERROR: No matches found: <p>Failed to find any books that match your search. Try making the search <b>less specific</b>. For example, use only the author's last name and a single distinctive word from the title.<p>To see the full log, click "Show details".

Running identify query with parameters:
{u'identifiers': {u'google': u'o5m6DgAAQBAJ', u'isbn': u'9789026339301'}, u'authors': [u'Tatiana de Rosnay'], u'timeout': 30, u'title': u'Obsessie'}
Using plugins: BOL_NL (3, 8, 17)
The log from individual plugins is below

****************************** BOL_NL (3, 8, 17) ******************************
Found 0 results
Downloading from BOL_NL took 0.996000051498
Querying: https://www.bol.com/nl/p/obsessie/9200000077515228/
BOL_NL url: 'https://www.bol.com/nl/p/obsessie/9200000077515228/'
Error parsing title for url: 'https://www.bol.com/nl/p/obsessie/9200000077515228/'
Traceback (most recent call last):
File "calibre_plugins.BOL_NL.worker", line 125, in parse_details
File "calibre_plugins.BOL_NL.worker", line 243, in parse_title
AttributeError: 'NoneType' object has no attribute 'find'

Could not find title for 'https://www.bol.com/nl/p/obsessie/9200000077515228/'
Could not find title/authors/bol_nl_id for 'https://www.bol.com/nl/p/obsessie/9200000077515228/'
bol_nl_id: 'nl/p/obsessie/9200000077515228' Title: None Authors: ['Tatiana de Rosnay']

************************************************** ******************************
The identify phase took 1.00 seconds
The longest time (0.996000) was taken by: BOL_NL
Merging results from different sources and finding earliest publication dates from the worldcat.org service
We have 0 merged results, merging took: 0.00 seconds

I only use now the bol plugin.

In google crome i do get results using the line 'https://www.bol.com/nl/p/obsessie/9200000077515228/'

Why do i get no results in calibre from bol plugin?
please help I need the dutch source

willempie · 01-31-2018, 07:17 AM

Running identify query with parameters:
{u'identifiers': {u'isbn': u'9022581446'}, u'authors': [u'Katie Fforde'], u'title': u'Een tuin vol bloemen', u'timeout': 30}
Using plugins: BOL_NL (3, 8, 17)
The log from individual plugins is below

****************************** BOL_NL (3, 8, 17) ******************************
Found 0 results
Downloading from BOL_NL took 1.68099999428
Querying: https://www.bol.com/nl/p/een-tuin-vo...0000079290702/
BOL_NL url: 'https://www.bol.com/nl/p/een-tuin-vol-bloemen/9200000079290702/'
Error parsing title for url: 'https://www.bol.com/nl/p/een-tuin-vol-bloemen/9200000079290702/'
Traceback (most recent call last):
File "calibre_plugins.BOL_NL.worker", line 125, in parse_details
File "calibre_plugins.BOL_NL.worker", line 243, in parse_title
AttributeError: 'NoneType' object has no attribute 'find'

Could not find title for 'https://www.bol.com/nl/p/een-tuin-vol-bloemen/9200000079290702/'
Could not find title/authors/bol_nl_id for 'https://www.bol.com/nl/p/een-tuin-vol-bloemen/9200000079290702/'
bol_nl_id: 'nl/p/een-tuin-vol-bloemen/9200000079290702' Title: None Authors: ['Katie Fforde']

************************************************** ******************************
The identify phase took 1.81 seconds
The longest time (1.681000) was taken by: BOL_NL
Merging results from different sources and finding earliest publication dates from the worldcat.org service
We have 0 merged results, merging took: 0.00 seconds

Pr.BarnArt · 01-31-2018, 03:53 PM

New update available at the start of this thread. v 3.8.18

Made some changes due to website changes of Bol.nl
and also a fix for downloading covers from literatuurplein

Enjoy!

Wipneus · 02-01-2018, 08:21 AM

Now i get in the comments this:

Op het chique premièrefeest van seizoen 2 van L.A. Candy krioelt het van de fans, en de paparazzi staan te dringen langs de rode loper. De realityshow is een hit en Jane Roberts dé grote ster. Maar onder de oppervlakte smeult er van alles tussen de vier meisjes van de show. Jane weigert scènes te doen met Madison, nadat die compromitterende foto's van haar aan Gossip heeft doorgespeeld. Madison zelf wil koste van het kost achter de identiteit van haar afperser komen, die dreigt te onthullen dat zij niet altijd een rijke, platinablonde Californische babe was... Gaby gedraagt zich opeens totaal anders sinds ze een nieuwe persagent heeft. En Scarlett ten slotte moet zien te laveren tussen haar vriendje en de show, want die twee samen gaan echt niet. En dan is seizoen 2 nog maar net begonnen...
(source: Bol.com 1)

Why starts the comment with spaces and why stands there everytime source: Bol.com 1

Can you change that? And now i get no covers from Bol.com.

(Dutch)
Bedankt voor de aanpassing. Wat nu opvalt is dat bij commentaar vakje dat de samenvatting begint met enkele spaties en onderaan staat nu source: Bol.com 1
Waarvoor staat die 1? Krijg nu ook geen covers van Bol.

ps. Zodra ik dit nu plaats zie je niet dat het begint met enkele spaties terwijl ik het wel zie bij aanpassing van het probleem

Gr. Wipneus

Pr.BarnArt · 02-01-2018, 03:24 PM

Quote:

Originally Posted by Wipneus

Why starts the comment with spaces and why stands there everytime source: Bol.com 1

Can you change that? And now i get no covers from Bol.com.

Thanks for the comments, I made another update for these issues.
Version 3.8.19 available.
Problems with covers were related to the maintenance-mode of the site literatuurplein and a not complete error-handling.

I think the update will work.
So enjoy.

bigpapaw · 02-02-2018, 08:20 PM

Thanks Pr BarnArt for your quick reaction solving my problem, but i found new/another annoying problem.
When you scrape the found information of a book from bol.com_NL to the database it appends the new information instead of delete/replace the existing text.

Hope you can look at it, thanks again

Pr.BarnArt · 02-03-2018, 04:34 AM

Quote:

Originally Posted by bigpapaw

Thanks Pr BarnArt for your quick reaction solving my problem, but i found new/another annoying problem.
When you scrape the found information of a book from bol.com_NL to the database it appends the new information instead of delete/replace the existing text.

Hope you can look at it, thanks again

Hi,

The plugin just delivers the information, where calibre process it
I think you have set the option "append to existing comments" in the configwindow of metadata download. I presume an uncheck will do.

dooiemus · 02-03-2018, 06:19 PM

Hi Pr.,
Different from previous versions the "review" (recenties) data doesn't seem to come through anymore. I have tried several settings: disabling all metadata download plugins except bol-nl and then disable and enable "retrieval of review" option. Haven't seen any review data in all tests. Is this because BOL.NL has changed or is it something in the plugin?

Pr.BarnArt · 02-04-2018, 08:06 AM

Quote:

Originally Posted by dooiemus

Different from previous versions the "review" (recenties) data doesn't seem to come through anymore.

I noticed that with many books there is no review-part anymore at the site.
(p.e. "ik herinner me tinus broederland", which had a review part before).
Can you give an example where the site has a review part in the description.

Pr.BarnArt · 02-06-2018, 02:34 PM

The dead sparrow seems to be dead

dooiemus · 02-09-2018, 07:40 PM

Not dead yet just slooowwww...anyway

Title : De Ondergang
Author(s) : Joachim Fest
Publisher : De Bezige Bij
Languages : dut
Rating : 2.5
Published : 2011-07-02T00:00:00+00:00
Identifiers : isbn:9789023466864

came with a review on jan 29th by Dr JLG v Oudheusden, today just the comment section and no review.

Pr.BarnArt · 02-10-2018, 06:20 PM

Quote:

Originally Posted by dooiemus

Not dead yet just slooowwww...anyway

came with a review on jan 29th by Dr JLG v Oudheusden, today just the comment section and no review.

happy your are still alive

so when reviews are not there , you cannot grab them
maybe ask Bol to put them back in place

thekn · 02-12-2018, 02:21 PM

Hi Pr.BarnArt,

I came across this forum while searching for a solution to an issue when using the BOL_NL plugin. After I start downloading the metadata and covers it results in an error after 2-3 minutes (depending on my query). After checking the log I found the following error messages for each of the books in the search query:

BOL_NL - Version 3.8.19

Code:

****************************** BOL_NL (3, 8, 19) ****************************** 
Found 0 results 
Downloading from BOL_NL took 0.672827005386 
Plugin BOL_NL failed
Traceback (most recent call last):
  File "site-packages/calibre/ebooks/metadata/sources/identify.py", line 48, in run
  File "calibre_plugins.BOL_NL.__init__", line 267, in identify
  File "calibre_plugins.BOL_NL.__init__", line 174, in create_query
  File "lib/python2.7/urllib2.py", line 154, in urlopen
  File "lib/python2.7/urllib2.py", line 435, in open
  File "lib/python2.7/urllib2.py", line 548, in http_response
  File "lib/python2.7/urllib2.py", line 467, in error
  File "lib/python2.7/urllib2.py", line 407, in _call_chain
  File "lib/python2.7/urllib2.py", line 654, in http_error_302
  File "lib/python2.7/urllib2.py", line 435, in open
  File "lib/python2.7/urllib2.py", line 548, in http_response
  File "lib/python2.7/urllib2.py", line 473, in error
  File "lib/python2.7/urllib2.py", line 407, in _call_chain
  File "lib/python2.7/urllib2.py", line 556, in http_error_default
HTTPError: HTTP Error 404: Not Found

Code:

****************************** BOL_NL Covers ****************************** 
Request extra headers: [('User-agent', u'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36')] 
Failed to download valid cover 
Took 0.748715877533 seconds 
No cached cover found, running identify
Failed to download cover from BOL_NL
Traceback (most recent call last):
  File "site-packages/calibre/ebooks/metadata/sources/covers.py", line 49, in run
  File "calibre_plugins.BOL_NL.__init__", line 488, in download_cover
  File "calibre_plugins.BOL_NL.__init__", line 267, in identify
  File "calibre_plugins.BOL_NL.__init__", line 174, in create_query
  File "lib/python2.7/urllib2.py", line 154, in urlopen
  File "lib/python2.7/urllib2.py", line 435, in open
  File "lib/python2.7/urllib2.py", line 548, in http_response
  File "lib/python2.7/urllib2.py", line 467, in error
  File "lib/python2.7/urllib2.py", line 407, in _call_chain
  File "lib/python2.7/urllib2.py", line 654, in http_error_302
  File "lib/python2.7/urllib2.py", line 435, in open
  File "lib/python2.7/urllib2.py", line 548, in http_response
  File "lib/python2.7/urllib2.py", line 473, in error
  File "lib/python2.7/urllib2.py", line 407, in _call_chain
  File "lib/python2.7/urllib2.py", line 556, in http_error_default
HTTPError: HTTP Error 404: Not Found

Code:

****************************** BOL_NL Covers ****************************** 
Request extra headers: [('User-agent', u'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36')] 
Downloaded cover: 512x840 
Took 0.531849861145 seconds 
Downloading cover from: //s.s-bol.com/imgbase0/imagebase3/large/FC/6/3/7/2/9200000028282736.jpg
Failed to download cover from: //s.s-bol.com/imgbase0/imagebase3/large/FC/6/3/7/2/9200000028282736.jpg
Traceback (most recent call last):
  File "calibre_plugins.BOL_NL.__init__", line 513, in download_cover
  File "site-packages/mechanize/_mechanize.py", line 239, in open_novisit
  File "site-packages/mechanize/_mechanize.py", line 270, in _mech_open
BrowserStateError: can't fetch relative reference: not viewing any document

In the cases above it was looking for the book "Rogue Squadron: Star Wars Legends (X-Wing) (Star Wars: X-Wing - Legends)" but the error is repeating for each search item.

I noticed that you are still very active on this forum, hope you will find this information useful. If you need any additional information, just let me know. Hope to use your plug-in very soon!!

Dompie · 02-14-2018, 01:31 PM

Yes every lookup without an ISBN gives a: HTTPError: HTTP Error 404: Not Found
When you have filled the ISBN everything is all right

Pr.BarnArt · 02-14-2018, 04:33 PM

Quote:

Originally Posted by Dompie

Yes every lookup without an ISBN gives a: HTTPError: HTTP Error 404: Not Found
When you have filled the ISBN everything is all right

Ok, I see.
Bol changed again their code, the bol_server now serves several different codes for the searchpages.
I made an update 3.8.20 to tackle the different codes as far as I could explore.
Maybe there will be more variations, please let me know.
Update available at the start of this thread

@Thekn: does this solve your problem too? Otherwise please mention book and author, so I can try to reproduce it.

01-31-2018, 07:17 AM	#362
willempie Junior Member Posts: 9 Karma: 10 Join Date: Jan 2018 Device: none	error message Running identify query with parameters: {u'identifiers': {u'isbn': u'9022581446'}, u'authors': [u'Katie Fforde'], u'title': u'Een tuin vol bloemen', u'timeout': 30} Using plugins: BOL_NL (3, 8, 17) The log from individual plugins is below **************************** BOL_NL (3, 8, 17) ************************** Found 0 results Downloading from BOL_NL took 1.68099999428 Querying: https://www.bol.com/nl/p/een-tuin-vo...0000079290702/ BOL_NL url: 'https://www.bol.com/nl/p/een-tuin-vol-bloemen/9200000079290702/' Error parsing title for url: 'https://www.bol.com/nl/p/een-tuin-vol-bloemen/9200000079290702/' Traceback (most recent call last): File "calibre_plugins.BOL_NL.worker", line 125, in parse_details File "calibre_plugins.BOL_NL.worker", line 243, in parse_title AttributeError: 'NoneType' object has no attribute 'find' Could not find title for 'https://www.bol.com/nl/p/een-tuin-vol-bloemen/9200000079290702/' Could not find title/authors/bol_nl_id for 'https://www.bol.com/nl/p/een-tuin-vol-bloemen/9200000079290702/' bol_nl_id: 'nl/p/een-tuin-vol-bloemen/9200000079290702' Title: None Authors: ['Katie Fforde'] ********************************************** **************************** The identify phase took 1.81 seconds The longest time (1.681000) was taken by: BOL_NL Merging results from different sources and finding earliest publication dates from the worldcat.org service We have 0 merged results, merging took: 0.00 seconds

02-01-2018, 08:21 AM	#364
Wipneus Connoisseur Posts: 77 Karma: 12 Join Date: Jan 2012 Location: Nederland Device: Ipad Pro	Now i get in the comments this: Op het chique premièrefeest van seizoen 2 van L.A. Candy krioelt het van de fans, en de paparazzi staan te dringen langs de rode loper. De realityshow is een hit en Jane Roberts dé grote ster. Maar onder de oppervlakte smeult er van alles tussen de vier meisjes van de show. Jane weigert scènes te doen met Madison, nadat die compromitterende foto's van haar aan Gossip heeft doorgespeeld. Madison zelf wil koste van het kost achter de identiteit van haar afperser komen, die dreigt te onthullen dat zij niet altijd een rijke, platinablonde Californische babe was... Gaby gedraagt zich opeens totaal anders sinds ze een nieuwe persagent heeft. En Scarlett ten slotte moet zien te laveren tussen haar vriendje en de show, want die twee samen gaan echt niet. En dan is seizoen 2 nog maar net begonnen... (source: Bol.com 1) Why starts the comment with spaces and why stands there everytime source: Bol.com 1 Can you change that? And now i get no covers from Bol.com. (Dutch) Bedankt voor de aanpassing. Wat nu opvalt is dat bij commentaar vakje dat de samenvatting begint met enkele spaties en onderaan staat nu source: Bol.com 1 Waarvoor staat die 1? Krijg nu ook geen covers van Bol. ps. Zodra ik dit nu plaats zie je niet dat het begint met enkele spaties terwijl ik het wel zie bij aanpassing van het probleem Gr. Wipneus Last edited by Wipneus; 02-01-2018 at 08:23 AM.

02-14-2018, 01:31 PM	#374
Dompie Zealot Posts: 102 Karma: 10 Join Date: Jul 2010 Location: Gouda, The Netherlands Device: Kobo Aura & Kobo Libra	Yes every lookup without an ISBN gives a: HTTPError: HTTP Error 404: Not Found When you have filled the ISBN everything is all right Last edited by Dompie; 02-14-2018 at 01:32 PM. Reason: Typo

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
[Metadata Source Plugin] Libri.hu [Deprecated]	Daermond	Plugins	5	10-02-2012 05:07 AM
[Metadata Source Plugin] Moly.hu [Deprecated]	Daermond	Plugins	7	09-23-2012 03:48 AM
Request : metadata source plugin for bol.com	bolligske	Plugins	8	06-17-2011 07:44 AM
[Metadata Download Plugin] Goodreads Metadata Deprecated	kiwidude	Plugins	30	04-23-2011 02:10 PM
metadata plugin	redneck_momma	Plugins	1	05-21-2010 08:41 PM

01-30-2018, 06:07 PM	#361
bigpapaw Junior Member Posts: 4 Karma: 10 Join Date: Jan 2018 Device: none	Hello, I have trouble scraping metadata with bol_NL. I get no results from Bol_NL. Each time there is the nothing found error. Even for books i know there is an entrance in bol. As example: The book "Obsessie" from Tatiana de Rosnay. In calibre i get error message: calibre, version 3.16.0 ERROR: No matches found: <p>Failed to find any books that match your search. Try making the search <b>less specific</b>. For example, use only the author's last name and a single distinctive word from the title.<p>To see the full log, click "Show details". Running identify query with parameters: {u'identifiers': {u'google': u'o5m6DgAAQBAJ', u'isbn': u'9789026339301'}, u'authors': [u'Tatiana de Rosnay'], u'timeout': 30, u'title': u'Obsessie'} Using plugins: BOL_NL (3, 8, 17) The log from individual plugins is below **************************** BOL_NL (3, 8, 17) ************************** Found 0 results Downloading from BOL_NL took 0.996000051498 Querying: https://www.bol.com/nl/p/obsessie/9200000077515228/ BOL_NL url: 'https://www.bol.com/nl/p/obsessie/9200000077515228/' Error parsing title for url: 'https://www.bol.com/nl/p/obsessie/9200000077515228/' Traceback (most recent call last): File "calibre_plugins.BOL_NL.worker", line 125, in parse_details File "calibre_plugins.BOL_NL.worker", line 243, in parse_title AttributeError: 'NoneType' object has no attribute 'find' Could not find title for 'https://www.bol.com/nl/p/obsessie/9200000077515228/' Could not find title/authors/bol_nl_id for 'https://www.bol.com/nl/p/obsessie/9200000077515228/' bol_nl_id: 'nl/p/obsessie/9200000077515228' Title: None Authors: ['Tatiana de Rosnay'] ********************************************** **************************** The identify phase took 1.00 seconds The longest time (0.996000) was taken by: BOL_NL Merging results from different sources and finding earliest publication dates from the worldcat.org service We have 0 merged results, merging took: 0.00 seconds I only use now the bol plugin. In google crome i do get results using the line 'https://www.bol.com/nl/p/obsessie/9200000077515228/' Why do i get no results in calibre from bol plugin? please help I need the dutch source

01-31-2018, 03:53 PM	#363
Pr.BarnArt Addict Posts: 397 Karma: 401800 Join Date: Jun 2011 Device: Pocketbook 902 / Ipad air/ kindle paperwhite	New update available at the start of this thread. v 3.8.18 Made some changes due to website changes of Bol.nl and also a fix for downloading covers from literatuurplein Enjoy!

02-02-2018, 08:20 PM	#366
bigpapaw Junior Member Posts: 4 Karma: 10 Join Date: Jan 2018 Device: none	Thanks Pr BarnArt for your quick reaction solving my problem, but i found new/another annoying problem. When you scrape the found information of a book from bol.com_NL to the database it appends the new information instead of delete/replace the existing text. Hope you can look at it, thanks again

02-03-2018, 06:19 PM	#368
dooiemus Junior Member Posts: 2 Karma: 10 Join Date: Feb 2018 Device: kobo one	Hi Pr., Different from previous versions the "review" (recenties) data doesn't seem to come through anymore. I have tried several settings: disabling all metadata download plugins except bol-nl and then disable and enable "retrieval of review" option. Haven't seen any review data in all tests. Is this because BOL.NL has changed or is it something in the plugin?

02-06-2018, 02:34 PM	#370
Pr.BarnArt Addict Posts: 397 Karma: 401800 Join Date: Jun 2011 Device: Pocketbook 902 / Ipad air/ kindle paperwhite	The dead sparrow seems to be dead

02-09-2018, 07:40 PM	#371
dooiemus Junior Member Posts: 2 Karma: 10 Join Date: Feb 2018 Device: kobo one	Not dead yet just slooowwww...anyway Title : De Ondergang Author(s) : Joachim Fest Publisher : De Bezige Bij Languages : dut Rating : 2.5 Published : 2011-07-02T00:00:00+00:00 Identifiers : isbn:9789023466864 came with a review on jan 29th by Dr JLG v Oudheusden, today just the comment section and no review.

Advert

Advert