[Metadata Source Plugin] Kobo Metadata

[NotSimone]

I started getting this issue too - it looks like Kobo recently enabled some kind of Cloudflare bot protection. It might not be possible to get past it but I'll take a look

[ownedbycats]

There's been similar CloudFlare issues with FanFicFare in the past few years, so it might be worth asking JimmXinu.

[JSWolf]

I am not getting any covers from Kobo Metadata.

[DNSB]

Originally Posted by JSWolf

I am not getting any covers from Kobo Metadata.

If you try a download cover for a ebook and then check the log file, you are likely to find 403 error.

Traceback (most recent call last):
  File "calibre_plugins.kobobooks.__init__", line 222, in identify
  File "mechanize\_mechanize.py", line 241, in open_novisit
  File "mechanize\_mechanize.py", line 313, in _mech_open
mechanize._response.get_seek_wrapper_class..httperror_seek_wrapper: HTTP Error 403: Forbidden

No cover found

[JSWolf]

Originally Posted by DNSB

If you try a download cover for a ebook and then check the log file, you are likely to find 403 error.

Traceback (most recent call last):
  File "calibre_plugins.kobobooks.__init__", line 222, in identify
  File "mechanize\_mechanize.py", line 241, in open_novisit
  File "mechanize\_mechanize.py", line 313, in _mech_open
mechanize._response.get_seek_wrapper_class..httperror_seek_wrapper: HTTP Error 403: Forbidden

No cover found

Is there a way around the 403 error?

[NotSimone]

From what I can tell the best way around this is to use something like Flaresolverr, which tbh seems quite involved in setting up

[JSWolf]

Originally Posted by NotSimone

From what I can tell the best way around this is to use something like Flaresolverr, which tbh seems quite involved in setting up

Thanks, but that won't work at all. I need a Windows based solution.

[Quoth]

Originally Posted by JSWolf

Thanks, but that won't work at all. I need a Windows based solution.

This is important!

It is recommended to install using a Docker container because the project depends on an external browser that is already included within the image.

Docker for Windows:
https://docs.docker.com/desktop/inst...ndows-install/

So actually it's not simpler to run on Linux than Windows. I don't use Docker for anything.


EDIT
Also on Flaresolverr git page, further down:

This is the recommended way for Windows users.

Download the FlareSolverr executable from the release's page. It is available for Windows x64 and Linux x64.
Execute FlareSolverr binary. In the environment variables section you can find how to change the configuration.

Edit 2
This might be a bad idea anyway for metadata downloads

NOTE: Web browsers consume a lot of memory. If you are running FlareSolverr on a machine with few RAM, do not make many requests at once. With each request a new browser is launched.

It is also possible to use a permanent session. However, if you use sessions, you should make sure to close them as soon as you are done using them.

[NotSimone]

Yeah tbh I'm not a huge fan of this solution but I'm not sure there's much else that can a be done about it

[JSWolf]

Originally Posted by NotSimone

Yeah tbh I'm not a huge fan of this solution but I'm not sure there's much else that can a be done about it

I found a possible solution. Yu could try to incorporate Cloudscraper into the Kobo Metadata.

https://pypi.org/project/cloudscraper

[NotSimone]

I tried it already and no luck

[JSWolf]

Originally Posted by NotSimone

I tried it already and no luck

How about making it a Firefox plugin?

[NotSimone]

I've played around with the api that the Kobo devices use to access the store and I think its a possibility. From what I can tell, it makes some kind of request to get a bearer token, and then that token can be used to start querying without cloudflare stopping us. Problem is that it requires a Kobo device (a unique device id is needed) and some user data to be dumped first from the .sqlite tables.

I'm not a massive fan of doing it this way either tbh.

Originally Posted by JSWolf

How about making it a Firefox plugin?

What do you mean by that?

[JSWolf]

Originally Posted by NotSimone

I've played around with the api that the Kobo devices use to access the store and I think its a possibility. From what I can tell, it makes some kind of request to get a bearer token, and then that token can be used to start querying without cloudflare stopping us. Problem is that it requires a Kobo device (a unique device id is needed) and some user data to be dumped first from the .sqlite tables.

I'm not a massive fan of doing it this way either tbh.

We don't need a Kobo Reader to get past Cloudflare. I can do this with Firefox and Microsoft Edge. So is there a way to get the cookie from the browser to use with the plugin so the plugin can be modified to use it to allow us in?

What do you mean by that?

Given that Firefox has no problem talking to kobo.com, a plugin to get the cover would work well. Yes we would have to type in the book title for the plugin to find the cover and yes we would have to copy the cover, but at least we'd not be blocked.

[Synopis]

Weird. I'm getting 403's and actual results. For example, I can't find anything via kobo plugin for Werewolf Skin but I find something for Caves of Steel