Kobo Website Wish List

[Quoth]

Is it only in order of books added?

I can't see how to sort by author or price. Maybe it's obvious and I'm just being stupid.

[John Hopper]

Pretty sure it can't be sorted any way. I wish it wasn't so big either .

Sent from my SM-F721B using Tapatalk

[isarl]

I have felt similarly stupid for the same failure to find any convenience methods for it, so this comment is a brazen stealth tag to see if anybody else has any clever ideas. I'm about ready to resort to writing a scraper for it that reuses my login cookies from my web browser.

[Quoth]

Export as csv file (with links), images not needed, title, author, price, blurb, URL of Kobo page, maybe URL of image?. Then any spreadsheet can sort, and you click on link of book to have it on Kobo web page

[Quoth]

Originally Posted by John Hopper

Pretty sure it can't be sorted any way. I wish it wasn't so big either.

No matter how many books I buy, the wish list at each site gets bigger

[isarl]

Originally Posted by Quoth

Export as csv file (with links), images not needed, title, author, price, blurb, URL of Kobo page, maybe URL of image?. Then any spreadsheet can sort, and you click on link of book to have it on Kobo web page

Pretty much exactly what I was thinking! Though I would amend “price” to “usual price” and “current price” to help highlight which items are currently on sale. The scraping library I use (ScraPy) also makes it super convenient to output in other formats too, e.g. JSON, but CSV as you say is probably more suited to spreadsheet software.

Oh dear. Today I found an item on my wishlist is simply no longer available at all – it's now only available as an audiobook? How unfortunate. But I suppose fortunate from the perspective of encountering that edge case before writing any wishlist scrapers, if I wind up doing that…

[Quoth]

Yes, that would be useful to know (item gone. Happens often on Amazon). I know what a cookie is, and there may be more than one; I didn't check, but I've only ever blocked them or deleted them. All 3rd party always blocked. So I've no idea how to use on in say wget (which can use them). I've used wget plenty.

[ownedbycats]

Originally Posted by isarl

Oh dear. Today I found an item on my wishlist is simply no longer available at all – it's now only available as an audiobook? How unfortunate. But I suppose fortunate from the perspective of encountering that edge case before writing any wishlist scrapers, if I wind up doing that…

Check whether it's now available on Kindle Unlimited. 90% of the time this has happened to books on my wishlist, it's from this.

[Quoth]

Originally Posted by ownedbycats

Check whether it's now available on Kindle Unlimited. 90% of the time this has happened to books on my wishlist, it's from this.

And you can buy stuff that's on KU, without having a subscription, then Download for USB transfer works.
But I've had plenty of ebooks, paper books and other stuff on my three wish lists on Amazon become unavailable too.

[ownedbycats]

Originally Posted by Quoth

And you can buy stuff that's on KU, without having a subscription, then Download for USB transfer works.

Only if you have a Kindle device.

[Quoth]

Originally Posted by ownedbycats

Only if you have a Kindle device.

A real eink one, and it needs to have Publisher option in Aa or Themes as this shows it can do azw3, not primitive old mobi.

[Cootey]

Originally Posted by Quoth

Is it only in order of books added?

I can't see how to sort by author or price. Maybe it's obvious and I'm just being stupid.

Isn’t it frustrating? I’d love to be able to sort by books on sale or by series. Even reverse sort of by order added would be helpful. No such luck.

[isarl]

I spent some time working out how to scrape my Kobo wishlist this evening. It's a bit fragile (I wasn't able to find a reliable and portable way to easily grab browser cookies, but did find a way that works for my setup) but I was able to figure out specifically which URL to query and how to parse the responses.

I will be curious to see in the coming days whether the cookies I currently have hard-baked into my script will expire and need updating. But in the meanwhile Scrapy is allowing me to generate a CSV with all the items on my wishlist.

This is not really in a shareable state yet. However, if you are reasonably comfortable with your browser's dev tools, and with tweaking code, then here's some ground I've already covered to help you replicate my results for your own wishlist. Depends on Scrapy.

#!/usr/bin/env python
import json
import scrapy

class KoboWishlistSpider(scrapy.Spider):
    name = 'kobowishlist'
    allowed_domains = ['kobo.com']
    magic_request_string = """replace the contents of this triple-quoted string as described below"""

    def parse(self, response):
        data = json.loads(response.body.decode())
        for item in data['Items']:
            yield {'authors': [a['Name'] for a in item['Authors']], 'title': item['Title'], 'price': item['Price'], 'page': data['PageIndex']}
        if data['PageIndex'] < data['TotalNumPages']:
            yield response.request.replace(body='{"pageIndex": ' + f"{data['PageIndex']+1}" + '}')

    def start_requests(self):
        yield scrapy.Request.from_curl(self.magic_request_string).replace(body='{"pageIndex": 1}')

Save this into a file like kobospider.py. In a browser logged into your Kobo account, visit your wishlist and open the developer tools. Click on the Network tab, start recording, and then click one of the buttons at the bottom of your wishlist to change pages. In the list of requests that get recorded in the Network page of the developer tools, look for the request to “fetch”. Right-click > Copy > Copy as cURL. (I believe this works in at least Chrome, Chromium, and Firefox.) Paste this value into the triple quotes for magic_request_string, replacing the placeholder value. You should now be able to run your spider from the commandline like:

scrapy runspider kobospider.py -O items.csv

I would be much happier if this were able to get your login cookies in a much more automatic way. StackOverflow suggested a Python library called browser-cookie3 which aims to do what I need, but when I tried it out in practice, it encountered issues with dbus.

[isarl]

Glad to receive a little positive feedback on this (outside this thread). Unfortunately (but not very surprisingly) after sleeping on it, the request parameters I had copied out of my browser yesterday evening have expired. Replacing with a fresh copy works, but this indicates that my session management could be greatly improved. Still easier than clicking through all the wishlist pages manually, but not as automatic as you'd really want. I'll have to spend some more time on this later. If anybody is a wizard with HTTP headers, sessions, requests, and so on, I would welcome any insight you can share.

Edit: worked on this some more today, but negative results only; haven't cracked it yet.

[Quoth]

Of course you can view your Kobo web site wish list on an Kobo ereader using the same log on, but you can't sort that either, unless it's cached somewhere on the device? Under Discover Tab on main GUI