[Metadata Source Plugin] StoryGraph

[ARR8]

Scrapes metadata and covers from https://thestorygraph.com/. Metadata include:

• title
• authors
• tags (genre, mood, and pace)
• publication information
• rich-text description
• rating
• series
• ISBN
• language

It also adds a StoryGraph GUID identifier and uses it to conduct searches.

Of note to me are the tags - this website is a great resource for crowdsourced mood and pace information (in fact, the only one I've found).

There is other potential information that could be fetched, such as page counts and content warnings, but I've limited this implementation to only the major information the StoryGraph itself puts in the book info header. If there is demand for the other info, though, let me know and I'll see about scraping and adding it.

Note that edition publication information from the StoryGraph is not the most reliable; to that end, I've provided a configuration option to use the first release year of the book, instead. This is only the year, though.

[rantanplan]

This plugin probably violates term 2 of their ToS:
https://app.thestorygraph.com/terms-of-service

2. Use License
1. Permission is granted to temporarily download one copy of the materials (information or software) on The StoryGraph's website for personal, non-commercial transitory viewing only. This is the grant of a license, not a transfer of title, and under this license you may not:

1. modify or copy the materials;

2. use the materials for any commercial purpose, or for any public display (commercial or non-commercial);

3. attempt to decompile or reverse engineer any software contained on The StoryGraph's website;

4. remove any copyright or other proprietary notations from the materials; or

5. transfer the materials to another person or "mirror" the materials on any other server.

[ARR8]

That's a fair point. By my read, use in one's personal Calibre library wouldn't fall under public display, but I'll ask them. I'll update with their response once I receive it.

[ARR8]

They state that they provide no official comment besides the terms themselves, but hope to formulate a clearer policy at some point in the future.

[dewd]

I'm interested in this plugin but it's failing to find any books. I've pasted the relevant bits of the error log below:

Plugin StoryGraph failed
Traceback (most recent call last):
File "lxml/cssselect.py", line 12, in <module>
ModuleNotFoundError: No module named 'cssselect'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "calibre/ebooks/metadata/sources/identify.py", line 47, in run
File "calibre_plugins.dummy1.__init__", line 342, in identify
books = root.cssselect(".book-pane")
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "src/lxml/etree.pyx", line 1613, in lxml.etree._Element.cssselect
File "bypy-importer.py", line 279, in exec_module
File "lxml/cssselect.py", line 14, in <module>
ImportError: cssselect does not seem to be installed. See https://pypi.org/project/cssselect/

[ARR8]

Hmm, that module is definitely included in my version of Calibre. What version are you using? And on what OS?

[JSWolf]

Originally Posted by ARR8

Hmm, that module is definitely included in my version of Calibre. What version are you using? And on what OS?

What you need to do is put in the first pot of this thresd what the minimum version of calibre your plugin will run.

[thiago.eec]

Originally Posted by ARR8

Hmm, that module is definitely included in my version of Calibre. What version are you using? And on what OS?

You are probably on linux, right? Maybe using a distro version of calibre? Those versions can differ from official calibre and have other libs added. I have the latest calibre (8.9) installed on my Windows 11 machine and the module is not included either.

[kovidgoyal]

If you want to use CSS based selectors with lxml trees in calibre use css_selectors, like this

from css_selectors import Select
select = Select(root)

for a in select('your css selector'):
   ...

[ARR8]

Thanks both. Uploading an updated version to use the internal library.

[ARR8]

Frustratingly, I apparently cannot edit my top post because of restrictions on new accounts. I've sent a message to whoever runs this website but I guess I have to wait in the meantime. It's also silently dropping my new posts.

I don't suppose there are any other distribution channels for plugins using the built-in installer?

[ARR8]

Apparently my account is too new to edit posts.

[ARR8]

And what I thought was messages being silently dropped is actually them being held back and posted in batches. Sorry to all subscribed.

[BetterRed]

Originally Posted by ARR8

Frustratingly, I apparently cannot edit my top post because of restrictions on new accounts. I've sent a message to whoever runs this website but I guess I have to wait in the meantime. It's also silently dropping my new posts.

I don't suppose there are any other distribution channels for plugins using the built-in installer?

Moderator Notice
To edit your own posts you need to get your post count up to 10 or more, post some replies in some of the threads in the Lounge.

Your string of duplicate posts went into moderation, I deleted them except the one I've quoted.

And this is the only place the calibre plugin updater knows about, and it only knows about plugins that are listed in the Index of plugins.

BR

[ARR8]

Understood, thanks for the explanation.