Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre > Development

Notices

Reply
 
Thread Tools Search this Thread
Old 07-06-2020, 08:03 AM   #1
Boilerplate4U
Member
Boilerplate4U began at the beginning.
 
Boilerplate4U's Avatar
 
Posts: 14
Karma: 10
Join Date: Jul 2020
Device: Kobo Clara HD
[Metadata Source Plugin] Questions regarding parse select, docs and ref templates

Hi all!

Some questions regarding the metadata plugin:

1. Are there any architecture docs for metadata plugin that describes the overall functionality of worker.py in more detail?

2. Is there any similar built in functionally the resembles of the BeautifulSoup selector ("Soup.select") in the example below:

Code:
Html = requests.get(URL)
Soup = BeautifulSoup(Html)
Elements = Soup.select("#main > product-grid > li:nth-child(1) > article > a")[0]
Href  = Elements[0].get('href')
3. Is there a good reference template for the metadata plugin?
--

Any suggestions are welcome, thanks in advance!
Boilerplate4U is offline   Reply With Quote
Old 07-06-2020, 09:16 AM   #2
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 37,311
Karma: 16447605
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
1. worker.py is irrelevant, all you need to do is subclass https://manual.calibre-ebook.com/plu...a.sources.base and implement identify() and download_cover() methods.

2. beautifulSoup is available in calibre. from calibre.ebooks.beautifulsoup import BeautifulSoup

3. Look at any of th eexisting metadata download plugin
kovidgoyal is offline   Reply With Quote
Advert
Old 07-06-2020, 09:35 AM   #3
Boilerplate4U
Member
Boilerplate4U began at the beginning.
 
Boilerplate4U's Avatar
 
Posts: 14
Karma: 10
Join Date: Jul 2020
Device: Kobo Clara HD
Kovidgoyal, thanks for the prompt reply!

Regarding plugin template, I'd really appreciate if you could recommend some clean implementations since the plugins I've seen so far were a bit to much spaghetti :-)

Btw, are there any simple step by step guides regarding develop, test and deploy?
Boilerplate4U is offline   Reply With Quote
Old 07-06-2020, 10:06 AM   #4
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 37,311
Karma: 16447605
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
Look for the builtin plugins google and google images are both fairly short and simple. As for how to develop and test, use calibre-customize to develop, see calibre-customize --help and for testing again look at the builtin plugins, near the bottom.
kovidgoyal is offline   Reply With Quote
Old 07-06-2020, 10:26 AM   #5
Boilerplate4U
Member
Boilerplate4U began at the beginning.
 
Boilerplate4U's Avatar
 
Posts: 14
Karma: 10
Join Date: Jul 2020
Device: Kobo Clara HD
These?

https://github.com/kovidgoyal/calibr...ogle_images.py
https://github.com/kovidgoyal/calibr...rces/google.py
Boilerplate4U is offline   Reply With Quote
Advert
Old 07-06-2020, 10:30 AM   #6
Boilerplate4U
Member
Boilerplate4U began at the beginning.
 
Boilerplate4U's Avatar
 
Posts: 14
Karma: 10
Join Date: Jul 2020
Device: Kobo Clara HD
When are you supposed to use "class Worker (Thread)" (in worker.py) to implement a metadata plugin?

PS...
If possible, please move this thread to the correct forum: "E-Book Software > Calibre > Plugins"

Last edited by Boilerplate4U; 07-06-2020 at 10:34 AM.
Boilerplate4U is offline   Reply With Quote
Old 07-06-2020, 10:37 AM   #7
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 37,311
Karma: 16447605
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
Never. Again worker.py is irrelevant.
kovidgoyal is offline   Reply With Quote
Old 07-06-2020, 11:04 AM   #8
Boilerplate4U
Member
Boilerplate4U began at the beginning.
 
Boilerplate4U's Avatar
 
Posts: 14
Karma: 10
Join Date: Jul 2020
Device: Kobo Clara HD
Quote:
Originally Posted by kovidgoyal View Post
Never. Again worker.py is irrelevant.
Ok. I'm a bit curious as to why almost all metadata-source-plugins that I have looked at so far, utilize the following basic structure (some kind of old methodology or auto generated?)
Code:
__init__.py
changelog.txt
worker.py
config.py
Plug-import-name goodreads.txt
Boilerplate4U is offline   Reply With Quote
Old 07-06-2020, 11:20 AM   #9
Boilerplate4U
Member
Boilerplate4U began at the beginning.
 
Boilerplate4U's Avatar
 
Posts: 14
Karma: 10
Join Date: Jul 2020
Device: Kobo Clara HD
Just some examples: Mondourania, Goodreads, Skoob Books, Moly_hu, Adlibris, Barnes & Noble, KoboBooks ...
Boilerplate4U is offline   Reply With Quote
Old 07-06-2020, 12:13 PM   #10
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 37,311
Karma: 16447605
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
Probably because they copy each other. You just need to implement those two methods, thats all. What other files you use is entirely up to you.
kovidgoyal is offline   Reply With Quote
Old 07-06-2020, 01:19 PM   #11
Boilerplate4U
Member
Boilerplate4U began at the beginning.
 
Boilerplate4U's Avatar
 
Posts: 14
Karma: 10
Join Date: Jul 2020
Device: Kobo Clara HD
Aha, then that riddle was resolved! :-) Thanks, have a nice day!
Boilerplate4U is offline   Reply With Quote
Old 07-06-2020, 06:56 PM   #12
BetterRed
null operator
BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.
 
Posts: 16,327
Karma: 19147306
Join Date: Mar 2012
Location: Sydney Australia
Device: none
Quote:
Originally Posted by Boilerplate4U View Post
When are you supposed to use "class Worker (Thread)" (in worker.py) to implement a metadata plugin?

PS...
If possible, please move this thread to the correct forum: "E-Book Software > Calibre > Plugins"
Moderator Notice

Bp4U - IMO your thread is where it belongs, discussion of plugin development is best done here. If it had started in the Plugins sub-forum me, or one of the other mods, would have moved it here

Most people think BeautifulSoup is what you rush out to buy when your town gets locked down

BR
BetterRed is offline   Reply With Quote
Old 07-07-2020, 02:33 AM   #13
Boilerplate4U
Member
Boilerplate4U began at the beginning.
 
Boilerplate4U's Avatar
 
Posts: 14
Karma: 10
Join Date: Jul 2020
Device: Kobo Clara HD
Quote:
Originally Posted by BetterRed View Post
Most people think BeautifulSoup is what you rush out to buy when your town gets locked down

BR
Well, I use BeautifulSoup every day and I'm pretty good looking as of today.
Boilerplate4U is offline   Reply With Quote
Old 07-07-2020, 02:35 AM   #14
Boilerplate4U
Member
Boilerplate4U began at the beginning.
 
Boilerplate4U's Avatar
 
Posts: 14
Karma: 10
Join Date: Jul 2020
Device: Kobo Clara HD
FYI, found "base.py" that works a well documented template.

(calibre/src/calibre/ebooks/metadata/sources/base.py)
Boilerplate4U is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
[Metadata Source Plugin] BOL.com (NL) Pr.BarnArt Plugins 436 07-19-2020 07:58 AM
[Metadata Source Plugin] MondoUrania Pr.BarnArt Plugins 30 06-10-2020 02:21 PM
[Metadata Source Plugin] Empty Plugin? (Fake Identifier) mneimeyer Plugins 3 11-11-2019 08:07 PM
[Metadata Source Plugin] Pitaval.cz (cz,sk) Miisha Plugins 1 10-09-2019 06:17 AM
[Metadata Source Plugin] FictionDB kiwidude Plugins 75 05-05-2019 10:39 PM


All times are GMT -4. The time now is 09:48 AM.


MobileRead.com is a privately owned, operated and funded community.