[GUI Plugin] Wiki Reader - Page 6

kovidgoyal · 06-18-2016, 08:35 AM

Done, though I have given it only the most rudimentary testing.

mcdummy · 06-21-2016, 06:13 AM

Excellent job. Thanks a lot.

Almost all incomplete references are corrected, only the links in the header section (<head>...</head>) and in the area-tags (<area href="...>) are still incomplete.

Thanks,

McDummy

Mixx · 06-25-2016, 05:16 PM

Quote:

Originally Posted by OrganicJerk

Also, another request, do you think it would be possible to have "parts" of the articles excluded? Being able to exclude things like Images, "See Also," "References," "External Links," and so on, would definitely be useful to many, let alone myself at the moment.

Again, thanks for the plugin!

Hello,

could I come back to this (now old) request? I am annoyed by the long list of "References" that I never need if I am reading on an e-reader. Could anyone please include an "exclude" (hah!) capability?

That would be very helpful.

Thanxx, Mixx

mcdummy · 07-05-2016, 06:38 AM

I hope it is not too complicated, but could the plugin be extended to WikiLivres?

http://wikilivres.ca

Regards,

McDummy

mcdummy · 08-12-2016, 07:08 AM

Concerning the correction of incomplete (relative) references, it seems that the current version only corrects links in <a>-tags but not in <area> or <link>-tags.

The links in these tags could be corrected by adding

# <area> references
for area in soup.findAll('area', href=lambda x: x and x.startswith('/')):
href = area['href']
if href.startswith('//'):
area['href'] = 'https:' + href
elif url_prefix:
area['href'] = url_prefix + area['href']
# <link> references
for link in soup.findAll('link', href=lambda x: x and x.startswith('/')):
href = link['href']
if href.startswith('//'):
link['href'] = 'https:' + href
elif url_prefix:
link['href'] = url_prefix + link['href']

in wikipedia.recipe

after the code

for a in soup.findAll('a', href=lambda x: x and x.startswith('/')):
href = a['href']
if href.startswith('//'):
a['href'] = 'https:' + href
elif url_prefix:
a['href'] = url_prefix + a['href']

I guess the code could be simplified, but it is just a quick workaround for the time being.

McDummy

compukol · 12-19-2016, 03:06 PM

Hi I would like to use the Wiki Reader concept with a list of blog articles URL's. When I feed the URL's to Wiki Reader the result eBook contain only the links and not the content of the articles.
I am attaching the test output from trying to import few blog links.

Basically I am looking to use Calibre as a replacement to readlist.com from Readability that was shut down.

Any help would be greatly appreciated.

kovidgoyal · 12-19-2016, 11:03 PM

The Wiki Reader plugin uses the calibre recipe system (it has a recipe designed for extracting content from wikipedia). You cannot use it directly with arbitrary HTML pages. See https://manual.calibre-ebook.com/news.html for how the recipe system in calibre works.

mcdummy · 11-06-2017, 02:29 AM

Hi,

when I try to create an ebook from

https://de.wikipedia.org/wiki/Hexe

Wiki Reader creates an empty epub. If I add this Wiki page to a collection of pages, this page is not included in the generated epub.

Is there a Problem in the plugin or in the page?

Thanks,

McDummy

mcdummy · 02-15-2018, 09:39 AM

Hi,

I've noticed that some writers on wiki articles use improper (not allowed) characters (such as, e.g., :,;()äöüß) in IDs and the corresponding references (href="..).

In the E-Books created by the Wiki Reader plugin, these characters are replaced in either the references or in the IDs (cannot remember which one) but not in both, which results in broken links.

Is there a simple way for replacing these characters in both the IDs and the reference? For instance in a pre-processing stage before calibre converts a Wiki-page to the final E-Book?

Regards,

McDummy

mcdummy · 02-15-2018, 09:51 AM

Hi again,

When the Wiki Reader creates an E-Book, it places a (feed-)navigation bar at the beginning and the end of each Wiki article.

These navigation bars contain dead links and are not necessary.

Is there a simple way to prevent the plugin from creating these navigation bars?

Regards,

McDummy

covo · 03-27-2020, 06:24 PM

hi,
is there a version of Wiki Reader for Calibre 2.0?

bonzi · 06-06-2020, 12:19 PM

This is an incredibly useful, practically flawlessly working plug-in, especially since Wikipedia has discontinued its e-book building tool, leaving only the practically useless "Print to PDF" feature. I have tried several other "Wikipedia bookifying" tools, and none comes close.

I have a pretty big feature request, though (two related ones, actually):

The easier one: when several Wikipedia pages are combined into a book, post-process the book so that Wikipedia hyperlinks leading to other articles in the same book are converted to intra-book hyperlinks.
The more difficult ones: after processing the requested pages, present a menu of other Wikipedia pages referenced from the book. Allow the user to select those that should be included, too. Process them, then repeat the process until done.
Of course, do the hyperlink translation as per the first request, too.
I know this I much more involved, requiring essentially iterating the whole processing the plugin does over and over, inserting a bit of UI dialog after every iteration, potentially rearranging already generated content etc. But it would be immensely cool.

I see your plugin does not have a contribution link. Well, I guess I will send a buck or two to Calibre itself, then

Thank you for this great tool!

bonzi · 06-06-2020, 12:27 PM

Quote:

Originally Posted by covo

hi,
is there a version of Wiki Reader for Calibre 2.0?

I hate it when people offer me advice instead of actual answers to my question, but anyway:

Is there a good reason not to upgrade Calibre? I have been using it from something like 0.75, IIRC, upgrading fairly regularly (I am now on the most recent, 4.18), and never had any problems. Some invaluable plugin not compatible with a more recent version? Something else?

kovidgoyal · 06-06-2020, 12:45 PM

@bonzi: IIRC Wiki Reader already resolves internal links that point to articles downloaded as part of the book (it uses the recipe systems resolve_internal_links). If it isnt happening for you post a couple of articles that reproduce the issue.

As for allowing inclusion of extra linked articles, post download, that is too much work, at least for me.

bonzi · 06-07-2020, 07:38 PM

Quote:

Originally Posted by kovidgoyal

@bonzi: IIRC Wiki Reader already resolves internal links that point to articles downloaded as part of the book (it uses the recipe systems resolve_internal_links). If it isnt happening for you post a couple of articles that reproduce the issue.

Of course, you are right. I tried to reproduce what I was certain I saw when I posted this, and, sure enough, the plugin resolves links perfectly.

Perhaps I tapped on a link I only thought leads to an included article...

My apologies, Kovid! I know how developers "love" sloppy bug reports and feature requests, especially as busy ones like yourself.

Quote:

As for allowing inclusion of extra linked articles, post download, that is too much work, at least for me.

Of course. And utility would be limited, because most articles of the lengths deserving "bookification" would serve hundreds of links.

Thank you again for this great tool!

07-05-2016, 06:38 AM	#79
mcdummy Connoisseur Posts: 56 Karma: 7130 Join Date: Apr 2015 Device: PRS-T3	WikiLivres I hope it is not too complicated, but could the plugin be extended to WikiLivres? http://wikilivres.ca Regards, McDummy

11-06-2017, 02:29 AM	#83
mcdummy Connoisseur Posts: 56 Karma: 7130 Join Date: Apr 2015 Device: PRS-T3	Wiki Reader does not download wiki-page Hi, when I try to create an ebook from https://de.wikipedia.org/wiki/Hexe Wiki Reader creates an empty epub. If I add this Wiki page to a collection of pages, this page is not included in the generated epub. Is there a Problem in the plugin or in the page? Thanks, McDummy

02-15-2018, 09:39 AM	#84
mcdummy Connoisseur Posts: 56 Karma: 7130 Join Date: Apr 2015 Device: PRS-T3	Removing improper characters from ids and references Hi, I've noticed that some writers on wiki articles use improper (not allowed) characters (such as, e.g., :,;()äöüß) in IDs and the corresponding references (href="..). In the E-Books created by the Wiki Reader plugin, these characters are replaced in either the references or in the IDs (cannot remember which one) but not in both, which results in broken links. Is there a simple way for replacing these characters in both the IDs and the reference? For instance in a pre-processing stage before calibre converts a Wiki-page to the final E-Book? Regards, McDummy

02-15-2018, 09:51 AM	#85
mcdummy Connoisseur Posts: 56 Karma: 7130 Join Date: Apr 2015 Device: PRS-T3	Creating E-Books without Feed-navigation bars Hi again, When the Wiki Reader creates an E-Book, it places a (feed-)navigation bar at the beginning and the end of each Wiki article. These navigation bars contain dead links and are not necessary. Is there a simple way to prevent the plugin from creating these navigation bars? Regards, McDummy

03-27-2020, 06:24 PM	#86
covo Junior Member Posts: 5 Karma: 10 Join Date: Feb 2014 Device: kindle5 classic	Wiki Reader for Calibre 2.0 hi, is there a version of Wiki Reader for Calibre 2.0?

06-18-2016, 08:35 AM	#76
kovidgoyal creator of calibre Posts: 43,835 Karma: 22666666 Join Date: Oct 2006 Location: Mumbai, India Device: Various	Done, though I have given it only the most rudimentary testing.

06-21-2016, 06:13 AM	#77
mcdummy Connoisseur Posts: 56 Karma: 7130 Join Date: Apr 2015 Device: PRS-T3	Excellent job. Thanks a lot. Almost all incomplete references are corrected, only the links in the header section (<head>...</head>) and in the area-tags (<area href="...>) are still incomplete. Thanks, McDummy

08-12-2016, 07:08 AM	#80
mcdummy Connoisseur Posts: 56 Karma: 7130 Join Date: Apr 2015 Device: PRS-T3	Concerning the correction of incomplete (relative) references, it seems that the current version only corrects links in <a>-tags but not in <area> or <link>-tags. The links in these tags could be corrected by adding # <area> references for area in soup.findAll('area', href=lambda x: x and x.startswith('/')): href = area['href'] if href.startswith('//'): area['href'] = 'https:' + href elif url_prefix: area['href'] = url_prefix + area['href'] # <link> references for link in soup.findAll('link', href=lambda x: x and x.startswith('/')): href = link['href'] if href.startswith('//'): link['href'] = 'https:' + href elif url_prefix: link['href'] = url_prefix + link['href'] in wikipedia.recipe after the code for a in soup.findAll('a', href=lambda x: x and x.startswith('/')): href = a['href'] if href.startswith('//'): a['href'] = 'https:' + href elif url_prefix: a['href'] = url_prefix + a['href'] I guess the code could be simplified, but it is just a quick workaround for the time being. McDummy

12-19-2016, 11:03 PM	#82
kovidgoyal creator of calibre Posts: 43,835 Karma: 22666666 Join Date: Oct 2006 Location: Mumbai, India Device: Various	The Wiki Reader plugin uses the calibre recipe system (it has a recipe designed for extracting content from wikipedia). You cannot use it directly with arbitrary HTML pages. See https://manual.calibre-ebook.com/news.html for how the recipe system in calibre works.

06-06-2020, 12:19 PM	#87
bonzi Enthusiast Posts: 46 Karma: 1530 Join Date: Nov 2011 Location: Zagreb, Croatia Device: K2i, K3/KK WiFi, Basic (old), 2xFire, 2xPW (old), Voyage, Scribe	This is an incredibly useful, practically flawlessly working plug-in, especially since Wikipedia has discontinued its e-book building tool, leaving only the practically useless "Print to PDF" feature. I have tried several other "Wikipedia bookifying" tools, and none comes close. I have a pretty big feature request, though (two related ones, actually): The easier one: when several Wikipedia pages are combined into a book, post-process the book so that Wikipedia hyperlinks leading to other articles in the same book are converted to intra-book hyperlinks. The more difficult ones: after processing the requested pages, present a menu of other Wikipedia pages referenced from the book. Allow the user to select those that should be included, too. Process them, then repeat the process until done. Of course, do the hyperlink translation as per the first request, too. I know this I much more involved, requiring essentially iterating the whole processing the plugin does over and over, inserting a bit of UI dialog after every iteration, potentially rearranging already generated content etc. But it would be immensely cool. I see your plugin does not have a contribution link. Well, I guess I will send a buck or two to Calibre itself, then Thank you for this great tool! Last edited by bonzi; 06-06-2020 at 12:22 PM.

06-06-2020, 12:45 PM	#89
kovidgoyal creator of calibre Posts: 43,835 Karma: 22666666 Join Date: Oct 2006 Location: Mumbai, India Device: Various	@bonzi: IIRC Wiki Reader already resolves internal links that point to articles downloaded as part of the book (it uses the recipe systems resolve_internal_links). If it isnt happening for you post a couple of articles that reproduce the issue. As for allowing inclusion of extra linked articles, post download, that is too much work, at least for me.

Thread Tools	Search this Thread
Show Printable Version Email this Page	Search this Thread: Advanced Search

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
[GUI Plugin] EpubSplit	JimmXinu	Plugins	422	04-07-2024 04:10 PM
[GUI Plugin] Open With	kiwidude	Plugins	403	04-01-2024 08:39 AM
[GUI Plugin] KindleUnpack - The Plugin	DiapDealer	Plugins	492	10-25-2022 08:13 AM
[GUI Plugin] Manage Sony x50 Reader Book List	kpw	Plugins	170	10-02-2014 08:23 PM
[GUI Plugin] Plugin Updater Deprecated	kiwidude	Plugins	159	06-19-2011 12:27 PM