[GUI Plugin] FanFicFare

[firefox26]

Hi,

Trying to download stuff from the mcstories website and got the following error (URL used was https://mcstories.com/MarcusPower/index.html)

calibre, version 4.21.0
ERROR: Unhandled exception: <b>InvalidStoryURL</b>:Bad Story URL: (https://mcstories.com/StoryTitle) for site: (mcstories.com) Example: (https://mcstories.com/StoryTitle https://mcstories.com/StoryTitle/index.html https://mcstories.com/StoryTitle/StoryTitle1.html)

calibre 4.21 embedded-python: True is64bit: True
Darwin-19.6.0-x86_64-i386-64bit Darwin ('64bit', '')
('Darwin', '19.6.0', 'Darwin Kernel Version 19.6.0: Sun Jul 5 00:43:10 PDT 2020; root:xnu-6153.141.1~9/RELEASE_X86_64')
Python 2.7.16
OSX: ('10.15.6', ('', '', ''), 'x86_64')
Interface language: None
Successfully initialized third party plugins: FanFicFare (3, 21, 0)
Traceback (most recent call last):
File "calibre_plugins.fanficfare_plugin.fff_plugin" , line 567, in get_urls_from_page_menu
File "calibre_plugins.fanficfare_plugin.fff_plugin" , line 586, in get_urls_from_page
File "calibre_plugins.fanficfare_plugin.fanficfare.getu rls", line 49, in get_urls_from_page
File "calibre_plugins.fanficfare_plugin.fanficfare.adap ters.__init__", line 244, in getAdapter
File "calibre_plugins.fanficfare_plugin.fanficfare.adap ters.adapter_mcstoriescom", line 38, in __init__
File "calibre_plugins.fanficfare_plugin.fanficfare.adap ters.base_adapter", line 113, in __init__
InvalidStoryURL: Bad Story URL: (https://mcstories.com/StoryTitle) for site: (mcstories.com) Example: (https://mcstories.com/StoryTitle https://mcstories.com/StoryTitle/index.html https://mcstories.com/StoryTitle/StoryTitle1.html)

[JimmXinu]

Originally Posted by oenwolf

When I search in the Viewer in flow mode, the result navigates to the top of the chapter containing the word, but not the precise location of the word.

If you want to PM me a story URL (and word) that demonstrates the problem, I'll look at it. (This site doesn't like NSFW links in public posts.)

[JimmXinu]

New Test Version Posted

2020-07-26
- Fix adapter_mcstoriescom getSiteExampleURLs()

Originally Posted by firefox26

Trying to download stuff from the mcstories website and got the following error

File "calibre_plugins.fanficfare_plugin.fff_plugin" , line 567, in get_urls_from_page_menu

There was a minor bug there. But I'm not sure you're doing it right.

You're trying to do 'Get Story URLs from web page' which was broken for mcstories. (It involves using the first example URL to get INI config for 'from web page'.)

But if you use that URL as a regular story URL it works fine.

You should also note that mobileread.com doesn't like NSFW links in public posts. I suggest you edit your prior post to remove them and PM NSFW links in future.

[firefox26]

Issues downloading material from mcstories
Hi,

Trying to download stuff from the mcstories website and got the following error (URL used was https:xxxx)

calibre, version 4.21.0
ERROR: Unhandled exception: <b>InvalidStoryURL</b>:Bad Story URL: (https://mcstories.com/StoryTitle) for site: (mcstories.com) Example: (https://mcstories.com/StoryTitle https://mcstories.com/StoryTitle/index.html https://mcstories.com/StoryTitle/StoryTitle1.html)

calibre 4.21 embedded-python: True is64bit: True
Darwin-19.6.0-x86_64-i386-64bit Darwin ('64bit', '')
('Darwin', '19.6.0', 'Darwin Kernel Version 19.6.0: Sun Jul 5 00:43:10 PDT 2020; root:xnu-6153.141.1~9/RELEASE_X86_64')
Python 2.7.16
OSX: ('10.15.6', ('', '', ''), 'x86_64')
Interface language: None
Successfully initialized third party plugins: FanFicFare (3, 21, 0)
Traceback (most recent call last):
File "calibre_plugins.fanficfare_plugin.fff_plugin" , line 567, in get_urls_from_page_menu
File "calibre_plugins.fanficfare_plugin.fff_plugin" , line 586, in get_urls_from_page
File "calibre_plugins.fanficfare_plugin.fanficfare. getu rls", line 49, in get_urls_from_page
File "calibre_plugins.fanficfare_plugin.fanficfare. adap ters.__init__", line 244, in getAdapter
File "calibre_plugins.fanficfare_plugin.fanficfare. adap ters.adapter_mcstoriescom", line 38, in __init__
File "calibre_plugins.fanficfare_plugin.fanficfare. adap ters.base_adapter", line 113, in __init__
InvalidStoryURL: Bad Story URL: (https://mcstories.com/StoryTitle) for site: (mcstories.com) Example: (https://mcstories.com/StoryTitle https://mcstories.com/StoryTitle/index.html https://mcstories.com/StoryTitle/StoryTitle1.html)

[3amWednesday]

Hello, is there a way to override the default Calibre Columns?

Basically I want FFF to update the Date column to what the dateUpdated is and not today.

Also, is there a masterlist of all the variables available for metadata replacings/removings/etc?

Thanks in advance

[JimmXinu]

Originally Posted by 3amWednesday

Hello, is there a way to override the default Calibre Columns?

Basically I want FFF to update the Date column to what the dateUpdated is and not today.

Also, is there a masterlist of all the variables available for metadata replacings/removings/etc?

Thanks in advance

Customization of standard columns is largely limited to whether they are filled only on new books or on every update.

I recommend (and personally use) a custom date column to hold dateUpdated.

What I call the standard or common metadata entries are listed by their label values toward the beginning of defaults.ini; which you can see from the personal.ini tab. Individual sites may have additional site-specific metadata entries defined father down defaults.ini in their own site sections.

[JimmXinu]

New Test Version Posted

2020-08-01
- Accept storyUrl with title in it for adapter_webnovelcom. Addresses #520

[ownedbycats]

Question: AO3 lets authors link to images in the summaries (example here). FFF breaks this by changing the links to ffdl-#.jpg. Since the images usually get set as a cover anyways I thought I might as well use replace_metadata to remove them.

This regex tested okay but it didn't work:

replace_metadata: 
## Removes images from summaries
 description=><img src=\"images/ffdl-\d.jpg\">=>

Any idea what went wrong? I tried it in both replace_metadata and add_to. I also tried changing it to ffdl-0.jpg specifically but that didn't work either. (I also made sure it was a jpg file.)

[JimmXinu]

Originally Posted by ownedbycats

Question: AO3 lets authors link to images in the summaries (example here). FFF breaks this by changing the links to ffdl-#.jpg. Since the images usually get set as a cover anyways I thought I might as well use replace_metadata to remove them. ...

 description=><img[^>]+>=>

[ownedbycats]

That worked. Thank you.

[3amWednesday]

Originally Posted by JimmXinu

Customization of standard columns is largely limited to whether they are filled only on new books or on every update.

I recommend (and personally use) a custom date column to hold dateUpdated.

What I call the standard or common metadata entries are listed by their label values toward the beginning of defaults.ini; which you can see from the personal.ini tab. Individual sites may have additional site-specific metadata entries defined father down defaults.ini in their own site sections.

Thanks! That really helps!

[Electricboa]

I have a couple specific questions about customizing the plugin. I recently updated after a long time (I’m horrible at doing it in a timely manner). And the output of the downloads have changed and I’m trying to get them to be like they were before. I’m not really versed in this, so I could just be missing what I need to do. I have two issues:

1) I use CoolReader on my tablet to open my epub files. After the update, new files now have a white background default. I want there to be no background / transparent so it will be whatever the background color of the app is. I went into the peronal.ini and tried to change the background color, but it hasn’t fixed it. Online, I found that adding 00 is supposed to make the hexadecimal code transparent. I've tried several variations and removing it entirely, but it doesn’t work:

## Allow customization of CSS. Make sure to keep at least one space
## at the start of each line and to escape % to %%. Also need
## background_color to be in the same section, if included in CSS.
## 'adobe-text-layout: optimizeSpeed;' prevents hyphenation on newer Nooks
## Only ini substs, like background_color are done.
##output_css: 
## body { background-color: #00000000;

2) Now, there is a table of contents entry (not a TOC page) for the Title Page. How do I remove that, while still keeping the actually chapter entries intact? When I merge different stories together, this makes it look like stories with only one chapter have multiple chapters because the Title Page is counted. I can remove them manually in Calibre, but that’s incredibly time-consuming for a large number of stories. My older version of the plugin didn't do that.

And I guess I have a request for a feature. Sites like Literotica sometimes have multi-part series, which are essentially chapters. When you put in one part, the plugin automatically grabs all the rest. If I try to download an anthology of everything from an author, the plugin counts each part/chapter and ends up creating duplicate stories for each chapter. So if there are 10 chapters, I end up with 10 duplicate stories with 10 chapters each. Would it be possible to have the plugin ignore the duplicates?

As it is, I end up having to ‘Get Story URL From Web Page,’ manually go through and pick out the extra parts/chapters, then put them back in and merge them into one file.

[JimmXinu]

Originally Posted by Electricboa

1) I use CoolReader on my tablet to open my epub files. After the update, new files now have a white background default. I want there to be no background / transparent so it will be whatever the background color of the app is. ...

I would suggest copying the output_css setting from under [epub] in defaults.ini into your personal.ini under [epub] and remove the background-color CSS setting.

I don't use CoolReader, so I can't say for sure what will work; you'll have to experiment and see. I would recommend, while experimenting, to edit an epub to make sure you're getting what you want and then worry about getting it correctly into FFF's personal.ini settings.

Originally Posted by Electricboa

2) Now, there is a table of contents entry (not a TOC page) for the Title Page. How do I remove that, while still keeping the actually chapter entries intact? When I merge different stories together, this makes it look like stories with only one chapter have multiple chapters because the Title Page is counted. I can remove them manually in Calibre, but that’s incredibly time-consuming for a large number of stories. My older version of the plugin didn't do that.

If you use EpubMerge either directly or using FFF's 'Make Anthology' feature, the title page TOC entry is removed automatically for one-chapter stories.

Or, at least, it's supposed to. I found a couple stories that didn't work on while spot testing to make sure I remembered correctly.
I'll have to get back to you about that.

Originally Posted by Electricboa

And I guess I have a request for a feature. Sites like Literotica sometimes have multi-part series, which are essentially chapters. When you put in one part, the plugin automatically grabs all the rest. If I try to download an anthology of everything from an author, the plugin counts each part/chapter and ends up creating duplicate stories for each chapter. So if there are 10 chapters, I end up with 10 duplicate stories with 10 chapters each. Would it be possible to have the plugin ignore the duplicates?

As it is, I end up having to ‘Get Story URL From Web Page,’ manually go through and pick out the extra parts/chapters, then put them back in and merge them into one file.

Literotica story URLs are a mess. There is no 'story URL' as FFF would normally define it and no way to normalize story URL before fetching metadata pages, by which time it's too late.

I don't think there's a reasonable solution for that.

[JimmXinu]

Originally Posted by JimmXinu

Or, at least, it's supposed to. I found a couple stories that didn't work on while spot testing to make sure I remembered correctly.
I'll have to get back to you about that.

A change made to epub output this past winter/spring broke EpubMerge's one-chapter story detection.

Please try the EpubMerge test version posted here.

[Electricboa]

Originally Posted by JimmXinu

I would suggest copying the output_css setting from under [epub] in defaults.ini into your personal.ini under [epub] and remove the background-color CSS setting.

I don't use CoolReader, so I can't say for sure what will work; you'll have to experiment and see. I would recommend, while experimenting, to edit an epub to make sure you're getting what you want and then worry about getting it correctly into FFF's personal.ini settings.

I already tried that. I never changed any of my setting in the personal.ini until after the update and it added the color background. It’s not an issue with CoolReader because nothing else ever has a white background, and Fanficfare stories didn’t until after I updated.

But I tried deleting the background color again in the personal.ini and it didn’t change anything. When I go to the stylesheet.css of a story, I see this:

body { background-color: #ffffff;
text-align: justify;
margin: 2%;
adobe-hyphenate: none; }
pre { font-size: x-small; }
h1 { text-align: center; }
h2 { text-align: center; }
h3 { text-align: center; }
h4 { text-align: center; }
h5 { text-align: center; }
h6 { text-align: center; }
.CI {
text-align:center;
margin-top:0px;
margin-bottom:0px;
padding:0px;
}
.center   {text-align: center;}
.cover    {text-align: center;}
.full     {width: 100%; }
.quarter  {width: 25%; }
.smcap    {font-variant: small-caps;}
.u        {text-decoration: underline;}
.bold     {font-weight: bold;}
.big { font-size: larger; }
.small { font-size: smaller; }

I’m not looking to change anything else, I just want the background to be transparent. Just the text. I can remove the background color on an individual file by deleting it or replacing it with #00000000 and it works, but I can't get FFF to do it with it downloads it. It always reverts back to the white, even if the personal.ini has it gone.

Originally Posted by JimmXinu

If you use EpubMerge either directly or using FFF's 'Make Anthology' feature, the title page TOC entry is removed automatically for one-chapter stories.

Or, at least, it's supposed to. I found a couple stories that didn't work on while spot testing to make sure I remembered correctly.
I'll have to get back to you about that.

The older version of FFF (I wish I remembered the version) just never created the title page. EpubMerge never removed them because they weren’t there to begin with. And near as I can tell, it’s on everything. I went to AO3 and it’s the same exact thing. The Title Page is always there and when merged it just carries it over:

Here’s what the original file table of contents looks like:



And the merged file:



The first story is mutli-chapter, but the other two are single chapter. Here are the links to them, if you want to recreate it:

https://archiveofourown.org/works/22061086
https://archiveofourown.org/works/25600168
https://archiveofourown.org/works/25694938

Or maybe is there an archive of older versions of the plugin somewhere? I’d happily try and go back until I get to the version where neither issue existed.

Originally Posted by JimmXinu

Literotica story URLs are a mess. There is no 'story URL' as FFF would normally define it and no way to normalize story URL before fetching metadata pages, by which time it's too late.

I don't think there's a reasonable solution for that.

That's unfortunate, but I understand.