[GUI Plugin] FanFicFare

[culytera]

Originally Posted by JimmXinu

The %(var)s substitution is a built-in feature of the INI file parser. It doesn't know anything about FFF metadata entries. (Honestly, I'd rather it didn't--we have to escape % to %% far more often than it's useful.)

I was all set to tell you it's not possible, when I realized, holy crap, it is! An ugly way I'm mildly ashamed of, but a way.

Way back when, I stuck output_css into the story metadata as a convenient (if cheesy) way to pass it to the writer code. And it's been that way ever since.

Which means output_css can be edited by replace_metadata.

Hey if it works, it works. Had to adjust the colors again to make the white text more readable, looks great now. Thank you as always!

[JimmXinu]

Originally Posted by Atherton25

I've been trying to get a couple of stories to update metadata correctly with storiesonline and I can't seem to get it to put a score in my custom column. It's worked before, not sure when it quit. I did have a problem a few months back and lost all my personal.ini settings. It may have happened then, but I could swear I have the right settings in there now.
...

I tried the posted settings and it works fine for me.

Only a couple possible issues come to mind:

• Is #score a float or text type column that will accept a 1.11 type value?
• Do you have any other [storiesonline.net] section(s) in personal.ini that may be conflicting?
• Not all stories have score values.

[DirCat]

Well, it seems that Outlook no longer allows FanFicFare to get the subscription emails anymore. I will be looking for another email service to replace them (when I had replaced Gmail with them) in a bit.

Here's the error I got in case it isn't because of Outlook:

calibre, version 7.4.0
ERROR: Unhandled exception: <b>error</b>:b'LOGIN failed.'

calibre 7.4 embedded-python: True
Windows-10-10.0.22631-SP0 Windows ('64bit', 'WindowsPE')
('Windows', '10', '10.0.22631')
Python 3.11.5
Windows: ('10', '10.0.22631', 'SP0', 'Multiprocessor Free')
Interface language: None
Successfully initialized third party plugins: Count Pages (1, 13, 4) && FanFicFare (4, 30, 0) && Find Duplicates (1, 10, 8) && Generate Cover (2, 3, 3) && View Manager (1, 10, 2)
Traceback (most recent call last):
File "calibre_plugins.fanficfare_plugin.fff_plugin" , line 566, in get_urls_from_imap_menu
File "C:\Users\{I removed my username}\AppData\Roaming\calibre\plugins\FanFicFa re.zip\fanficfare\geturls.py", line 189, in get_urls_from_imap
File "imaplib.py", line 612, in login
imaplib.IMAP4.error: b'LOGIN failed.'

[JimmXinu]

That error just says your login failed.

A little searching lead me to this link: POP, IMAP, and SMTP settings for Outlook.com. Which suggests that IMAP does still work.

[ownedbycats]

I think Outlook.com might be having IMAP issues in general. I keep getting failures in Mozilla Thunderbird and FairEmail (Android), and confirmed that another friend is also having issues.

[JimmXinu]

New Test Version Posted

2024-01-16
- Update adapter_royalroadcom to remove 'stolen' injections. Thanks, grenskul

See #1028 for discussion.

[JimmXinu]

New Test Version Posted

2024-01-16
- Update adapter_royalroadcom to remove 'stolen' injections. Thanks, grenskul

Take two.

[applegaa]

Originally Posted by JimmXinu

New Test Version Posted

2024-01-16
- Update adapter_royalroadcom to remove 'stolen' injections. Thanks, grenskul

Take two.

Does it remove them from previous chapters too, or just newly downloaded ones?

- Andrew

EDIT: Never mind. It looks like they had changed it again after the initial rollout that could be countered with a single CSS change.

[JimmXinu]

Originally Posted by applegaa

Does it remove them from previous chapters too, or just newly downloaded ones?

- Andrew

EDIT: Never mind. It looks like they had changed it again after the initial rollout that could be countered with a single CSS change.

Only newly downloaded chapters.

It seems to be working right now, but yes, the site could break this at any time with any number of reasonably small changes.

Personally, I'm very conflicted about both this change and supporting RoyalRoad in general, since it's a site of primarily original material with only a little fanfiction.

[applegaa]

Originally Posted by JimmXinu

Personally, I'm very conflicted about both this change and supporting RoyalRoad in general, since it's a site of primarily original material with only a little fanfiction.

I highly doubt that the folks stealing stories to sell on Amazon are using FanFicFare. They will be using tools that are more streamlined for their purpose and don't include your throttling.

To draw an analogy from the past, it's not really any different than recording TV shows on a VCR. If I was watching it while recording, I often paused the recording during commercials. I have also been known to use two VCRs to remove commercials in the original recording. I also used that method to combine movies and their sequels onto one VHS tape. (Back then, Star Wars was only 3 movies...)

Using Calibre and FanFicFare to capture the signal from the Royal Road web server, for example, is no different. As long as it is for personal use, the retention of freely provided content is completely legal. Converting it from a standard HTML document to an ebook is no different than copying your Betamax tapes to VHS or having a text to speech utility convert it to audio. (I bet those folks are annoyed by these inserts as well.)

So, while I understand your concerns and appreciate that you are ethical enough to have them, I think you're overly worrying about it.

- Andrew

[mmmmmmmm]

hi! im new here and i dont know if this is how im supposed to post a question?

but ive been trying to use FFF to download from ao3. i wanted to keep the different metadata categories (like relationships, fandom etc.) but im really struggling. i feel like it only works sometimes? and not in all of the categories.
i know others have asked about this before, and ive tryed to follow the tips/instructions but im still lost.

im pretty sure im doing something wrong. i dont understand coding at all, but ive pasted some code that others have shared in an attempt to fix it?

code:
## This is an example of what your personal configuration might look
## like. Uncomment options by removing the '#' in front of them.

[defaults]
## [defaults] section applies to all formats and sites but may be
## overridden at several levels. See
## https://github.com/JimmXinu/FanFicFare/wiki/INI-File for more
## details.
extra_valid_entries:fandoms



## Some sites also require the user to confirm they are adult for
## adult content. Uncomment by removing '#' in front of is_adult.
is_adult:true

## Don't like the numbers at the start of chapter titles on some
## sites? You can use strip_chapter_numbers to strip them off. Just
## want to make them all look the same? Strip them off, then add them
## back on with add_chapter_numbers. Don't like the way it strips
## numbers or adds them back? See chapter_title_strip_pattern and
## chapter_title_add_pattern in defaults.ini.
strip_chapter_numbers:true
add_chapter_numbers:true


[epub]
## Include images from img tags in the body and summary of stories.
## Images will be converted to jpg for size if possible. Images work
## in epub format only. To get mobi or other format with images,
## download as epub and use Calibre to convert.
## true by default, uncomment and set false to not include images.
include_images:true

## If set false, the summary will have all html stripped for safety.
## Both this and include_images must be true to get images in the
## summary.
## true by default, uncomment and set false to not keep summary html.
#keep_summary_html:true

## If set true, and there isn't a specific cover image, the first
## image found in the story will be made the cover image. If
## keep_summary_html is true, images in the summary will be before any
## in chapters.
## true by default, uncomment and set false to turn off
make_firstimage_cover:false


## Most common, I expect will be using this to save username/passwords
## for different sites. Here are a few examples. See defaults.ini
## for the full list.

[www.twilighted.net]
#username:XXXXXXXX
#password:XXXXXXXX
## default is false
#collect_series: true

[www.fimfiction.net]
is_adult:true
#fail_on_password:XXXXXXXX

[www.tthfanfic.org]
is_adult:true
## tth is a little unusual--it doesn't require user/pass, but the site
## keeps track of which chapters you've read and won't send another
## update until it thinks you're up to date. If you set
## username/password, FFF will login to download. Then the site
## thinks you're up to date.
#username:XXXXXXXX
#password:XXXXXXXX

[archiveofourown.org]
username:XXXXXXXX
password:XXXXXXXX
is_adult:true
exclude_metadata_pre:
freeformtags,genre==Other Additional Tags to Be Added
warnings==Creator Chose Not To Use Archive Warnings
warnings==No Archive Warnings Apply


## This section will override anything in the system defaults or other
## sections here.
[overrides]
## default varies by site. Set true here to force all sites to
## collect series.
collect_series: true

extra_valid_entries:fandoms, freeformtags, freefromtags, ao3categories, comments, chapterslashtotal, chapterstotal, kudos, hits, bookmarks, collections, byline, bookmarked, bookmarktags, bookmarksummary, bookmarkprivate, bookmarkrec, restricted, series00, series01, series02, series03, series04, series05, series06, series07, series08, series09, series00Url, series01Url, series02Url, series03Url, series04Url, series05Url, series06Url, series07Url, series08Url, series90Url, series00HTML, series01HTML, series02HTML, series03HTML, series04HTML, series05HTML, series06HTML, series07HTML, series08HTML, series09HTML
fandoms_label:fandoms
freeformtags_label:Freeform Tags
freefromtags_label:Freeform Tags
ao3categories_label:AO3 Categories
comments_label:Comments
chapterslashtotal_label:Chapters/Total Chapters
chapterstotal_label:Total Chapters
## kudos_label:Kudos
## hits_label:Hits
## collections_label:Collections
## Count of bookmarks on story by all users bookmarks_label:Bookmarks
## Tags & Summary from *your* bookmark on the story. Only collected
## if always_login:true bookmarked_label:I Bookmarked Story bookmarktags_label:My Bookmark Tags
## bookmarksummary_label:My Bookmark Summary
## bookmarkprivate_label:My Bookmark Private
## bookmarkrec_label:My Bookmark Rec
## restricted_label:Restricted to Registered Users
series00HTML_label:Series
series01HTML_label:Additional Series
series02HTML_label:Additional Series
series03HTML_label:Additional Series
series04HTML_label:Additional Series
series05HTML_label:Additional Series
series06HTML_label:Additional Series
series07HTML_label:Additional Series
series08HTML_label:Additional Series
series09HTML_label:Additional Series
make_linkhtml_entries:series00,series01,series02,s eries03,series04,series05,series06,series07,series 08,series09
titlepage_entries: seriesHTML,category,genre,language,characters,ship s,status,datePublished,dateUpdated,dateCreated,rat ing,warnings,numChapters,numWords,site,description extra_titlepage_entries: fandoms,freeformtags,ao3categories,comments,chapte rslashtotal,chapterstotal,kudos,hits,bookmarks,boo kmarked,bookmarktags,bookmarksummary,series01HTML, series02HTML,series03HTML,series04HTML,series05HTM L,series06HTML,series07HTML,series08HTML,series09H TML,byline



custom_columns_settings:
fandoms:=>#fandom,a


if anyone could help me out, i'd really apreciate it!!

[JimmXinu]

Originally Posted by mmmmmmmm

hi! im new here and i dont know if this is how im supposed to post a question?

You are in the right place for this sort of question, yes. Welcome.

FYI, as a new user, your posts may be stuck waiting for moderator approval at first. Wait a few hours before reposting if you don't immediately see your own posts.

Originally Posted by mmmmmmmm

but ive been trying to use FFF to download from ao3. i wanted to keep the different metadata categories (like relationships, fandom etc.) but im really struggling. i feel like it only works sometimes? and not in all of the categories.
i know others have asked about this before, and ive tryed to follow the tips/instructions but im still lost.
...

Existing Settings
First, your posted settings shouldn't need most of what you have under [overrides]--those mostly look like default settings for [archiveofourown.org] that you shouldn't need to copy to personal.ini at all.

Under [overrides] is a valid place for custom_columns_settings, but it needs correct whitespace, and you have an extra ':'.

For future reference, it's best practice to put lengthy personal.ini chunks in [code] tags to preserve important whitespace nested inside [spoiler] tags to make easier reading for others. Like this:

Spoiler:

## Note that there's a space before each line under custom_columns_settings
custom_columns_settings:
 fandoms=>#fandom,a

category vs fandoms
For historical reasons, FFF usually stores a fanfictions' parent series (aka fandom) in category. AO3 OTOH, call M/F, F/F, M/M, etc 'Categories'. By defaults FFF includes AO3's Additional (aka Free Form) Tags and Categories in genre and Fandoms in category.

Also for historical reasons, FFF uses the include_subject_tags setting to determine what goes into Calibre's standard Tags column. By defaults, that includes genre and category. The comments in defaults.ini are considered the primary documentation for a lot of settings.

You might want to use the GUI Custom columns settings to put Category into your custom column instead of the AO3 specific fandoms. It's up to you.

AO3 Metadata in general
AO3 really does not make it easy to wrangle its metadata. The site lets authors put whatever they want into most any field they want, without regard to correct placement, consistency, or even accurate spelling.

I will try to help when you have specific problems, ie, you can provide a story URL(s) and your settings and clearly tell me what you see happen and what you want to see instead.

But I have long ago given up on trying to 'solve' AO3 metadata as a general problem.

[John Maverick]

Since 3 weeks ago I can't update or download stories any from ff.net using FlareSolverr. This is the error shown:

Estado
Título
Autor
Comentario
URL
Error
Unknown
Unknown
Connection to flaresolverr proxy server failed.  Is flaresolverr started?
https://www.fanfiction.net/s/14321954/1/Dragon-Ball-M-Mirai-New-Future

Regardless what the error message said FlareSolverr is running. But I think that is the default failure message.

Is anyone else having the same problem?

My setup is the following:
OS: Linux Mint 21.2 x86_64
Calibre version: 7.4
FanFicFare version: v4.30.0
FlareSolverr version: 3.3.13

Like always thanks in advance.

[JimmXinu]

Originally Posted by John Maverick

Since 3 weeks ago I can't update or download stories any from ff.net using FlareSolverr. This is the error shown:
...

Assuming you are using the default port numbers and running both on the same computer, I would suspect that there's a firewall module or something blocking the connection.

[mmmmmmmm]

Thank you for responding so fast! and for the helpful pointers!!
i'm sorry i wasnt able do respond fast too, but i really do apreciate it!

i'll put my Personal.ini here (thanks for showing me how to do that, its really cool!):

Spoiler:

## This is an example of what your personal configuration might look
## like.  Uncomment options by removing the '#' in front of them.

[defaults]
## [defaults] section applies to all formats and sites but may be
## overridden at several levels.  See
## https://github.com/JimmXinu/FanFicFare/wiki/INI-File for more
## details.
extra_valid_entries:fandoms



## Some sites also require the user to confirm they are adult for
## adult content.  Uncomment by removing '#' in front of is_adult.
is_adult:true

## Don't like the numbers at the start of chapter titles on some
## sites?  You can use strip_chapter_numbers to strip them off.  Just
## want to make them all look the same?  Strip them off, then add them
## back on with add_chapter_numbers.  Don't like the way it strips
## numbers or adds them back?  See chapter_title_strip_pattern and
## chapter_title_add_pattern in defaults.ini.
strip_chapter_numbers:true
add_chapter_numbers:true


[epub]
## Include images from img tags in the body and summary of stories.
## Images will be converted to jpg for size if possible.  Images work
## in epub format only.  To get mobi or other format with images,
## download as epub and use Calibre to convert.
## true by default, uncomment and set false to not include images.
include_images:true

## If set false, the summary will have all html stripped for safety.
## Both this and include_images must be true to get images in the
## summary.
## true by default, uncomment and set false to not keep summary html.
#keep_summary_html:true

## If set true, and there isn't a specific cover image, the first
## image found in the story will be made the cover image.  If
## keep_summary_html is true, images in the summary will be before any
## in chapters.
## true by default, uncomment and set false to turn off
make_firstimage_cover:false 


## Most common, I expect will be using this to save username/passwords
## for different sites.  Here are a few examples.  See defaults.ini
## for the full list.


[archiveofourown.org]
username:XXXXXXXX
password:XXXXXXXX
is_adult:true


## This section will override anything in the system defaults or other
## sections here.
[overrides]
## default varies by site.  Set true here to force all sites to
## collect series.
collect_series: true

extra_valid_entries:fandoms, freeformtags, freefromtags,  ao3categories, comments, chapterslashtotal, chapterstotal, kudos,  hits, bookmarks, collections, byline, bookmarked, bookmarktags,  bookmarksummary, bookmarkprivate, bookmarkrec, restricted, series00,  series01, series02, series03, series04, series05, series06, series07,  series08, series09, series00Url, series01Url, series02Url, series03Url,   series04Url, series05Url, series06Url, series07Url, series08Url, series90Url,  series00HTML, series01HTML, series02HTML, series03HTML, series04HTML,  series05HTML, series06HTML, series07HTML, series08HTML, series09HTML 
fandoms_label:fandoms 
freeformtags_label:Freeform Tags 
freefromtags_label:Freeform Tags 
ao3categories_label:AO3 Categories 
comments_label:Comments 
chapterslashtotal_label:Chapters/Total Chapters 
chapterstotal_label:Total Chapters 
## kudos_label:Kudos 
## hits_label:Hits 
## collections_label:Collections 
## Count of bookmarks on story by all users bookmarks_label:Bookmarks 
## Tags & Summary from *your* bookmark on the story.  Only collected 
## if always_login:true bookmarked_label:I Bookmarked Story bookmarktags_label:My Bookmark Tags 
## bookmarksummary_label:My Bookmark Summary 
## bookmarkprivate_label:My Bookmark Private 
## bookmarkrec_label:My Bookmark Rec 
## restricted_label:Restricted to Registered Users 
series00HTML_label:Series 
series01HTML_label:Additional Series 
series02HTML_label:Additional Series 
series03HTML_label:Additional Series 
series04HTML_label:Additional Series 
series05HTML_label:Additional Series 
series06HTML_label:Additional Series 
series07HTML_label:Additional Series 
series08HTML_label:Additional Series 
series09HTML_label:Additional Series  
make_linkhtml_entries:series00,series01,series02,series03,series04,series05,series06,series07,series08,series09  
titlepage_entries: seriesHTML,category,genre,language,characters,ships,status,datePublished,dateUpdated,dateCreated,rating,warnings,numChapters,numWords,site,description extra_titlepage_entries: fandoms,freeformtags,ao3categories,comments,chapterslashtotal,chapterstotal,kudos,hits,bookmarks,bookmarked,bookmarktags,bookmarksummary,series01HTML,series02HTML,series03HTML,series04HTML,series05HTML,series06HTML,series07HTML,series08HTML,series09HTML,byline  



custom_columns_settings: 
 fandoms:=>#fandom,a

should i just change everything back to the default and go from there?
----
you mentioned FFF usually calls AO3's Free Form Tags and Categories in genre and Fandoms in category. does that mean i should change the names of my columns?
----
i dont know how to use include_subject_tags but i will check the wiki.
----

You might want to use the GUI Custom columns settings to put Category into your custom column instead of the AO3 specific fandoms. It's up to you.

i dont understand what you mean by this? should i re name my fandom column to 'category'?
-----

AO3 really does not make it easy to wrangle its metadata. The site lets authors put whatever they want into most any field they want, without regard to correct placement, consistency, or even accurate spelling.

i can see how that makes things more complicated!

-------
here is a story ive tried downloading:
https://archiveofourown.org/works/24280306

problems:
- nothing shows up in my fandoms colomn.
- all the tags from all the categories are showing up in my tags column
- nothing shows up in content rating (mature, teen and up, etc. )
- the relationships column sort of works. 'characterA/characterB' tags show up fine, but 'characterA & characterB' tag doesnt really work? like it shows up, but as separate tags? like 'characterA' & 'characterB', so i cant filter by those tags. is there a way to fix that?
- nothing shows up in my category column (M/M, F/F, etc.) but you mentioned FFF calls them genre? should i rename the coulumn? im happy to do that, if it will help


Thank you for your time and effort!