[GUI Plugin] FanFicFare

[JimmXinu]

New Test Version Posted

2023-03-26
- adapter_inkbunnynet: Honor keep_summary_html, allow alternate cover image. Thanks hmonsta #912.

[ownedbycats]

Any errors in this before I add to the MetadataManagement page? I'm mostly concerned about the titlepage_entries.

### Alternate Method for Series Descriptions

This requires Calibre 7.0 or later, which added [category notes](https://calibre-ebook.com/new-in/sixteen).

To begin with, go to Preferences > Add your own columns, and create a new column with a lookup name of `seriesdesc`, type of 'column, built from other columns', and a template of ``program: get_note('series', $series, '')``. You may also want to use 'Show with comments in book details.'

This will allow you to add a note to the series by right-clicking the entry in Book Details.

With 'Pass Calibre Columns into FanFicFare on Update/Overwrite' enabled, this allows the user to then insert the series description on the title page:

    [defaults]
    calibre_cust_seriesdesc_label:Series Description
    titlepage_entries: seriesHTML,category,genre,language,characters,ships,status,datePublished,dateUpdated,dateCreated,rating,warnings,numChapters,numWords,publisher,calibre_cust_seriesdesc,description

Note that upon downloading a book from a series from the first time, you will need to add the series note and then run 'Update Always' to have it appear.

[JimmXinu]

I don't use calibre notes, so I have no comment on that part. For example, I don't know what the advantage of using notes over a long text column is.

Might want to add calibre_cust_seriesdesc to wide_titlepage_entries, but that's not as important as it was back when title page was a <table>. That's why description formats differently.

[defaults] titlepage_entries is overridden by [txt] version in defaults.ini, but I doubt anyone doing this will be using txt.

[ownedbycats]

PSA: Archive of our Own is currently down for maintenance to fix some issues.

Originally Posted by JimmXinu

I don't use calibre notes, so I have no comment on that part. For example, I don't know what the advantage of using notes over a long text column is.

Lack of duplication is the main one for me. Instead of having to populate a column with the same description on every book (and needing to run a search-and-replace to update this), I just have one note.

[JimmXinu]

Fair enough. I didn't realize the note was on the series record, not the book record.

Hmmm. I get an error trying to open the Series Manager. I wonder what's up with that?

Update: Restarted Calibre and it works again.

[ownedbycats]

It's working for me on 7.7.100.

[nomcocainenom]

So, I have a weird case with the plugin. I was downloading some prospective stories to read from sufficient velocity, and it refused to download ONE story specifically. Looking at the error log, it seems to say 404. Which is wild because every other story works fine. Anyone got a clue what's going on here?

Spoiler:

calibre, version 7.7.0
ERROR: Error Updating Metadata: <p>An error has occurred while FanFicFare was updating calibre's metadata for <a href='https://forums.sufficientvelocity.com/threads/the-dungeon-without-a-system.97590/'>Unknown</a>.</p>

Traceback (most recent call last):
File "C:\Users\nomnom\AppData\Roaming\calibre\plugins\F anFicFare.zip\fanficfare\fetchers\fetcher_requests .py", line 128, in request
File "C:\Users\nomnom\AppData\Roaming\calibre\plugins\F anFicFare.zip\requests\models.py", line 943, in raise_for_status
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://forums.sufficientvelocity.co...-system.97590/

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "C:\Users\nomnom\AppData\Roaming\calibre\plugins\F anFicFare.zip\fanficfare\adapters\base_adapter.py" , line 325, in getStoryMetadataOnly
File "C:\Users\nomnom\AppData\Roaming\calibre\plugins\F anFicFare.zip\fanficfare\adapters\base_adapter.py" , line 429, in doExtractChapterUrlsAndMetadata
File "C:\Users\nomnom\AppData\Roaming\calibre\plugins\F anFicFare.zip\fanficfare\adapters\base_xenforoforu m_adapter.py", line 481, in extractChapterUrlsAndMetadata
File "C:\Users\nomnom\AppData\Roaming\calibre\plugins\F anFicFare.zip\fanficfare\requestable.py", line 111, in get_request_redirected
File "C:\Users\nomnom\AppData\Roaming\calibre\plugins\F anFicFare.zip\fanficfare\fetchers\base_fetcher.py" , line 133, in get_request_redirected
File "C:\Users\nomnom\AppData\Roaming\calibre\plugins\F anFicFare.zip\fanficfare\fetchers\cache_basic.py", line 122, in fetcher_do_request
File "C:\Users\nomnom\AppData\Roaming\calibre\plugins\F anFicFare.zip\fanficfare\fetchers\decorators.py", line 102, in fetcher_do_request
File "C:\Users\nomnom\AppData\Roaming\calibre\plugins\F anFicFare.zip\fanficfare\fetchers\base_fetcher.py" , line 106, in do_request
File "C:\Users\nomnom\AppData\Roaming\calibre\plugins\F anFicFare.zip\fanficfare\fetchers\fetcher_requests .py", line 149, in request
fanficfare.exceptions.HTTPErrorFFF: HTTP Error in FFF '404 Client Error: Not Found for url: https://forums.sufficientvelocity.co...7590/'(404)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "calibre_plugins.fanficfare_plugin.dialogs", line 715, in do_loop
File "calibre_plugins.fanficfare_plugin.fff_plugin" , line 1356, in prep_download_loop
File "calibre_plugins.fanficfare_plugin.fff_plugin" , line 1236, in get_story_metadata_only
File "C:\Users\nomnom\AppData\Roaming\calibre\plugins\F anFicFare.zip\fanficfare\adapters\base_adapter.py" , line 328, in getStoryMetadataOnly
fanficfare.exceptions.StoryDoesNotExist: Story does not exist: (https://forums.sufficientvelocity.co...-system.97590/)

[JimmXinu]

Xenforo forum sites show user-only threads as 404 unless you are logged in.

You need to add your personal.ini either:

## For just this story
[https://forums.sufficientvelocity.com/threads/the-dungeon-without-a-system.97590/]
username:YourName
password:yourpassword
always_login:true

## or for all SV downloads:
[forums.sufficientvelocity.com]
username:YourName
password:yourpassword
always_login:true

[nomcocainenom]

Thanks. That's wild that after years of getting stories from the website with your (great!) plugin, I've never had this problem before. But I didn't realize it was in the unlisted fiction subforum either, should've noticed that before.

But that solution a bit of a problem itself, I assume it can't get past the Two-Step Verification, since I tried that solution and it didn't work. Hmm, I guess I'm out of luck unless I disable the Two-Step, right?

[JimmXinu]

Correct. IMO, anything that needs 2FA levels of security, you shouldn't be giving that password to FFF anyway.

[JimmXinu]

New Version Released

Version 4.33.0 - 2024-04-01
- Update translations.
- adapter_inkbunnynet: Honor keep_summary_html, allow alternate cover image. Thanks hmonsta #912.
- adapter_deviantartcom: Add a 6th different message to indicate 'mature content'. #1052
- adapter_deviantartcom: Bad username fails separately than bad pass. #1052
- Paste into ini edit as plain text only.
- syosetu: make numeric metadata robust against wording changes - thanks praschke
- New site Japanese language site kakuyomu.jp, thanks praschke
- adapter_deviantartcom: Add another way to remove comments section.
- Don't try to set imap tags before checking for 'good' update.
- Move new exception catching for metadata errors

[jazzhands]

Hoping I'm not repeating something here. FYI, I observed the problem before updating FFF and Epub Merge, updated both plugins, restarted Calibre and the issue is still here.

I'm attempting to use the Update Anthology function to update a series that's already in my calibre.

The series I'm updating has 47 works in it (example: https://archiveofourown.org/series/2579419; 3 pages of individual works). But when I attempt to update, FFF is only pulling in the first 20 works of the series -- just the URLs from page 1 .


Is this a known issue?

Thanks again for all that you do. FFF is such an amazing plugin.

ETA: I have screenshots of the issue if you want them. But I couldn't quickly find a place to host the pics, so I'll just describe them

the first is the Story URLs for Anthology popup, showing only 20 urls found.

the second image is the alert showing that the rest of the works will be deleted from the anthology if I choose to update, because FFF didn't pull those URLs.

[DNSB]

Originally Posted by jazzhands

ETA: I have screenshots of the issue if you want them. But I couldn't quickly find a place to host the pics, so I'll just describe them

the first is the Story URLs for Anthology popup, showing only 20 urls found.

the second image is the alert showing that the rest of the works will be deleted from the anthology if I choose to update, because FFF didn't pull those URLs.

You don't need to host them anywhere. In Advanced mode for message entries, either click on the paperclip icon in the toolbar or the Manage Attachments button in Additional Options below the message entry box and attach your images to the message.

[JimmXinu]

Originally Posted by jazzhands

The series I'm updating has 47 works in it (example: https://archiveofourown.org/series/2579419; 3 pages of individual works). But when I attempt to update, FFF is only pulling in the first 20 works of the series -- just the URLs from page 1 .

I don't recall seeing paginated series on AO3 before.

As a general rule, FFF does not fetch multiple pages with 'Get Story URLs from Web Page' or 'Make Anthology Epub from Web Page'. It's 'Page' singular.

Partly to limit complexity and partly to limit users from deciding to download 100s or 1000s of stories at a time.

However, FFF already has specialized code for series on AO3 (aka OTW), so I will at least consider spinning pages for it.

[nan1923]

Originally Posted by JimmXinu

I don't recall seeing paginated series on AO3 before.

As a general rule, FFF does not fetch multiple pages with 'Get Story URLs from Web Page' or 'Make Anthology Epub from Web Page'. It's 'Page' singular.

Partly to limit complexity and partly to limit users from deciding to download 100s or 1000s of stories at a time.

However, FFF already has specialized code for series on AO3 (aka OTW), so I will at least consider spinning pages for it.

AO3 seems to have changed that today, all series with more than 20 stories are paginated now, I checked multiple I have in my bookmarks and downloaded.

I would really appreciate it, if you would spin pages for it. Even if it's probably only a problem for a handful of users.
And I use slow_down_sleep_time, I usually set it to 2 for ao3, for the odd series with 100+ stories I set it to 4, never had the Retry Later error page with that.

Yes I could use Get URLs from webpage for each page, then copy paste them into make anthology from urls and then add the new url in the right place when it gets updated.

But that would get old really fast since many authors add stories in the middle of a series.

examples:
https://archiveofourown.org/series/1683661
https://archiveofourown.org/series/1034549
https://archiveofourown.org/series/1533116