[GUI Plugin] FanFicFare

[DandalfTheWhite]

May be user error on my end but weird thing happened. I am in 7.18.0 in Linux Mint with the addon up to date. When I downloaded all the links from my email today, instead of updating the stories in my library already, it downloaded new copies of the stories it updated and didn't give an error or anything. I just used the find duplicates and used FFF to updated the already existing books and deleted the new versions and there were no issues so I am not sure why it made new library books instead of updating. I have the setting ticked to "Update EPUB if New Chapters" and everything else looks fine, and this is the first time this issue has appeared. Just wanting to give you a heads up in case it helps you with development.

[ownedbycats]

Originally Posted by wildfire070

I think I got it by adding

use_workskin:true

to the AO3 section. Can I continue to leave that in my personal.ini or should I only edit it to true for certain stories. Will it do more harm than good by leaving it default.

If you notice that the workskin is somehow screwing up a specific story download, you can just use_workskin:false in a storyUrl section.

[JimmXinu]

Originally Posted by DandalfTheWhite

May be user error on my end but weird thing happened. I am in 7.18.0 in Linux Mint with the addon up to date. When I downloaded all the links from my email today, instead of updating the stories in my library already, it downloaded new copies of the stories it updated and didn't give an error or anything. I just used the find duplicates and used FFF to updated the already existing books and deleted the new versions and there were no issues so I am not sure why it made new library books instead of updating. I have the setting ticked to "Update EPUB if New Chapters" and everything else looks fine, and this is the first time this issue has appeared. Just wanting to give you a heads up in case it helps you with development.

Did you note what the titles/authors/story URLs of the 'new' vs 'old' stories were? Or at least what site?

There was a change to how FFF searches for existing stories to update, but it should be matching more, not less.

[DandalfTheWhite]

Originally Posted by JimmXinu

Did you note what the titles/authors/story URLs of the 'new' vs 'old' stories were? Or at least what site?

There was a change to how FFF searches for existing stories to update, but it should be matching more, not less.

They were all AO3 links and there was like 60 in my update which were impacted. They came from AO3 email alerts in my FFF gmail account and so I don't think the URLs would be different--other than maybe I added them as "view_full_work=true" and the update URL would not have that, but that's the only thing I can think. Most (I didn't check all of them) had the same title and author. A few had some custom metadata fields like when I read it, etc.

I just tried to update a fic that was one I noticed did the dupe issue and then I entered the URL manually and I got "FanFicFare doesn't recognize chapters in existing epub, epub is probably from a different source. Use Overwrite to force update". I tried it out on a few other fics I updated today and they all gave the same error so I will do the force overwrite on them today. All the epubs were made with FFF. Let me know if you need more info and I will try to provide it!

Edit: After more testing most of the books when I try to update them just say no new chapters so that may be unrelated.

[JimmXinu]

AO3 story URLs don't contain a title or anything that changes. So unless you managed to remove the identifiers:url metadata from the Calibre books, it should have found them. Unless you had selected FFF's 'Add Book' mode for some reason. FFF also has an option to match by title/author.

As for "FanFicFare doesn't recognize chapters..." that is almost always caused by editing the epubs with some other tool that restructures the epub, or breaks FFF's hidden metadata.

An alternative would if you have a chapter_start setting that doesn't include the hidden <meta> tags correctly.

If you post one of the "doesn't recognize chapters" epubs, I can probably tell you which. SFW please.

[DandalfTheWhite]

Originally Posted by JimmXinu

AO3 story URLs don't contain a title or anything that changes. So unless you managed to remove the identifiers:url metadata from the Calibre books, it should have found them. Unless you had selected FFF's 'Add Book' mode for some reason. FFF also has an option to match by title/author.

As for "FanFicFare doesn't recognize chapters..." that is almost always caused by editing the epubs with some other tool that restructures the epub, or breaks FFF's hidden metadata.

An alternative would if you have a chapter_start setting that doesn't include the hidden <meta> tags correctly.

If you post one of the "doesn't recognize chapters" epubs, I can probably tell you which. SFW please.

Sorry, I overwrote them all but I do not edit the book files with anything other FFF, and sometimes Epub Merge if it is an anthology. I will say that some of the stories I added fresh, not from an update, that I had in my FFF email account also made that kind of file though.

[madame_newbie]

Okay, I am not sure if this a bug but I noticed that whenever I tried to download a very image heavy fic from AO3, it takes a very, very long time.

(My record is one fic that took 4 hours+). And it's rather inconsistent? I managed to download one image heavy fic just fine but some will take hours.

I managed to bypass it by downloading the fic via html and using the calibre editor to download the images but is there a way for me to add something that says if image is heavier than certain mb/gb, fanficfare will bypass it automatically?

[JimmXinu]

Originally Posted by madame_newbie

Okay, I am not sure if this a bug but I noticed that whenever I tried to download a very image heavy fic from AO3, it takes a very, very long time. ...

The slow_down_sleep_time setting (AO3 default: 2sec) applies: ie, FFF sleeps that long between making network requests. So if a chapter has 150 images, it will take at least 300sec (5 minutes)--and probably more--just for that one chapter.

It's also entirely possible that the image hosting server adds more delay by serving the images slowly if it notices you hitting it a lot. Or is just slow in general.

When you say "downloading the fic via html" do you mean the "Download > HTML" site button, or using FFF's html mode?

FFF doesn't know the image size until after it's been downloaded. In theory, FFF could make an HTTP HEAD request, but not all sites implement it. I'm not much interested in trying.

[madame_newbie]

Originally Posted by JimmXinu

The slow_down_sleep_time setting (AO3 default: 2sec) applies: ie, FFF sleeps that long between making network requests. So if a chapter has 150 images, it will take at least 300sec (5 minutes)--and probably more--just for that one chapter.

It's also entirely possible that the image hosting server adds more delay by serving the images slowly if it notices you hitting it a lot. Or is just slow in general.

When you say "downloading the fic via html" do you mean the "Download > HTML" site button, or using FFF's html mode?

FFF doesn't know the image size until after it's been downloaded. In theory, FFF could make an HTTP HEAD request, but not all sites implement it. I'm not much interested in trying.

Ah, ok ok. I had to download i via download>html from the site itself and download the metadata using fanficfare and calibre editor to download the images manually.

Considering the extra work needed I will continue to use the above method for any image heavy fics. Thanks.

[wildfire070]

I know AO3 had some login issues yesterday, but they seem to have resolved it. I'm able to login on mobile and mac Safari, Firefox and Vivaldi. But when trying to download via FFF, it gives me a pop up window to log in multiple times. And at the end gives me a "Failed to Login for URL". Any ideas?

[Ealdwulf]

Originally Posted by wildfire070

I know AO3 had some login issues yesterday, but they seem to have resolved it. I'm able to login on mobile and mac Safari, Firefox and Vivaldi. But when trying to download via FFF, it gives me a pop up window to log in multiple times. And at the end gives me a "Failed to Login for URL". Any ideas?

I am running into the same issue as far as prompting for login information. only seems to happen with M and E rated stories though. I even verified nothing had changed in my personal.ini file.

[JimmXinu]

I'm not seeing this happen. But I don't have a story handy that's both 'restricted' (requires login) and Mature; I tested each separately.

If somebody PM's me a restricted Mature story URL, I'll try it, too.

[wildfire070]

Originally Posted by JimmXinu

I'm not seeing this happen. But I don't have a story handy that's both 'restricted' (requires login) and Mature; I tested each separately.

If somebody PM's me a restricted Mature story URL, I'll try it, too.

Thanks for looking into it. I PM'd you a couple of URLs that were giving me the login error.

ETA: Update options I'm using are epub output and 'Update if New Chapters'

[Ealdwulf]

Originally Posted by JimmXinu

I'm not seeing this happen. But I don't have a story handy that's both 'restricted' (requires login) and Mature; I tested each separately.

If somebody PM's me a restricted Mature story URL, I'll try it, too.

I just sent you a URL. Just before sending I tried to do an update to see if it fixed and still got prompted for login information. Option I used for update was epub and 'Overwrite if Newer'

[Ealdwulf]

Now things get really weird.

A while back I created a test library so I could work with things like add_to_replace_metadata, exclude_metadata_pre, replace_metadata, etc. but had never used it.

I copy that first story to that test library and then change to it, attempt to update and get the message website is not newer which I know is wrong because there is a newer chapter than what I have so I use 'Overwrite Always' and it pulls it so I can update. I tell it 'No' since I am only using as a test.

I go back to the original library and try that same story again with original option of 'Overwrite if Newer' and it gives me no error but detects a newer version and offers to update, so again I tell it 'No', as I am still testing things. That is when I go test the second story I sent you the URL for.

It still gives the error of wanting my to provide login credentials. So I have No clue what is happening.