[GUI Plugin] FanFicFare - Page 332

ownedbycats · 12-19-2020, 01:28 PM

For anyone not checking the GitHub thread, I found that FictionPress is currently doing a migration: https://twitter.com/FictionPress

Looks like they were also getting DDoSed.

ownedbycats · 12-19-2020, 01:54 PM

Quote:

Originally Posted by willowtree3145

Dear Plug-In Authors,

I'm new to the forum, but the most recent addition to the FFF Plug-In for Calibre seems to have a problem. I can no longer download or update books from fanfiction.net (FFNET).

Note: I was trying one book at a time and turning on my VPN doesn't do anything. I also tried multiple books and links. None of them worked. I don't think the plug-in was broken (by me) because I haven't touched the settings. Also, it downloaded two AO3 links just fine.

The error message: "HTTP Error 403: Forbidden"

See this thread:

https://www.mobileread.com/forums/sh...postcount=4961

tl;dr: Cloudflare and server migration.

fanfictioneer · 12-20-2020, 06:02 AM

FYI, I just managed to download some stories by adding my browsers (Vivaldi, Chrome fork) user agent header when retrieving from fanfiction.net.

ownedbycats · 12-20-2020, 06:11 AM

I set ffnet's user_agent to Mozilla/5.0 and it worked.

icecold · 12-20-2020, 06:23 AM

Thank you, setting my user_agent to Mozilla/5.0 also worked for me and I'm using Vivaldi.

Arkeus · 12-20-2020, 09:32 AM

What is this user_agent you are talking about how do you set it to something else?

EDIT: Added a line to personal.ini saying user_agent with said Mozilla/5.0 thing and it worked, but can someone explain to me how you know what numbers to say after mozilla/etc?

vikarti · 12-20-2020, 10:31 AM

Quote:

Originally Posted by Arkeus

What is this user_agent you are talking about how do you set it to something else?

EDIT: Added a line to personal.ini saying user_agent with said Mozilla/5.0 thing and it worked, but can someone explain to me how you know what numbers to say after mozilla/etc?

It's convention that every client program("user-agent") which talks over HTTP should identify.
Original idea was just identification. Later sites started to tweak itself according to user-agent, because it's usually browser and it's only way to determine browser and show 'This is best viewed with X,Y , please update your older Z'(or just provide special version of site to Z)

Because of a lot of historcal bagge, it's got rather complicated.
example:Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.88 Safari/537.36

This is Chrome 87(it's really is), but it's want pages for Safari (so if site detects Safari and NOT Chrome - correct page will be choosen) (because Blink engine was once WebKit, WebKit was originally Safari's engine). AppleWebKit is another Safari's rudiment
KHTML; like Gecko - this is inherited from KDE's Konquer (WebKit was created from Konquer's engine named KHTML), 'Gecko' was started it's life as Netscape Navigator 6.0 and Mozilla Suite's engine(Firefox didn't exist at this time).
'Mozilla' is rudiment of pre-6.0 Netscape Navigator
Win NT/Win64/x64 - just OS data.
Numbers are versions of older engines browser says it's compatible to.

If you want to see user agent of your regular browser - just visit https://developers.whatismybrowser.c...user-agent=yes
Try from different devices/browsers if you have more than one.sa

It's usually possible to fake user agent (setting user-agent: in personal.ini does just that) but it's also possible to detect user agent via other means, indirectly.

New versions of chrome starting to depricate this in favor of 'client hints' (basially 'I can do X').

fdwojo · 12-20-2020, 10:36 AM

I'm guessing that even though JimmXinu could answer this question best, I'm hoping there are a few other savants here who can answer it for me.

In short, 2 months ago, the latest Calibre wouldn't install when downloaded. I know they fixed it after a week or so, but because the standard assumption is that "The problem must have been because of something I did", I made the stupid choice of fully uninstalling and removing all program data for Calibre and tried again, and again, and again. Until they fixed whatever the problem was. Although, as a precaution, I ZIPped up all my Calibre libraries first for safe keeping, wrongly assuming all the library data was contained in those folders. (can you see the second problem I made for myself?)

Anyways, I'm finally getting back into using Calibre again, mostly because Fanfiction.net has changed something yesterday (Dec 19th) and the FanFictionDownloader program (by R. Eisele at fanfictiondownloader.net) doesn't work anymore, so I'm now needing to rely (again) on FanFicFare.

So, my question. Does FanFicFare work "right outta the box" ? Or do I need to set up things to make it work. I used to use it and first installed it back in 2017(?) and can't remember if there was any specific setup to do with it. I exclusively get all my FanFic from FanFiction.net, if that helps any.

Sorry to bother you folks with what is probably a dumb question. The only reason I ask is that I've tried several times yesterday, both with single story URLs and muti-story groups of URLs and they don't work. Of course, it's possible that whatever change FF.net made, it's screwed up ALL the FF downloading methods.

Help?

Frank

P.S. As for the second problem, that you can't just create libraries and copy all the old data back into it, I'm at a loss as to how to restore my Calibre libraries. The only library that works is my default library that I was in and had been using when I uninstalled Calibre. Any ideas on how to recreate the other libraries I had? As I said, I still have the libraries still zipped up. But apparently that's not enough...

ghost9876222 · 12-20-2020, 11:29 AM

Quote:

Originally Posted by fdwojo

I'm guessing that even though JimmXinu could answer this question best, I'm hoping there are a few other savants here who can answer it for me.

In short, 2 months ago, the latest Calibre wouldn't install when downloaded. I know they fixed it after a week or so, but because the standard assumption is that "The problem must have been because of something I did", I made the stupid choice of fully uninstalling and removing all program data for Calibre and tried again, and again, and again. Until they fixed whatever the problem was. Although, as a precaution, I ZIPped up all my Calibre libraries first for safe keeping, wrongly assuming all the library data was contained in those folders. (can you see the second problem I made for myself?)

Anyways, I'm finally getting back into using Calibre again, mostly because Fanfiction.net has changed something yesterday (Dec 19th) and the FanFictionDownloader program (by R. Eisele at fanfictiondownloader.net) doesn't work anymore, so I'm now needing to rely (again) on FanFicFare.

So, my question. Does FanFicFare work "right outta the box" ? Or do I need to set up things to make it work. I used to use it and first installed it back in 2017(?) and can't remember if there was any specific setup to do with it. I exclusively get all my FanFic from FanFiction.net, if that helps any.

Sorry to bother you folks with what is probably a dumb question. The only reason I ask is that I've tried several times yesterday, both with single story URLs and muti-story groups of URLs and they don't work. Of course, it's possible that whatever change FF.net made, it's screwed up ALL the FF downloading methods.

Help?

Frank

P.S. As for the second problem, that you can't just create libraries and copy all the old data back into it, I'm at a loss as to how to restore my Calibre libraries. The only library that works is my default library that I was in and had been using when I uninstalled Calibre. Any ideas on how to recreate them? As I said, I still have the libraries still zipped up. But apparently that's not enough...

recreate new library and after use add multiple book from archives one book per directory in the archive

JimmXinu · 12-20-2020, 11:31 AM

FFnet has made changes over the weekend that broke FFF for everybody. I hope to be able to fix it, but there are no guarantees.

To the best of my knowledge, if you saved everything in the libary directory, unzipping it as a directory and then adding that dir to Calibre under Library -> Switch/create Library should work. Worst case, you add all the saved ebook files to a new library.

But that is as much as I know about it. You'll need to use your google-fu or ask over in the Library Management sub-forum for more help on libraries.

fdwojo · 12-20-2020, 11:54 AM

What I did was I went into Calibre (32bit, by the way), and first recreated all the libraries without any books in them. Then, I unzipped all those other libraries back into their original folders. However, when I start instruct Calibre to look at one of those other libraries, it still shows as being empty with no books, even though I look in that library folder and all the author folders with their books are still there. (When I originally zipped my libraries up, I choose to maintain the directory tree structure with all library folders, author folders, and book folders contained within them just like before when it was working fine before I uninstalled it. And, JimmXinu, does FanFicFare require any setup to be used, or can you just paste in URL's and it'll work without any setup?

Frank

P.S. As much as I am trying to at least be able to see my books, I'm also worried with regard to the custom columns I created for the FanFic I used your plugin for. I had to create 4 or 5 custom folders for specific metadata and I'm concerned I'll have to recreate those custom columns from scratch...

ownedbycats · 12-20-2020, 12:18 PM

Quote:

Originally Posted by Arkeus

What is this user_agent you are talking about how do you set it to something else?

EDIT: Added a line to personal.ini saying user_agent with said Mozilla/5.0 thing and it worked, but can someone explain to me how you know what numbers to say after mozilla/etc?

I just copied it from another of the site entries in defaults.ini.

JimmXinu · 12-20-2020, 08:42 PM

FYI:

Yes, ffnet support is still broken.

The only fix I've found so far would involve packaging at least three more modules in the FFF plugin, one of which(cloudscraper) is Python3 only and would (theoretically) only work in Calibre 5+.

They (ffnet) are also still actively fixing issues with their migration, so I'm not rushing while it's still a moving target.

Further discussion in github issue.

Tanjamuse · 12-21-2020, 01:11 AM

Quote:

Originally Posted by ownedbycats

I just copied it from another of the site entries in defaults.ini.

Like this: user_agent:Mozilla/5.0

Because I added that, but it still says error.

ownedbycats · 12-21-2020, 02:12 AM

I speculated on the GitHub thread that the differences in results is due to location.

If certain ISPS/IP ranges are known for having bad actors, it's plausible that Cloudflare applies more stringent measures to them. I'm not sure if this is actually done, but it lines up with reports of users from certain countries having difficulty accessing the website.

12-19-2020, 01:28 PM	#4966
ownedbycats Custom User Title Posts: 11,003 Karma: 75337983 Join Date: Oct 2018 Location: Canada Device: Kobo Libra H2O, formerly Aura HD	For anyone not checking the GitHub thread, I found that FictionPress is currently doing a migration: https://twitter.com/FictionPress Looks like they were also getting DDoSed. Last edited by ownedbycats; 12-19-2020 at 01:32 PM.

12-20-2020, 06:11 AM	#4969
ownedbycats Custom User Title Posts: 11,003 Karma: 75337983 Join Date: Oct 2018 Location: Canada Device: Kobo Libra H2O, formerly Aura HD	I set ffnet's user_agent to Mozilla/5.0 and it worked. Last edited by ownedbycats; 12-20-2020 at 06:14 AM.

12-20-2020, 09:32 AM	#4971
Arkeus Enthusiast Posts: 27 Karma: 10 Join Date: Oct 2016 Device: Kobo	What is this user_agent you are talking about how do you set it to something else? EDIT: Added a line to personal.ini saying user_agent with said Mozilla/5.0 thing and it worked, but can someone explain to me how you know what numbers to say after mozilla/etc? Last edited by Arkeus; 12-20-2020 at 09:37 AM.

12-20-2020, 10:36 AM	#4973
fdwojo Member Posts: 23 Karma: 10 Join Date: Jun 2012 Device: Samsung Note20 Ultra, Kindle Fire HD10	I'm guessing that even though JimmXinu could answer this question best, I'm hoping there are a few other savants here who can answer it for me. In short, 2 months ago, the latest Calibre wouldn't install when downloaded. I know they fixed it after a week or so, but because the standard assumption is that "The problem must have been because of something I did", I made the stupid choice of fully uninstalling and removing all program data for Calibre and tried again, and again, and again. Until they fixed whatever the problem was. Although, as a precaution, I ZIPped up all my Calibre libraries first for safe keeping, wrongly assuming all the library data was contained in those folders. (can you see the second problem I made for myself?) Anyways, I'm finally getting back into using Calibre again, mostly because Fanfiction.net has changed something yesterday (Dec 19th) and the FanFictionDownloader program (by R. Eisele at fanfictiondownloader.net) doesn't work anymore, so I'm now needing to rely (again) on FanFicFare. So, my question. Does FanFicFare work "right outta the box" ? Or do I need to set up things to make it work. I used to use it and first installed it back in 2017(?) and can't remember if there was any specific setup to do with it. I exclusively get all my FanFic from FanFiction.net, if that helps any. Sorry to bother you folks with what is probably a dumb question. The only reason I ask is that I've tried several times yesterday, both with single story URLs and muti-story groups of URLs and they don't work. Of course, it's possible that whatever change FF.net made, it's screwed up ALL the FF downloading methods. Help? Frank P.S. As for the second problem, that you can't just create libraries and copy all the old data back into it, I'm at a loss as to how to restore my Calibre libraries. The only library that works is my default library that I was in and had been using when I uninstalled Calibre. Any ideas on how to recreate the other libraries I had? As I said, I still have the libraries still zipped up. But apparently that's not enough... Last edited by fdwojo; 12-20-2020 at 11:44 AM.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
[GUI Plugin] KindleUnpack - The Plugin	DiapDealer	Plugins	523	07-15-2025 06:45 PM
[GUI Plugin] Open With	kiwidude	Plugins	404	02-21-2025 05:42 AM
[GUI Plugin] Marvin XD	Philantrop	Plugins	126	01-29-2017 12:48 PM
[GUI Plugin] KiNotes	-axel-	Plugins	0	07-14-2013 06:39 PM
[GUI Plugin] Plugin Updater Deprecated	kiwidude	Plugins	159	06-19-2011 12:27 PM

12-20-2020, 06:02 AM	#4968
fanfictioneer Enthusiast Posts: 26 Karma: 110 Join Date: Oct 2018 Device: Kindle	FYI, I just managed to download some stories by adding my browsers (Vivaldi, Chrome fork) user agent header when retrieving from fanfiction.net.

12-20-2020, 06:23 AM	#4970
icecold Enthusiast Posts: 45 Karma: 10 Join Date: Mar 2020 Device: Android,	Thank you, setting my user_agent to Mozilla/5.0 also worked for me and I'm using Vivaldi.

12-20-2020, 11:31 AM	#4975
JimmXinu Plugin Developer Posts: 6,981 Karma: 4604635 Join Date: Dec 2011 Location: Midwest USA Device: Kobo Clara Colour running KOReader	FFnet has made changes over the weekend that broke FFF for everybody. I hope to be able to fix it, but there are no guarantees. To the best of my knowledge, if you saved everything in the libary directory, unzipping it as a directory and then adding that dir to Calibre under Library -> Switch/create Library should work. Worst case, you add all the saved ebook files to a new library. But that is as much as I know about it. You'll need to use your google-fu or ask over in the Library Management sub-forum for more help on libraries.

12-20-2020, 11:54 AM	#4976
fdwojo Member Posts: 23 Karma: 10 Join Date: Jun 2012 Device: Samsung Note20 Ultra, Kindle Fire HD10	What I did was I went into Calibre (32bit, by the way), and first recreated all the libraries without any books in them. Then, I unzipped all those other libraries back into their original folders. However, when I start instruct Calibre to look at one of those other libraries, it still shows as being empty with no books, even though I look in that library folder and all the author folders with their books are still there. (When I originally zipped my libraries up, I choose to maintain the directory tree structure with all library folders, author folders, and book folders contained within them just like before when it was working fine before I uninstalled it. And, JimmXinu, does FanFicFare require any setup to be used, or can you just paste in URL's and it'll work without any setup? Frank P.S. As much as I am trying to at least be able to see my books, I'm also worried with regard to the custom columns I created for the FanFic I used your plugin for. I had to create 4 or 5 custom folders for specific metadata and I'm concerned I'll have to recreate those custom columns from scratch...

12-20-2020, 08:42 PM	#4978
JimmXinu Plugin Developer Posts: 6,981 Karma: 4604635 Join Date: Dec 2011 Location: Midwest USA Device: Kobo Clara Colour running KOReader	FYI: Yes, ffnet support is still broken. The only fix I've found so far would involve packaging at least three more modules in the FFF plugin, one of which(cloudscraper) is Python3 only and would (theoretically) only work in Calibre 5+. They (ffnet) are also still actively fixing issues with their migration, so I'm not rushing while it's still a moving target. Further discussion in github issue.

12-21-2020, 02:12 AM	#4980
ownedbycats Custom User Title Posts: 11,003 Karma: 75337983 Join Date: Oct 2018 Location: Canada Device: Kobo Libra H2O, formerly Aura HD	I speculated on the GitHub thread that the differences in results is due to location. If certain ISPS/IP ranges are known for having bad actors, it's plausible that Cloudflare applies more stringent measures to them. I'm not sure if this is actually done, but it lines up with reports of users from certain countries having difficulty accessing the website.