[GUI Plugin] FanFicFare

[aleyx]

Originally Posted by midnightmane

Okay let me put some context I'm not massively updating, over the span of a week i have only updated 60 stories. but I was wondering is if there was a way I could set it up on the schedule So that I can spread out my update over the course of days To go easy on the web servers I already know this is strongly discouraged but that's why I'm looking for options on how to update my library without putting strain or hassle on the servers.

There's no scheduling feature that I'm aware of. You could configure a default tweak_fg_sleep for each site to a large value, although others would need to confirm and propose proper values; I'm not familiar with that setting.

You would still have to manually manage batches of books though, if for no other reason that once you've launched the batch, you can't interrupt it without wasting that round, and you have to validate at the end anyways.

If your library is that huge (> 5k books? >6k? ) my advice would be to plan beforehand.

Audit your library. Tag books you actually read; there's no point updating books you forgot you even had, or completed books. The point is to read and enjoy books, not to have the most up-to-date library possible (unless you're an archivist). Create a virtual library based on that tag, and work only on that library.

Once you have a core of books you actually want to read, make sure you have mail notifications enabled for them, and tag them with something like "Mail-managed" so you know you won't have to deal with them manually anymore afterwards.

Now you can start updating this core list. For bonus, you can rate them beforehand and update them in order of rating (you can create a custom column for that if you want). Update by batches of, say, five. One those five are updated, you can untag them to remove them from your virtual library (knowing that you'll be notified from now on).

Start a second batch as a reading buffer but, and here's the trick: if you decided to rate them, don't start another batch until you've finished reading the first one. Because if you didn't read the best batch you won't read the next, and updating a book you won't read is a waste of work.


I understand the natural instinct of the completist: MUST. HAVE. LATEST.

I have to fight this instinct too. But while updating is satisfying in itself, the point is to read something you enjoy. Concentrate on that!


Additionally, you can try to use a custom column to calculate the approximate lateness of an update:

Spoiler:

program:
# Days since the last update
days_since_last=days_between(today(), raw_field('#updated'));

# The first chapter doesn't count in the average; nor do books with no chapters (since they generate a divide by zero error)
chapters=cmp(raw_field('#chapters'), 2, 1, subtract(raw_field('#chapters'), 1), subtract(raw_field('#chapters'), 1));

# Days between chapters, in average.  If the average is less than one day, we up to one day.  This is so incomplete multi-chapters uploaded in one go don't get crazy values.
pub_length=days_between(raw_field('#updated'), raw_field('pubdate'));
avg=divide(cmp(pub_length, 0, 1, 1, pub_length), chapters);
avg=cmp(avg, 1, 1, 1, avg);
# Number of days an update should have come up, based on average
days_delayed=subtract(days_since_last, avg);

# Number of 'deadlines' missed.
deadlines=divide(days_since_last, avg);

# We only display a value for books with a Last Updated value. Among those, we dismiss completed books.
# You'd think a date column with an undefined value would be considered empty by ifempty(), but nope.  It's a column with a 'None' value.
contains( raw_field('#updated'), 'None', '',
	str_in_list(field('tags'), ",", "Completed", "",
		cmp( deadlines, 2,
		"", "", finish_formatting( deadlines, "0.0f", "x" , finish_formatting(days_delayed, "0.0f", " (", finish_formatting(avg, "0.0f", ", avg ", ")")))
		)
	)
)

The more it's late, the more likely it's been dropped anyways.

(I know Calibre introduced new functions since I wrote that code; I haven't had time to rewrite it yet)

N.

[ownedbycats]

Originally Posted by aleyx

Once you have a core of books you actually want to read, make sure you have mail notifications enabled for them, and tag them with something like "Mail-managed" so you know you won't have to deal with them manually anymore afterwards.

I have a bool (yes/no) "subscribed" column, and a rule to show an icon when it's unset.

I had something like 150 in-progresses to subscribe to. not fun. I did it over the course of a few weeks.

[ownedbycats]

Originally Posted by aleyx

Additionally, you can try to use a custom column to calculate the approximate lateness of an update:

Spoiler:

program:
# Days since the last update
days_since_last=days_between(today(), raw_field('#updated'));

# The first chapter doesn't count in the average; nor do books with no chapters (since they generate a divide by zero error)
chapters=cmp(raw_field('#chapters'), 2, 1, subtract(raw_field('#chapters'), 1), subtract(raw_field('#chapters'), 1));

# Days between chapters, in average.  If the average is less than one day, we up to one day.  This is so incomplete multi-chapters uploaded in one go don't get crazy values.
pub_length=days_between(raw_field('#updated'), raw_field('pubdate'));
avg=divide(cmp(pub_length, 0, 1, 1, pub_length), chapters);
avg=cmp(avg, 1, 1, 1, avg);
# Number of days an update should have come up, based on average
days_delayed=subtract(days_since_last, avg);

# Number of 'deadlines' missed.
deadlines=divide(days_since_last, avg);

# We only display a value for books with a Last Updated value. Among those, we dismiss completed books.
# You'd think a date column with an undefined value would be considered empty by ifempty(), but nope.  It's a column with a 'None' value.
contains( raw_field('#updated'), 'None', '',
	str_in_list(field('tags'), ",", "Completed", "",
		cmp( deadlines, 2,
		"", "", finish_formatting( deadlines, "0.0f", "x" , finish_formatting(days_delayed, "0.0f", " (", finish_formatting(avg, "0.0f", ", avg ", ")")))
		)
	)
)

The more it's late, the more likely it's been dropped anyways.

(I know Calibre introduced new functions since I wrote that code; I haven't had time to rewrite it yet)

N.

You do realize the user can set a "last checked" datetime column with fff? Might make things easier.

Personall I use Action Chains' single-field edit to change the status column to Abandoned if there's either a non-transient error or it's old enough.

Note my template specifically will only work on the last few versions of Calibre due to heavy use of field references.

program:
d1 = format_date(today(),'iso');
d2 = $$#fanficupdated;
status = $#fanficstatus;

	if
		and(
			status=='In-Progress',
			days_between(d1, d2) ># 730,
			)
	then
		"Abandoned"
		elif
			and(
				status=='In-Progress',
				$#fanficerror,
				)
		then
			"Abandoned"

	else
		status
	fi

[Science Friction]

I saw the question earlier but hadn't really thought of it in terms of WIP subs. Is this all from ffnet? I think ao3 is more download friendly, so it may be preferable to use ao3 for stories that are crossposted to there. An even better approach might be ff2ebook.com or some similar epub sharing scheme to cache the latest epub updates without hammering ffnet all the time. Then maybe FFF could (optionally) redirect ffnet requests to the shared cache if a snapshot is available.

I've been using Calibre to read Basilisk Born since the formatting in the text file is rather messed up, at least near the beginning. I've used it before but I like it a lot better this time. For fanfic I've traditionally been a throwback and preferred reading plain text files over web pages or epubs, but I'm starting to see the attraction of epubs. Basilisk Born is a great story for those of you into that sort of thing.

[aleyx]

Originally Posted by ownedbycats

You do realize the user can set a "last checked" datetime column with fff? Might make things easier.

It would absolutely be useful for manually checked books, certainly, because for them you can't always be certain of when you last checked for new updates.

For the vast majority of cases though, the hosting site probably has a mechanism for mail notifications. In those cases you know when you last checked, since you usually would batch-update directly from the mail notifications, and not check otherwise.

The column I've pasted tries to guess the publication frequency of the book and, from that and the date of last update, displays how many "deadlines" (and how many days) are missed. This is mainly so that you don't start checking every day for stories that only update every few weeks or months, or inversely so you know to check stories (or at least the website) that have a handful of missed deadlines.

N.

[midnightmane]

What is a good slow _sleep_time mine is currently 100

[JimmXinu]

It depends a lot on which sites you download from and your download habits.

As far as I know, most people get by fine with the default settings.

[ownedbycats]

I suspect this is due to my own practices, but I found that FFF has trouble detecting author changes if the url: identifier is missing.

Details:

I'd downloaded a oneshot from AO3. Since it was completed, I removed the url: identifier from the Calibre metadata and left just my custom ao3: identifier.

The author then orphaned the work.

I forgot I downloaded the fic and tried to redownload it. I didn't change the settings from the default "update EPUB if new chapters," but instead of skipping the story it instead downloaded a new one with author set to orphan_account (replace_metadata'd to 'Unknown').

I believe it'd been stated FFF had a third way to check for stories in-between checking for a url: identifier and using title/author match, so I am not sure if this is meant to happen.

It's probably a minor issue because most people probably just leave the url identifiers in there, but if it's worth looking into I've attached the duplicated ebooks + Calibre OPFs for investigation.

[JimmXinu]

Originally Posted by ownedbycats

I suspect this is due to my own practices, but I found that FFF has trouble detecting author changes if the url: identifier is missing.

Correct.

FFF looks for identifier url: (and uri: for historical reasons) and optionally title/author. If you've removed the identifier url: and the author has changed, it's not going to match anymore.

If there's a third way for FFF to match books, I don't remember it.

(Considering all the other things you record in various ways, I have to say; removing the identifier url: to indicate completed seems odd.)

You could, however, add such URLs to the reject list without removing the book from the library.

[ownedbycats]

Originally Posted by JimmXinu

(Considering all the other things you record in various ways, I have to say; removing the identifier url: to indicate completed seems odd.)

It's not so much that I remove it to indicate completed, but I just leave it on the in-progresses to make updates easier.

[Rei97]

Help, I can no longer download stories from quotev.com , on every story i get this error
Error
Unknown
Unknown
Story does not exist: (https://www.quotev.com/story/13855149)
https://www.quotev.com/story/13855149

I even get this error on the stories I have downloaded in the past, when I try to update them. but when I go to the story it is still there. Maybe they have changed something on their website?

[JimmXinu]

Originally Posted by Rei97

Help, I can no longer download stories from quotev.com , on every story i get this error
Error
Unknown
Unknown
Story does not exist: (https://www.quotev.com/story/13855149)
https://www.quotev.com/story/13855149

I even get this error on the stories I have downloaded in the past, when I try to update them. but when I go to the story it is still there. Maybe they have changed something on their website?

I'm able to download from quotev.com. Are you using a current version of FFF?

Run FFF with Calibre in debug mode (under Preferences dropdown) and post the debug log for further help.

[Rei97]

Originally Posted by JimmXinu

I'm able to download from quotev.com. Are you using a current version of FFF?

Run FFF with Calibre in debug mode (under Preferences dropdown) and post the debug log for further help.

I checked and I am using the latest version of FFF. Here is the debug log : https://pastebin.com/GgJjJDhd

I wonder if its one of my plugins that is interfering , or I messed up some settings ...

[JimmXinu]

New Test Version Posted

2021-05-23
- quotev.com: use_cloudscraper:true by default.
- Update translations

Rei97: The same error came up on the mailing list and setting use_cloudscraper:true for quotev.com fixed it. So I'm making that default.

Originally Posted by Rei97

I checked and I am using the latest version of FFF. Here is the debug log : https://pastebin.com/GgJjJDhd

I wonder if its one of my plugins that is interfering , or I messed up some settings ...

[Rei97]

Thank you so much !