[GUI Plugin] FanFicFare

[JimmXinu]

I don't have any better ideas than combining the different entries together and then cleaning.

I have said before:

Originally Posted by JimmXinu

Also, I'll tell you the same thing I tell everyone trying to normalize AO3 metadata: it's going to be a lot of work and it's never going to be perfect as long as AO3 allows authors to enter whatever they want.

If AO3 had a mode that displayed the 'canonical' tags instead we could at least leverage the work already done by AO3 'tag wranglers' team. (And no, looking up each tag to find the canonical version isn't feasible; authors routinely put tens or even hundreds of tags on a story.)

And you are trying to go another step beyond normalization.

[ownedbycats]

Originally Posted by JimmXinu

Most metadata entries are populated during the 'get metadata' phase. cover_image can be filled much later, depending on the value.

As currently coded, cover_image is being filled after covertype has already been cached as an empty list. And the metadata cache does not trace back through include_in_* settings.

IE, cover_image is uncached when set, but not entries derived from it, whether from include_in_* or replace_metadata conditionals. Most of the time, it doesn't matter; and the cache saves significant time for users with extensive metadata editing. Until now, I would have said all the time.

I am considering whether this corner case is worth addressing or not.

This feels a bit familiar. Was there a similar case before?

I believed I asked once before if FFF could populate the column, but then it was more of a theoretical idea. This time I re-doing all my covers (various adjustments to the GC profiles over the years made for a lot of mismatched covers) so it seemed like a good time to set up the column.

[dailyalice]

I've generally created a column type of "Text, but with a fixed set of permitted values" for use as a status column with the permitted values being "Completed,In-Progress,Abandoned,Unknown" and that's worked well for me. I set "Abandoned" manually usually for some works and I've set "Unknown" as the "Default Value" for cases where an archive doesn't give me a status.

I've been assuming most (all?) sites give a status of Completed or In-Progress or that FanFicFare interprets things that way, but wanted to make sure I had this right. Are there other statuses supported for other sites? I've mostly been focused on work with AO3 lately so don't have a lot of experience.

I'm just wondering if there's any reason I should use a different column type (or offer it as an option in that FanFicFare/calibre guide I'm working on). Would a text column or tag column offer more flexibility? I mean, obviously you could then add your own data, but I'm wondering about what data archives typically provide and what FanFicFare accounts for.

(I've searched this thread a bit but didn't have much luck finding info, which may entirely be search failure on my part.)

Attached is a screenshot of how I usually set up my status column.

[JimmXinu]

Originally Posted by dailyalice

I've been assuming most (all?) sites give a status of Completed or In-Progress or that FanFicFare interprets things that way, but wanted to make sure I had this right. Are there other statuses supported for other sites? I've mostly been focused on work with AO3 lately so don't have a lot of experience.

I try to standardized the values 'Completed' and 'In-Progress' for all sites, and those two cover the vast majority.

But there are sites that have additional states. Abandoned, Paused, Cancelled, etc. I don't have a complete list because some adapters use the site text directly.

Unless it's important to you to limit the values, I'd used a free form text field instead.

[dailyalice]

Originally Posted by JimmXinu

I try to standardized the values 'Completed' and 'In-Progress' for all sites, and those two cover the vast majority.

But there are sites that have additional states. Abandoned, Paused, Cancelled, etc. I don't have a complete list because some adapters use the site text directly.

Unless it's important to you to limit the values, I'd used a free form text field instead.

Thank you! I thought this might be the case, but wanted to be sure.

I actually have the following bit in the status column section of my guide already from when I was working on it a month ago:

An alternate method to use, which might be good if you know you'll be downloading from multiple archives and think they may have statuses available beyond "Complete" and "In-Progress," would be to go with a simple column like this:

This has no fixed values, when I used this column type with the works I downloaded from AO3, they showed up as "Complete" and "In-Progress." I set a default value of "Unknown" to use when an archive doesn't provide any sort of value for status.

This method probably future proofs the column for if you start downloading from a site that offers more status items or different ones than AO3. You won't have color coded words, but you will have whatever words an archive provides. (Whereas with a fixed set of values, those are the only available options.)

And attached is the screenshot of the column.

I'm working on a major revision to the guide (maybe one day I'll finish it) and I think I'll just use this as the status column suggestion and not bother with the fixed set of values thing. Unless, I suppose, I wanted to demonstrate that column type for some reason. Hmm.

(The guide so far has an AO3 focus, though of course much of it applies elsewhere.)

[JimmXinu]

Personally, I don't bother with a status column.

By default, FFF lists status in the include_subject_tags setting, so it gets put in the Tags column.

[dailyalice]

Originally Posted by JimmXinu

Personally, I don't bother with a status column.

By default, FFF lists status in the include_subject_tags setting, so it gets put in the Tags column.

I know you like putting a lot of data in the Tags column, I tend to prefer having things in separate columns and having a very limited selection in actual calibre Tags. I may be weird that way (among other ways). (One thing that is making this guide interesting is attempting to document the different ways people might handle this stuff. I only cover a few, of course, or I would be working on this guide for the rest of my life probably.)

Which reminds me . . . is there a way to prevent FanFicFare from putting status in Tags when you make Anthologies? I'm currently running a set up where just about the only thing I put in the calibre Tags field is "Anthology" for anthologies and I like that functionality, but could do without the status info.

[JimmXinu]

Originally Posted by dailyalice

Which reminds me . . . is there a way to prevent FanFicFare from putting status in Tags when you make Anthologies? I'm currently running a set up where just about the only thing I put in the calibre Tags field is "Anthology" for anthologies and I like that functionality, but could do without the status info.

'Completed' and 'In-Progress' are removed from the list of tags an Anthology inherits from its contained books.

But some sites (notably AO3) also have a 'Complete: yes/no' for series and that is hard coded to be put into Tags and status for the merged anthology.

[dailyalice]

Originally Posted by JimmXinu

'Completed' and 'In-Progress' are removed from the list of tags an Anthology inherits from its contained books.

But some sites (notably AO3) also have a 'Complete: yes/no' for series and that is hard coded to be put into Tags and status for the merged anthology.

Gotcha. Okay. I appreciate the AO3 series status going into status, I'll just continue to remove it from Tags. (It confused me for awhile that I was still seeing status type info in Tags until I realized it was just associated with anthologies, I figured it had something to do with how they work.)

[ownedbycats]

Originally Posted by dailyalice

Gotcha. Okay. I appreciate the AO3 series status going into status, I'll just continue to remove it from Tags. (It confused me for awhile that I was still seeing status type info in Tags until I realized it was just associated with anthologies, I figured it had something to do with how they work.)

Use Action Chains' single-field edit to run this template on tags:

program:

	if 
		$#fanficstatus == 'Anthology' 
	then
		cleaned_tags = list_difference(
			$tags,
			'In-Progress, Complete',
			',')
	else
		$tags
	fi

EDIT: Slightly modified version for when 'Anthology' is in tags.

program:

	if 
		'Anthology' in $tags
	then
		cleaned_tags = list_difference(
			$tags,
			'In-Progress, Complete',
			',')
	else
		$tags
	fi

[ownedbycats]

As seen in this action chain, I use a lot of templates to adjust metadata beyond what FFF can do internally. (Note that this is just Calibre metadata; the eBooks themselves need to be edited manually.) Here's two more that I think may be of use to @dailyalice or other users.

When run on #fanficcat (fandom column), this will remove a "crossover" category (or whatever is added by add_category_when_multi_category) from specific sets of categories.

program:
	f =	list_sort($#fanficcat,0,',');

	if 
## As this checks the sorted string, make sure it's alphabetical order (e.g. 'bar, Crossover, foo')
		contains(f, '^Crossover, Mass Effect Trilogy, Mass Effect: Andromeda$', '1', '')
	then 
		'Mass Effect Trilogy, Mass Effect: Andromeda' 

	elif
		contains(f, '^Crossover, Half-Life, Portal$', '1', '')	
	then 
		'Half-Life, Portal' 

	else
		$#fanficcat
	fi

When run on #fanficstatus (status column), this sets the value to 'Abandoned' if there's an error (check first to make sure it isn't transient!) and to 'Dormant' if it hasn't been updated in the last year.

program:
	now = format_date(today(),'iso');
	updated = $$#fanficupdated;
	status = $#fanficstatus;

	if 
		status=='In-Progress' && $#fanficerror
	then 
		'Abandoned'
		

	elif
		status=='In-Progress' && days_between(now, updated) ># 365
	then 
		'Dormant'

	else 
		status
	fi

[ownedbycats]

An AO3 fanfic (PM'd the link because of story rating) has an embedded gifv image (thankfully not mature-rated), which ended up as a "failedtoload." While I'm just inclined to remove it—I'm not even sure if ePubs can handle animations—is it fixable?

[JimmXinu]

Googling gifv reveals it is a video format, not just an image format. And I'm honestly a bit surprised that the FFF image processing handles it without choking.

Downloading that story, several other gifv files from media.tumblr.com download fine, just not the last one, which fails with server error 500.

There's no apparent difference, and trying again several minutes later without downloading the previous chapter images again succeeds.

So I conclude there's a limit being imposed by media.tumblr.com. You could try setting a higher slow_down_sleep_time, but there's no guarantee.

I'd try deleting that chapter and updating after some time has past.

[dailyalice]

Thanks for these action chains, @ownedbycats! I've never used action chains before, but the stuff in both these posts looks very promising.

[ownedbycats]

Originally Posted by JimmXinu

Googling gifv reveals it is a video format, not just an image format. And I'm honestly a bit surprised that the FFF image processing handles it without choking.

Downloading that story, several other gifv files from media.tumblr.com download fine, just not the last one, which fails with server error 500.

There's no apparent difference, and trying again several minutes later without downloading the previous chapter images again succeeds.

So I conclude there's a limit being imposed by media.tumblr.com. You could try setting a higher slow_down_sleep_time, but there's no guarantee.

I'd try deleting that chapter and updating after some time has past.

That worked. Thank you.