[GUI Plugin] FanFicFare

[Nchalada]

Dang sites changing the way they fundamentally work!

[JimmXinu]

New Test Version Posted

2024-04-26
- adapter_literotica: Rewrite(mostly) for site changes. #1058
- adapter_literotica: Remove use_meta_keywords option. #1058

This also removes literotica features:

• averrating metadata entry - I haven't found a reasonably parsable version in the new pages.
• order_chapters_by_date option - New story series page doesn't give chapter dates.
• use_meta_keywords option - Unnecessary now, they're collected elsewhere

EDIT: Also adds cover image, but only for multi-chapter stories?

[ChaoticAdventure]

Hi! I searched the thread for this but didn't find anything, so I'm just wondering:
(for context all of this may only apply to ao3, it's the only site I use)
It seems that when I update anthologies, they don't call the custom columns in calibre the same way normal fics do. I use some tags that I add manually to manage cover settings, but with anthologies, those tags don't appear to do anything. I don't suppose there is a way to have that happen? I understand it may just not work because of how the anthology function works, or may be cumbersome to add, and I don't want to create tons of extra work for you.
As an example of what I mean: I would manually add the tag "BNHAMulti" to a fic after I upload it to calibre for the first time. That tags column does not update when the fic updates, so it won't get overwritten. Then, when I update the fic, calibre will automatically use the "BNHAMulti" cover thanks to my cover settings. With anthologies, that just doesn't happen, and I have to change the cover manually after updating.

I also have a second question. For anthologies, is there any way to call the "total word count" statistic? When I call the word count, it seems to call the individual word counts for each fic in the anthology, which is just not what I'm looking for. I'm happy to keep updating those manually, but if the function already exists and I just can't find it, I would love to know.
Thanks in advance.

[JimmXinu]

Originally Posted by ChaoticAdventure

... Then, when I update the fic, calibre will automatically use the "BNHAMulti" cover thanks to my cover settings. With anthologies, that just doesn't happen, and I have to change the cover manually after updating.

Have you seen this section of the cover settings wiki page?

Anthology cover settings are a bit different and more limited, I suspect that's what's happening.

Originally Posted by ChaoticAdventure

I also have a second question. For anthologies, is there any way to call the "total word count" statistic? When I call the word count, it seems to call the individual word counts for each fic in the anthology, which is just not what I'm looking for. I'm happy to keep updating those manually, but if the function already exists and I just can't find it, I would love to know.

What do you mean by "call the word count"? Because the anthology feature automatically sums the numWords values for each book together before setting custom columns for anthologies.

[ChaoticAdventure]

Thank you for linking me to the wiki. I always feel a bit silly when that happens but for some reason, I assumed it worked a certain way when it just... doesn't, as shown in the wiki lol. I might be able to find a workaround there with the forced option but for now I think I'll just need to keep doing them manually.

As for the word counts, the anthology does sum them up correctly for the "word count" column, as it should. I have a separate column that I use to show the "word count range" of a fic, which uses the "numWords" function and a conditional to calculate range. It works great for individual fics, but with anthologies it will output [<10k, 10k-25k, 100k-150k] instead of the summed range (which might be 500k-600k, for example). It's a fairly specific issue so if there isn't a way to solve it, no problem, I can continue to input manually.

[kamille3]

Used the latest test version for literotica and noticed a few things.

Certain authors (like authors/Naughtybecca/works/stories) have a lot of stories in their profile which adds a new "Show more" button on the website. I think FFF for now can't go beyond this page thus misses a lot of the stories.

Also when I try to download from this author's page- authors/somethingfar/works/stories

I'm getting this error for the series California Dreamin' - 'NoneType' object has no attribute 'group'

Another thing I see is that when I try to update existing books for authors I previously downloaded, FFF tries to redownload the story series even though there has been no update to the series.

Also is it possible to remove the completed tag? I just want to keep the genre tag like SciFi, Romance, etc otherwise it makes my library look a bit messy. Thanks.

[JimmXinu]

Originally Posted by kamille3

Certain authors (like authors/Naughtybecca/works/stories) have a lot of stories in their profile which adds a new "Show more" button on the website. I think FFF for now can't go beyond this page thus misses a lot of the stories.

FFF isn't reading the author page anymore, so that shouldn't matter. You just give FFF a story (or 'chapter') URL.

Originally Posted by kamille3

I'm getting this error for the series California Dreamin' - 'NoneType' object has no attribute 'group'

All the 'Story Series' URLs I saw before had /series/se/99999, but apparently they can also be alphanumeric.

Originally Posted by kamille3

Another thing I see is that when I try to update existing books for authors I previously downloaded, FFF tries to redownload the story series even though there has been no update to the series.

This is a significant site change. For many (all multi-chapter) stories the 'canonical' story URL is changing, which may cause that depending on what you're doing.

The individual chapter URLs changing could also have that effect. I honestly don't know if the chapter URLs changed or not--I don't have any old epubs for this site.

Short answer: I'm not surprised that's happening. I'd have to see your 'before' epubs to know why exactly, and even then it may not be avoidable.

Besides, you're only updating when there's new chapters, not attempting updates on lots 'just in case', right?

Originally Posted by kamille3

Also is it possible to remove the completed tag? I just want to keep the genre tag like SciFi, Romance, etc otherwise it makes my library look a bit messy. Thanks.

The include_subject_tags setting includes status in Calibre's Tags column by default. It's just now the site has a status for FFF to collect.

Search this thread for past discussions on changing include_subject_tags to customize it.

[JimmXinu]

New Test Version Posted

2024-04-26
- adapter_literotica: Rewrite(mostly) for site changes. #1058
- adapter_literotica: Remove use_meta_keywords option. #1058
- adapter_literotica: Allow /series/se/alphanumeric instead of just numeric

This also removes literotica features:

• averrating metadata entry - I haven't found a reasonably parsable version in the new pages.
• order_chapters_by_date option - New story series page doesn't give chapter dates.
• use_meta_keywords option - Unnecessary now, they're collected elsewhere

In exchange it adds:

• Cover images, but only for multi-chapter stories?
• status metadata is collected

[JimmXinu]

Originally Posted by ChaoticAdventure

As for the word counts, the anthology does sum them up correctly for the "word count" column, as it should.

Hurray!

Oh wait, there's more.

Originally Posted by ChaoticAdventure

I have a separate column that I use to show the "word count range" of a fic, which uses the "numWords" function and a conditional to calculate range. It works great for individual fics, but with anthologies it will output [<10k, 10k-25k, 100k-150k] instead of the summed range (which might be 500k-600k, for example). It's a fairly specific issue so if there isn't a way to solve it, no problem, I can continue to input manually.

I'm not finding a Calibre numWords() function, so I assume you have defined your own numWords() function as a Calibre template language function? I base that on the use of the word 'function'.

(Which is different than FFF's numWords metadata entry.)

I don't know much of anything about Calibre template language, honestly, I can't help there.

Going in entirely the opposite direction, if you've magicked FFF's replace_metadata system into making word ranges for you, FFF is behaving as designed--replace_metadata, et al only operate on individual stories, not on anthologies. The correct answer would be a list of different ranges like that.

In that case, I'd suggest looking at using a Calibre Custom column 'built from other columns' that operates on a read word count column using Calibre template language.

Then it would affect all books, not just FFF downloads.

[kamille3]

FFF isn't reading the author page anymore, so that shouldn't matter. You just give FFF a story (or 'chapter') URL.

I don't get it. I pasted author page url into FFF and it didn't grab all the stories for me.

When I manually clicked show more button on the author to reveal all the story urls and then extracted the story links with a link grabber extension and pasted it into FFF, a few of the series downloaded as separate chapters not as one whole story with multiple chapters.

This is a significant site change. For many (all multi-chapter) stories the 'canonical' story URL is changing, which may cause that depending on what you're doing.

The individual chapter URLs changing could also have that effect. I honestly don't know if the chapter URLs changed or not--I don't have any old epubs for this site.

Short answer: I'm not surprised that's happening. I'd have to see your 'before' epubs to know why exactly, and even then it may not be avoidable.

I guess FFF is redownloading because in the earlier versions the main url for a story series used to be the url of the first chapter of the series but now they are using a separate series /se/ url.

Besides, you're only updating when there's new chapters, not attempting updates on lots 'just in case', right?

Yes, I was just checking if things were working same as before or not.

Also, I notice that now story series include multiple genres in its tags while in the previous versions only the genre of the first chapter of the series was used. Is this the intended behavior?

And again thanks for the all the fixes.

[applegaa]

Originally Posted by JimmXinu

New Test Version Posted

2024-04-26
- adapter_literotica: Rewrite(mostly) for site changes. #1058
- adapter_literotica: Remove use_meta_keywords option. #1058
- adapter_literotica: Allow /series/se/alphanumeric instead of just numeric

This also removes literotica features:

• averrating metadata entry - I haven't found a reasonably parsable version in the new pages.
• order_chapters_by_date option - New story series page doesn't give chapter dates.
• use_meta_keywords option - Unnecessary now, they're collected elsewhere

In exchange it adds:

• Cover images, but only for multi-chapter stories?
• status metadata is collected

I noticed that on some stories with multiple chapters, it tells me that the epub has x chapters while the web site has 1. This isn't true, as the web site has the same x chapters. I tried manually changing the Story URL in the metadata, but that had no effect. However, I found that changing the URL in Ids from the first chapter to the series resolved this and it was able to find all of the chapters again.

I finally figured out that if the chapter listed as the Ids URL is listed both in a series and individually, it picks the not in a series version and thus treats it as a single chapter story. Changing the Ids to match the series bypasses this bug in the listing.

I'll PM an example URL.

- Andrew

[JimmXinu]

Originally Posted by kamille3

I don't get it. I pasted author page url into FFF and it didn't grab all the stories for me.

You mean 'Get Story URLs from Web Page'?

That's as good as it's going to get. That feature can only read the HTML on the page as downloaded.

You can drop/drop or copy/paste selected blocks of a page into FFF's Download from URL dialog, however, and FFF will pick out the rendered URLs.

Originally Posted by kamille3

When I manually clicked show more button on the author to reveal all the story urls and then extracted the story links with a link grabber extension and pasted it into FFF, a few of the series downloaded as separate chapters not as one whole story with multiple chapters.

I generally can't address what I don't see. I need examples. Either PM, partial URLs like before, or in the github issue.

Originally Posted by kamille3

Also, I notice that now story series include multiple genres in its tags while in the previous versions only the genre of the first chapter of the series was used. Is this the intended behavior?

Yes. FFF is now using the /series/se page as the definitive data source. Since all 'chapter' category values are now visible at once, FFF is collecting them.

[JimmXinu]

Originally Posted by applegaa

I noticed that on some stories with multiple chapters, it tells me that the epub has x chapters while the web site has 1. This isn't true, as the web site has the same x chapters. I tried manually changing the Story URL in the metadata, but that had no effect. However, I found that changing the URL in Ids from the first chapter to the series resolved this and it was able to find all of the chapters again.

I finally figured out that if the chapter listed as the Ids URL is listed both in a series and individually, it picks the not in a series version and thus treats it as a single chapter story. Changing the Ids to match the series bypasses this bug in the listing.

I'll PM an example URL.

I've looked at that example. When you click that 'story' link, the page you get doesn't have a 'Story Series' link on it. Therefore, FFF has no way to know it's part of a series.

Honestly, that's the site being broken, IMO. We've also seen examples of the same 'chapter' in two 'story series'.

And before anyone suggests going back to parsing the author page, just no. There's too much hidden behind javascript that I'm just not dealing with.

Fortunately, in this case at least, the answer is simple. Use the /series/se URL instead.

[ChaoticAdventure]

Originally Posted by JimmXinu

In that case, I'd suggest looking at using a Calibre Custom column 'built from other columns' that operates on a read word count column using Calibre template language.

Then it would affect all books, not just FFF downloads.

I had never delved into columns built from other columns before now, it seemed too complicated for me, but it worked a treat once I spent some time figuring it out. Luckily I had done all the heavy lifting already while trying to get FFF to do it, so all I had to do was transfer it over into the template format. Thank you for your patience and knowledge!

[ownedbycats]

On AO3, I posted a story in an anonymous collection. For most users, this displays the username as "Anonymous", but for either the story author or collection owner, it displays "Anonymous [Username]"

Out of curious to see what would happen, I then enabled always_login:true and when I downloaded it it was just my username.

Here's the author byline of an anonymized story:

<h3 class="byline heading">
Anonymous [<a rel="author" href="/users/MyUsername/pseuds/MyUsername">MyUsername</a>]
</h3>

And non-anonymized:

<h3 class="byline heading">
<a rel="author" href="/users/MyUsername/pseuds/MyUsername">MyUsername</a>
</h3>

Ive not looked at the adapter code but I'm guessing it's only capturing the part inside the square brackets.

Definitely a corner case here, but thought it was worth documenting in the unlikely case that something else was intended.