[GUI Plugin] FanFicFare

[ownedbycats]

replace_metadatas support regular expressions, which should help. Unfortunately, I can't really help anymore than this - my few attempts didn't really work reliably

[FandomWitch]

Thanks anyway. I have tried experimenting with various regex to try to make it work. But I don't have enough experience/knowledge to make it work...

Any tips or directionality would be much appreciated. I don't mind messing around myself to try to make it work, I'm just out of my depth with no idea where to go.

[ownedbycats]

I got it roughly working. Play around with it here: https://regex101.com/r/rVliD2/1

I think this would be:

ships=>((\d+)th Doctor)(/| &amp; )(.*)=>\g<4>\g<3>\g<1>

I couldn't entirely account for a third character - you can see that "Rose Tyler & Jack Harkness" is actually one capture group. I think there's ways around this (greedy regexes, maybe) but it's a bit past my knowledge.

Also, you might need to change the 'th' part to something like (rd|th) for other Doctors.

[JimmXinu]

Originally Posted by FandomWitch

Okay I have searched the forum and haven't been able to find a solution. And my basic knowledge of regex and experimentation hasn't worked. Is there a way to have a specific character always be the first one listed in a ship. I would prefer to not have to use alphabetical, like really prefer not to.

So have it do something like this, but not have to write out every possible ship.

 ships=>9th Doctor & Rose Tyler & Jack Harkness=>Rose Tyler & 9th Doctor & Jack Harkness
 ships=>9th Doctor/Rose Tyler=>Rose Tyler/9th Doctor
 ships=>10th Doctor/Rose Tyler=>Rose Tyler/10th Doctor

Basically, move the character I want to the front and shift the rest or sort those alphabetically.

This feels really familiar. I'm sure I've done something similar before, but I don't immediately find it either.

Detecting and moving one name out of a list to the front isn't difficult. Doing it while preserving the delimiters between names when the target name can be first, middle or last is a little trickier.

Here's what I've come up with using named regexp grouping for clarity:

 # assume leading delim, otherwise, already first
 ships=>^(?P<pre>.*?)(?P<delim>[ ]*(&amp;|/)[ ]*)(?P<first>Rose Tyler)(?P<post>.*)$=>\g<first>\g<delim>\g<pre>\g<post>

1. This assumes that you know what delimiters can exist between ships. [ ]*(&|/)[ ]* should be pretty universal.
2. This also assumes that each ship will only have one kind of delimiter--eg, you won't have A & B / C in one ships entry.
3. As the comment says, the regexp assumes a delimiter before the desired first name, otherwise it's already first and no change is needed.
4. Everything before the first delimiter and the desired first name is grouped in pre, the delimiter in delim, desired first name in first, and everything after (including delimiters) in post.
5. Rearrange to be first, delim, pre, then post.
6. pre is assumed to always exist, but post could be empty. If it's not, it starts with a delimiter.
7. sort_ships is applied before replace_metadata, so you can do both if you want.

If you get the wild idea to apply more than one of these, remember that replace_metadata lines are applied in order, which in this case means the last line 'wins'. See INI-File wiki and add_to_keyword in particular if you try to get fancy with [defaults] vs different sites.

Example:

 ships=>^(?P<pre>.*?)(?P<delim>[ ]*(&amp;|/)[ ]*)(?P<first>Rose Tyler)(?P<post>.*)$=>\g<first>\g<delim>\g<pre>\g<post>
 ships=>^(?P<pre>.*?)(?P<delim>[ ]*(&amp;|/)[ ]*)(?P<first>Jack Harkness)(?P<post>.*)$=>\g<first>\g<delim>\g<pre>\g<post>
# Jack ends up before Rose when both appear.

[ownedbycats]

For #2, AO3 doesn't mix ship-types/delimiters. I can't say about other sites, though.

[FandomWitch]

Oh my god. Thank you so much Jimm.

My brain doesn't like it when ships don't follow a certain pattern and this is amazing. And so easy to adapt to other characters, which requires another thank you.

Also thank you for the explanation of what the pieces are and the reminder that it reads top down. Hopefully this amazing guide helps others too.

[Handers]

Thanks for your answer.

Originally Posted by JimmXinu

I've looked at that a couple times and thrown my hands up each time. The forum paradigm (without threadmarks or an index post) is significantly different from the story & chapters paradigm FanFicFare uses. Should posts be chapters? Or pages?

I believe that each threadmark should delimitate a chapter, with any post coming after without a threadmark simply added to the chapter in progress.

Originally Posted by JimmXinu

And then there's the issue of over-use; because inevitably someone(s) would decide to use it to archive the entirety of all their favourite long discussion threads. If enough people start doing that, it's likely to use significant server resources, drawing attention from the admins and possibly prompting them to block the tool.

Do you think that an option to download all the post made by the author would be a viable strategy to mitigate any abuse?

[JimmXinu]

Originally Posted by Handers

I believe that each threadmark should delimitate a chapter, with any post coming after without a threadmark simply added to the chapter in progress.

And threads with no threadmarks at all? Or semi-random threadmarks?

Take for example, the current Harry and the Shipgirls thread--something I wish I could download myself:

• The original poster(OP) hasn't added a new threadmark in over a year. Can't use threadmarks for chapter beginnings.
• There are multiple posting 'authors'. Some posts are considered 'canon', some side story but canon, and some 'omake'. Can't collect OP posts only.
• The OP posts non-story comments. Can't collect all OP posts.

Originally Posted by Handers

Do you think that an option to download all the post made by the author would be a viable strategy to mitigate any abuse?

Not really. Because it wouldn't actually reduce the number of HTTP requests--we'd still need to download every page, then scan every post and discard non-OP posts.

Anything that could work for more than a few specific cases would end up being 'download the entire thread' in some form. Which I don't want to do because of server load--the potential for abuse is really just 'more server load', not a separate issue.

[JimmXinu]

New Test Version Posted

2023-06-29
- Fix for specific cover error.

[ownedbycats]

Originally Posted by JimmXinu

This feels really familiar. I'm sure I've done something similar before, but I don't immediately find it either.

Detecting and moving one name out of a list to the front isn't difficult. Doing it while preserving the delimiters between names when the target name can be first, middle or last is a little trickier.

Here's what I've come up with using named regexp grouping for clarity:

 # assume leading delim, otherwise, already first
 ships=>^(?P<pre>.*?)(?P<delim>[ ]*(&amp;|/)[ ]*)(?P<first>Rose Tyler)(?P<post>.*)$=>\g<first>\g<delim>\g<pre>\g<post>

1. This assumes that you know what delimiters can exist between ships. [ ]*(&|/)[ ]* should be pretty universal.
2. This also assumes that each ship will only have one kind of delimiter--eg, you won't have A & B / C in one ships entry.
3. As the comment says, the regexp assumes a delimiter before the desired first name, otherwise it's already first and no change is needed.
4. Everything before the first delimiter and the desired first name is grouped in pre, the delimiter in delim, desired first name in first, and everything after (including delimiters) in post.
5. Rearrange to be first, delim, pre, then post.
6. pre is assumed to always exist, but post could be empty. If it's not, it starts with a delimiter.
7. sort_ships is applied before replace_metadata, so you can do both if you want.

If you get the wild idea to apply more than one of these, remember that replace_metadata lines are applied in order, which in this case means the last line 'wins'. See INI-File wiki and add_to_keyword in particular if you try to get fancy with [defaults] vs different sites.

Example:

 ships=>^(?P<pre>.*?)(?P<delim>[ ]*(&amp;|/)[ ]*)(?P<first>Rose Tyler)(?P<post>.*)$=>\g<first>\g<delim>\g<pre>\g<post>
 ships=>^(?P<pre>.*?)(?P<delim>[ ]*(&amp;|/)[ ]*)(?P<first>Jack Harkness)(?P<post>.*)$=>\g<first>\g<delim>\g<pre>\g<post>
# Jack ends up before Rose when both appear.

This might be useful to add to MetadataManagement page

[ronnie0602]

Hello!

I'm brand new here although I've been reading posts ever since I discovered FFF a couple of years ago. I am grateful for the plugin - it allowed me to save my "fandom treasures" for a while now with minimal effort. So thank you very much for your work and your generosity in teaching even a novice such as me how to complete simple tasks to update my Calibre Library.

I have an older PC with Windows 7, use Calibre version 3.48 and get fanfics mostly from Fanfiction.net, AO3 and StarsLibrary. My Calibre and FFF setup is not sophisticated by any means.

Yesterday I have successfully downloaded a couple of new stories from Fanfiction.net; I also attempted to update an existing story and got an error message telling me that the ebook has been updated but the metadata has not. When I open the book in Calibre I can see in the story summary and in the created update log the new details (i.e. number of chapters and story update date) but the Calibre screen is not showing the new story update date (i.e. last update I had for the story was 5 chapters on March 13, 2023, today's download added 3 chapters and the Calibre screen table should show the most recent story update date of May 4, 2023 but it's still showing March 13, 2023).

I apologize for being so long-winded. I tried running the Metadata update only and I get the same error.

I have a debug file but can't figure out what went wrong. The last time I updated stories from FFnet was at the beginning of June with no issues and I'm not sure what I might have messed up on my PC since...


I would really appreciate it if you could spare a little time to help me.

Thank you very much.

[JimmXinu]

Originally Posted by ronnie0602

...
I have an older PC with Windows 7, use Calibre version 3.48 and get fanfics mostly from Fanfiction.net, AO3 and StarsLibrary. My Calibre and FFF setup is not sophisticated by any means.
...

I can see what went wrong, but I can't tell you why.

When FFF attempted to set the Author field, Calibre ran into an issue with Windows denying write access to the story dir in the library:

FFF: ERROR: 2023-06-30 13:17:45,960: calibre_plugins.fanficfare_plugin.fff_plugin(1801): Error Updating Metadata:
Traceback (most recent call last):
  File "calibre_plugins.fanficfare_plugin.fff_plugin", line 1798, in update_books_loop
  File "calibre_plugins.fanficfare_plugin.fff_plugin", line 2272, in update_metadata
  File "site-packages\calibre\db\legacy.py", line 806, in func
  File "site-packages\calibre\db\cache.py", line 62, in call_func_with_lock
  File "site-packages\calibre\db\cache.py", line 1122, in set_field
  File "site-packages\calibre\db\cache.py", line 1136, in update_path
  File "site-packages\calibre\db\backend.py", line 1590, in update_path
  File "os.py", line 157, in makedirs
WindowsError: [Error 5] Access is denied: u'C:\\Users\\STEVE\\Calibre Library - Copy\\Pink Hair and Roses\\Night Shift (11637)'

This isn't something FFF is doing (I tested that story in Cal 3.48 myself), but is probably something with your system: an anti-virus issue, file system permissions, possibly library corruption.

A very common problem (I don't know if this error matches exactly) is trying to put your library in network/cloud storage. That is not supported by Calibre.

Googling that error and Calibre finds some additional older threads.

Using Win7, you may have trouble find anyone that can help you more specifically--it's very unsupported by now.

[ronnie0602]

Jimm, thank you so much for taking the time. I can't think of anything that might have happened in the past few weeks since my last update - I'll check my very outdated system... Thanks again, I'll also check the suggested reading resources.

Best always!

[JimmXinu]

Okay, this is weird.

I just got a similar error to ronnie0602, occurring a couple lines earlier while setting most of the other book metadata.

Except I'm on Win10 with Calibre 6.22.

PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\\Users\\USER\\Documents\\Library\\Andrew Joshua Talon\\Paint and Powder (10896)'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "calibre_plugins.fanficfare_plugin.fff_plugin", line 1798, in update_books_loop
  File "calibre_plugins.fanficfare_plugin.fff_plugin", line 2268, in update_metadata
  File "calibre\db\legacy.py", line 588, in set_metadata
  File "calibre\db\cache.py", line 85, in call_func_with_lock
  File "calibre\db\cache.py", line 1691, in set_metadata
  File "calibre\db\cache.py", line 1686, in set_field
  File "calibre\db\cache.py", line 1481, in set_field
  File "calibre\db\backend.py", line 1726, in windows_check_if_files_in_use
  File "calibre\utils\copy_files.py", line 250, in windows_check_if_files_in_use
  File "calibre\utils\copy_files.py", line 136, in __enter__
  File "calibre\utils\copy_files.py", line 132, in open_all_handles
  File "calibre\utils\copy_files.py", line 123, in _open_file
  File "calibre\utils\copy_files.py", line 112, in _open_file
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\\Users\\USER\\Documents\\Library\\Andrew Joshua Talon\\Paint and Powder (10896)'

And trying to change the title of that book with Calibre's Edit dialog gives the same error, so it's not just FFF. But it doesn't happen for other books, so it's not the whole library, either.

Rebooting the OS seems to have cleared it. Unfortunately, I didn't find a good way to determine which processes have a file open until after I rebooted. Really wish I'd found that first.

I'd chalk it up to weirdness from installing a JDK and an IDE this morning, but the similarity of ronnie0602's issue is concerning...

[JimmXinu]

Okay, I think I've figured out what caused mine, at least.

I went and did several of the things I did earlier today that I don't usually and managed to duplicate the error and then use Resource Monitor to ID the other process. Which was explore.exe--the basic Windows File Explorer.

And one of the unusual things I had done was run a file search in File Explorer.

Closing all the File Explorer windows (I'd left the search some time ago) closed the process after a short delay and I could rename that book again without error.

I knew that an open File Explorer window into a dir could cause issues, but not that a search done hours ago could...