[GUI Plugin] FanFicFare

[JimmXinu]

Originally Posted by Gerani

So I had to do a reinstall of windows 10, I saved my data and reinstalled Calibre and Fanficfare. I have a problem that now it won't download or update anything.
When I try I just get the comment: %d format: a number is required, not unicode

I removed everything and did a clean install of calibre and fanficfare. Made a new library and tried again.

It still won't download, and just shows error code: %d format: a number is required, not unicode

Any idea on what's wrong?

I've seen problems like that before that turned out to be issues with the translation into another language. What language is your windows install?

If English, run Calibre in debug, try again and post the log, please.

[Tanjamuse]

Originally Posted by JimmXinu

I've been debating adding a feature to look up (and cache) AO3's tag hierarchy, since they've started equating random tags with more general ones (see Humor for example). But it would be quite a database.

Did you ever look at doing that?

I don't know the coding but I would love to help, especially weed out some of those sentences that I (at least) find very confusing or annoying to contend with.

I found this on their website: https://archiveofourown.org/wrangling_guidelines/2

If it can be coded, I would love to help with separating the tags like that, then it could be added into different columns in Calibre or simply left out if not needed.

[JimmXinu]

Well, that's a post from 05-16-2015 for one thing. I don't find anything written in my notes, so anything I remember is suspect.

But I believe that I decided it was a sisyphean task and I didn't care enough.

It would involve, at a minimum:
- Generating the list of all 'canonical' tags (685,301 Found)
- Fetching the page(s?) for each
- Determining what do to with all the 'Tags with the same meaning' vs all the 'sub tags'
- Are 'sub tags' also 'canonical'?
- Sub tags can have sub tags...
- Tags can be 'fandom', 'characters', etc
- Tags can have 'parent' tags that are not of the same type example
- Figuring out how and where to store all of the tag data (7,354,056 tags Found)
- ...and update it...
- ...and use it...

Now factor in that each user is still going to have their own opinion about which tags are important and which aren't and how they should be presented...



I know that in recent years I've taken to telling people that as long as AO3 allows anyone to write any tag they like, complete metadata consistency is going to be impossible.

[Tanjamuse]

Thanks for the considering, I just wished for a way to weed out all the run-on sentences or so instead of having to do it all manually Or simply keep what they call common-tags Shame they don't have a separate code for the "un-common" tags

[JimmXinu]

New Test Version Posted

2018-09-19
- Add 'date' chapter metadata to AO3. Closes #336

[rishidiams]

Hi there,

I'm trying to get Calibre to automatically change the "In-Progress" status to "Abandoned" based on the last time the fic was updated. I figure for simplicity's sake (and since I have no idea whether or not I could make this work for a specific number of months prior to whatever day I'm updating the fic), it would be easiest to say any time not in this year, and I can easily go in and change the year as needed.

Included is the relevant portion of my ini file. I'm getting a 'need more than 1 value to unpack' error when I try to save.

[archiveofourown.org] 
username: XXXXXXXXXXXXX
password: XXXXXXXXXXXXX 
include_subject_tags:,characters,status,fandoms,freeformtags,ships,dateupdated


[defaults] 
add_to_extra_valid_entries:,myships,mycharacters,rawcharacters,mytags,rawtags,myfandoms,mystatus,lastupdated
include_in_mystatus:status 
include_in_lastupdated:dateupdated
custom_columns_settings: 
 mystatus=>#status 
 lastupdated=>#updated
replace_metadata: 
 status=>In-Progress=>Abandoned&&lastupdated!=(.*)2018

I've also tried the following, based on adding "Oneshot" to the status if there is only one chapter.

[archiveofourown.org] 
username: XXXXXXXXXXXXX
password: XXXXXXXXXXXXX 
include_subject_tags:,characters,status,fandoms,freeformtags,ships,dateupdated


[defaults] add_to_extra_valid_entries:,abandoned,myships,mycharacters,rawcharacters,mytags,rawtags,myfandoms,mystatus,lastupdated
include_in_abandoned:status
add_to_include_subject_tags:,abandoned
include_in_mystatus:status 
include_in_lastupdated:dateupdated
custom_columns_settings: 
 mystatus=>#status 
 lastupdated=>#updated
replace_metadata: 
 abandoned=>In-Progress=>Abandoned&&lastupdated!=(.*)2018

[JimmXinu]

You have two problems.

First, you can't use != in replace_metadata. include/exclude have that, but not replace_metadata. Maybe someday. See https://github.com/JimmXinu/FanFicFa...eplaceMetadata

However, you can emulate it with a negative look ahead regex. I confess I got it from Stack Trace

Second, note lastUpdated (vs lastupdated).

This works for me:

[defaults] 
add_to_extra_valid_entries:,myships,mycharacters,rawcharacters,mytags,rawtags,myfandoms,mystatus,lastupdated
include_in_mystatus:status 
include_in_lastupdated:dateUpdated
custom_columns_settings: 
 mystatus=>#status 
 lastupdated=>#updated
replace_metadata: 
 status=>In-Progress=>Abandoned&&lastupdated=>^((?!2018).)*$

(I'm not sure why you have mystatus and lastupdated from this snippet, but I assume you have reasons.)

[rishidiams]

Originally Posted by JimmXinu

You have two problems.

First, you can't use != in replace_metadata. include/exclude have that, but not replace_metadata. Maybe someday. See https://github.com/JimmXinu/FanFicFa...eplaceMetadata

However, you can emulate it with a negative look ahead regex. I confess I got it from Stack Trace

Second, note lastUpdated (vs lastupdated).

This works for me:

[defaults] 
add_to_extra_valid_entries:,myships,mycharacters,rawcharacters,mytags,rawtags,myfandoms,mystatus,lastupdated
include_in_mystatus:status 
include_in_lastupdated:dateUpdated
custom_columns_settings: 
 mystatus=>#status 
 lastupdated=>#updated
replace_metadata: 
 status=>In-Progress=>Abandoned&&lastupdated=>^((?!2018).)*$

(I'm not sure why you have mystatus and lastupdated from this snippet, but I assume you have reasons.)

Thank you! That works perfectly!

Another question: is there a way to force the title column to use title case? For some reason, some AO3 authors think they're e.e. cummings.

[JimmXinu]

Originally Posted by rishidiams

Another question: is there a way to force the title column to use title case? For some reason, some AO3 authors think they're e.e. cummings.

Not in the existing code. Nor do I see a way to add such code that is both reasonably correct and only reasonably difficult.

Well, I suppose there is a way, but it is also a bit less than 'reasonably correct', only works for English and similar languages and is ugly to boot:

add_to_replace_metadata:
 title,author=>((^| )+)a=>\1A
 title,author=>((^| )+)b=>\1B
 title,author=>((^| )+)c=>\1C
 title,author=>((^| )+)d=>\1D
 title,author=>((^| )+)e=>\1E
 title,author=>((^| )+)f=>\1F
 title,author=>((^| )+)g=>\1G
 title,author=>((^| )+)h=>\1H
 title,author=>((^| )+)i=>\1I
 title,author=>((^| )+)j=>\1J
 title,author=>((^| )+)k=>\1K
 title,author=>((^| )+)l=>\1L
 title,author=>((^| )+)m=>\1M
 title,author=>((^| )+)n=>\1N
 title,author=>((^| )+)o=>\1O
 title,author=>((^| )+)p=>\1P
 title,author=>((^| )+)q=>\1Q
 title,author=>((^| )+)r=>\1R
 title,author=>((^| )+)s=>\1S
 title,author=>((^| )+)t=>\1T
 title,author=>((^| )+)u=>\1U
 title,author=>((^| )+)v=>\1V
 title,author=>((^| )+)w=>\1W
 title,author=>((^| )+)x=>\1X
 title,author=>((^| )+)y=>\1Y
 title,author=>((^| )+)z=>\1Z

Note that articles and other short words ('of', 'a', 'an', etc) that technically shouldn't be capitalized still are using this. And only the first 'E' of 'e.e.' would be capitalized, etc, etc.

Some of those issues could be addressed with even more or more complex patterns. Those are left as an exercise for the reader.

Or fix them individually when you find one that offends you. That's what I do; example:

[https://archiveofourown.org/works/999999]
add_to_replace_metadata:
 title=>.*=>Title as I think it Should Be

[rishidiams]

Originally Posted by JimmXinu

Not in the existing code. Nor do I see a way to add such code that is both reasonably correct and only reasonably difficult.

Well, I suppose there is a way, but it is also a bit less than 'reasonably correct', only works for English and similar languages and is ugly to boot:

add_to_replace_metadata:
 title,author=>((^| )+)a=>\1A
 title,author=>((^| )+)b=>\1B
 title,author=>((^| )+)c=>\1C
 title,author=>((^| )+)d=>\1D
 title,author=>((^| )+)e=>\1E
 title,author=>((^| )+)f=>\1F
 title,author=>((^| )+)g=>\1G
 title,author=>((^| )+)h=>\1H
 title,author=>((^| )+)i=>\1I
 title,author=>((^| )+)j=>\1J
 title,author=>((^| )+)k=>\1K
 title,author=>((^| )+)l=>\1L
 title,author=>((^| )+)m=>\1M
 title,author=>((^| )+)n=>\1N
 title,author=>((^| )+)o=>\1O
 title,author=>((^| )+)p=>\1P
 title,author=>((^| )+)q=>\1Q
 title,author=>((^| )+)r=>\1R
 title,author=>((^| )+)s=>\1S
 title,author=>((^| )+)t=>\1T
 title,author=>((^| )+)u=>\1U
 title,author=>((^| )+)v=>\1V
 title,author=>((^| )+)w=>\1W
 title,author=>((^| )+)x=>\1X
 title,author=>((^| )+)y=>\1Y
 title,author=>((^| )+)z=>\1Z

Note that articles and other short words ('of', 'a', 'an', etc) that technically shouldn't be capitalized still are using this. And only the first 'E' of 'e.e.' would be capitalized, etc, etc.

Some of those issues could be addressed with even more or more complex patterns. Those are left as an exercise for the reader.

Or fix them individually when you find one that offends you. That's what I do; example:

[https://archiveofourown.org/works/999999]
add_to_replace_metadata:
 title=>.*=>Title as I think it Should Be

I worried it would be something like that. Thanks!

[davidfor]

Originally Posted by JimmXinu

Not in the existing code. Nor do I see a way to add such code that is both reasonably correct and only reasonably difficult.

Well, I suppose there is a way, but it is also a bit less than 'reasonably correct', only works for English and similar languages and is ugly to boot:

For the record, calibre has some methods in ebooks.metadata.sources.base.py to handle the case of titles and authors. Most of the metadata source plugins use these via Source.clean_downloaded_metadata. An option to call something like this would at least mean FFF downloads are consistent with other books.

[JimmXinu]

I could add options to apply the calibre fix title/author routines in the plugin. Or I suppose I could try and incorporate those routines in FFF to use in CLI and web service too.

Are there other users who would like that feature? And if so, how widely? Applicable to title/author only? Or to any metadata entry?

[morsai]

Hello

I have a problem about italics.

After I downloaded this novel, I found out that there were no italic sentences in the novel. That's why, I went to website and opened the source code of the novel and I found out that they use <span style="font-style: italic"> instead of <em> or <i>. I assume that this is the reason but I'm not sure actually.

How can I solve this problem?

Thanks a lot.

I found the solution I think.

There was a similar problem (underline) in the forum and you gave an answer.

# keep style for all stories
[defaults]
add_to_keep_html_attrs:,style
# keep style for all ffnet stories
[fanfiction.net]
add_to_keep_html_attrs:,style
# keep style for story 7262793 only.
[https://www.fanfiction.net/s/7262793/1/]
add_to_keep_html_attrs:,style

I tried and my problem is solved.

Thanks.

[JimmXinu]

New Test Version Posted

2018-09-27
- Add fix calibre title/author cases options, move force title/author sort values to Standard Columns config Tab.

New options for title & author on Standard Columns config Tab. These apply calibre.ebooks.metadata.sources.base.fixcase and calibre.ebooks.metadata.sources.base.fixauthors to title and author. These only effect the metadata in Calibre, not in FFF.

Calibre's case fixing is actually quite complex due to different language localizations, so I'm not interested in adding it to FFF for CLI/web service.

[baggins41]

Hey, Jimm- I've run into a minor annoyance lately. I was hoping you might be able to either point me in the direction of a setting I may need to change or fix it if there's a bug involved.

Lately, any time I download a single chapter story from FimFiction (e.g. this one), the chapter title is no longer displayed before the text in the resultant epub. The chapter title is present in the metadata and in the table of contents, but it doesn't show up as a header in the actual text at the start of the chapter, as it used to.

I'm not sure precisely when it started (as I've been mostly downloading multi-chapter stories for the past few months). Looking through my Calibre library, I see stories that I downloaded at the end of July display the chapter title as an <H3> header at the start of the text, while stories I downloaded in the middle of August omit it.

Chapter titles on these one-shot stories sometimes serve as a subtitle for the story as a whole and may even help set the mood of the piece to some degree, so it's a bit disappointing not to have them. Any help would in getting them back would be appreciated.