Register Guidelines E-Books Today's Posts Search

Go Back   MobileRead Forums > E-Book Software > Calibre > Plugins

Notices

Reply
 
Thread Tools Search this Thread
Old 05-17-2022, 02:06 PM   #7141
DandalfTheWhite
Enthusiast
DandalfTheWhite began at the beginning.
 
Posts: 26
Karma: 10
Join Date: Nov 2011
Device: Kindle Paperwhite 4
I didn't realize it read links like that. Thanks for letting me know. I've never used the Reject list before but good idea. Thank you!
DandalfTheWhite is offline   Reply With Quote
Old 05-17-2022, 03:13 PM   #7142
ownedbycats
Custom User Title
ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.
 
ownedbycats's Avatar
 
Posts: 10,993
Karma: 75337983
Join Date: Oct 2018
Location: Canada
Device: Kobo Libra H2O, formerly Aura HD
I've had the same thing happen when dragging and dropping from my email client, but thankfully I can just remove it from the URL dialog before running the download.
ownedbycats is offline   Reply With Quote
Advert
Old 05-17-2022, 03:52 PM   #7143
ackomb
Zealot
ackomb has a spectacular aura aboutackomb has a spectacular aura aboutackomb has a spectacular aura aboutackomb has a spectacular aura aboutackomb has a spectacular aura aboutackomb has a spectacular aura aboutackomb has a spectacular aura aboutackomb has a spectacular aura aboutackomb has a spectacular aura aboutackomb has a spectacular aura aboutackomb has a spectacular aura about
 
Posts: 106
Karma: 4486
Join Date: Mar 2020
Location: Netherlands
Device: i-pad
Hey Jimm,

I just noticed something weird. Or at least it seems weird to me.
U was running an "update metadata from saved metadata column" as I was testing some new code but all failed. When I looked at the error code it said:
Code:
Connection to flaresolverr proxy server failed.  Is flaresolverr started?
So I started flaresolver, tried again and it works fine.
But I don't understand why it is trying to connect to the website at all, since I'm not trying to. Especially while testing.

Ps. their actually stories that are no longer online, but that should not matter should it?

Update: And just as I posted this I noticed there was no saved metadata in these stories because back then I didn't save it.
ackomb is offline   Reply With Quote
Old 05-17-2022, 07:04 PM   #7144
ackomb
Zealot
ackomb has a spectacular aura aboutackomb has a spectacular aura aboutackomb has a spectacular aura aboutackomb has a spectacular aura aboutackomb has a spectacular aura aboutackomb has a spectacular aura aboutackomb has a spectacular aura aboutackomb has a spectacular aura aboutackomb has a spectacular aura aboutackomb has a spectacular aura aboutackomb has a spectacular aura about
 
Posts: 106
Karma: 4486
Join Date: Mar 2020
Location: Netherlands
Device: i-pad
As for my previous post, I still find it kind of strange that when you ask it to look for a saved column, it still goes to check on the site. Perhaps this is some kind of fallback but, I'd rather be warned ahead to avoid unnecessary traffic to the sites or switch it off completely.

That said, I tried testing with a few different stories that actually have saved metadata, but I keep getting the following error:
Spoiler:
calibre, version 5.42.0
ERROR: Error Updating Metadata: <p>An error has occurred while FanFicFare was updating calibre's metadata for <a href='https://archiveofourown.org/works/27936862'>Impermanence</a>.</p>The ebook has been updated, but the metadata has not.

Traceback (most recent call last):
File "calibre_plugins.fanficfare_plugin.fff_plugin" , line 1825, in update_books_loop
File "calibre_plugins.fanficfare_plugin.fff_plugin" , line 2358, in update_metadata
ValueError: too many values to unpack (expected 2)

Story Details:title:
Impermanence
author_sort:
['SimplyKorra']
author:
['SimplyKorra']
comments:
<div class="userstuff">
<p>This necklace - a dark blue band with a silver pendant - was a promise, more than anything. A promise Asami intended to make to Korra that meant forever. That meant no matter how often the world called on Korra to save it, Asami would be here for her, waiting and ready to tend to her wounds and find her smile again. </p><p>Asami and Korra found their happy ending. If only the world would let them enjoy it.</p>
</div>
good:
True
status:
Meta
showerror:
True
calibre_id:
57
begin:
None
end:
None
comment:
Metadata collected.
url:
https://archiveofourown.org/works/27936862
site:
archiveofourown.org
series:

added:
False
pubdate:
2020-12-07 00:00:00+01:00
publisher:
archiveofourown.org
listorder:
2
collision:
Update Calibre Metadata from Saved Metadata Column
is_adult:
False
username:
(was set, removed for security)
password:

icon:
metadata.png
all_metadata:
category:
Avatar: Legend of Korra
genre:
Anxious Korra, F/F, Future Fic, Gun Violence, Minor Character Death, Post-Canon, Post-Traumatic Stress Disorder - PTSD, Protective Asami, Takes things from the comics, but the comics aren't canon to the story
characters:
Asami Sato, Bolin (Avatar), Korra, Korra (Avatar), Mako (Avatar), Opal (Avatar), Tenzin (Avatar)
ships:
Korra/Asami Sato
warnings:
No Archive Warnings Apply
extratags:

author:
SimplyKorra
authorId:
SimplyKorra
authorUrl:
https://archiveofourown.org/users/Si...ds/SimplyKorra
lastupdate:
Last Update Year/Month: 2021/11, Last Update: 2021/11/12
series:

seriesUrl:

language:
English
status:
Completed
datePublished:
2020-12-07
dateUpdated:
2021-11-12
dateCreated:
2021-11-13 21:16:02
rating:
Mature
numChapters:
23
numWords:
102,892
words_added:

site:
archiveofourown.org
publisher:
archiveofourown.org
storyId:
27936862
title:
Impermanence
titleHTML:
<a class='titlelink' href='https://archiveofourown.org/works/27936862'>Impermanence</a>
storyUrl:
https://archiveofourown.org/works/27936862
sectionUrl:
https://archiveofourown.org/works/27936862
description:
<div class="userstuff">
<p>This necklace - a dark blue band with a silver pendant - was a promise, more than anything. A promise Asami intended to make to Korra that meant forever. That meant no matter how often the world called on Korra to save it, Asami would be here for her, waiting and ready to tend to her wounds and find her smile again. </p><p>Asami and Korra found their happy ending. If only the world would let them enjoy it.</p>
</div>
formatname:
epub
formatext:
.epub
siteabbrev:
ao3
version:
FanFicFare v4.12.4
authorHTML:
<a class='authorlink' href='https://archiveofourown.org/users/SimplyKorra/pseuds/SimplyKorra'>SimplyKorra</a>
seriesHTML:

langcode:
en
output_css:

cover_image:

fandoms:
Avatar: Legend of Korra
freeformtags:
Anxious Korra, Future Fic, Gun Violence, Minor Character Death, Post-Canon, Post-Traumatic Stress Disorder - PTSD, Protective Asami, Takes things from the comics, but the comics aren't canon to the story
freefromtags:
Anxious Korra, Future Fic, Gun Violence, Minor Character Death, Post-Canon, Post-Traumatic Stress Disorder - PTSD, Protective Asami, Takes things from the comics, but the comics aren't canon to the story
ao3categories:
F/F
comments:
679
chapterslashtotal:
23/23
chapterstotal:
23
kudos:
1113
hits:
35725
bookmarks:
196
collections:

byline:
SimplyKorra
bookmarked:
False
bookmarktags:

bookmarksummary:

bookmarkprivate:

bookmarkrec:

restricted:

series00:

series01:

series02:

series03:

series00Url:

series01Url:

series02Url:

series03Url:

series00HTML:

series01HTML:

series02HTML:

series03HTML:

ships2chars:
Asami Sato, Korra
relationships:
Korra/Asami Sato
crossovers:
Anxious Korra, F/F, Future Fic, Gun Violence, Minor Character Death, Post-Canon, Post-Traumatic Stress Disorder - PTSD, Protective Asami, Takes things from the comics, but the comics aren't canon to the story
savemetacol:
<p><span class='label'>Adapter_Classes</span>: <div class='metadata list' id='adapter_classes'><ul>
<li>ArchiveOfOurOwnOrgAdapter</li>
</ul></div><p>

<p><span class='label'>AO3 Categories</span>: <div class='metadata list' id='ao3categories'><ul>
<li>F/F</li>
</ul></div><p>

<p><span class='label'>Author</span>: <div class='metadata list' id='author'><ul>
<li>SimplyKorra</li>
</ul></div><p>

<p><span class='label'>Authorhtml</span>: <div class='metadata' id='authorHTML'><a class='authorlink' href='https://archiveofourown.org/users/SimplyKorra/pseuds/SimplyKorra'>SimplyKorra</a></div><p>

<p><span class='label'>Author ID</span>: <div class='metadata list' id='authorId'><ul>
<li>SimplyKorra</li>
</ul></div><p>

<p><span class='label'>Author URL</span>: <div class='metadata list' id='authorUrl'><ul>
<li>https://archiveofourown.org/users/SimplyKorra/pseuds/SimplyKorra</li>
</ul></div><p>

<p><span class='label'>I Bookmarked Story</span>: <div class='metadata' id='bookmarked'>False</div><p>

<p><span class='label'>Bookmarks</span>: <div class='metadata' id='bookmarks'>196</div><p>

<p><span class='label'>My Bookmark Summary</span>: <div class='metadata' id='bookmarksummary'></div><p>

<p><span class='label'>Byline</span>: <div class='metadata' id='byline'>SimplyKorra</div><p>

<p><span class='label'>Chapters/Total Chapters</span>: <div class='metadata' id='chapterslashtotal'>23/23</div><p>

<p><span class='label'>Characters</span>: <div class='metadata list' id='characters'><ul>
<li>Korra (Avatar)</li>
<li>Asami Sato</li>
<li>Mako (Avatar)</li>
<li>Bolin (Avatar)</li>
<li>Opal (Avatar)</li>
<li>Tenzin (Avatar)</li>
</ul></div><p>

<p><span class='label'>Comments</span>: <div class='metadata' id='comments'>679</div><p>

<p><span class='label'>Packaged</span>: <div class='metadata datetime' id='dateCreated'>2021-11-13T21:16:02.286987</div><p>

<p><span class='label'>Published</span>: <div class='metadata datetime' id='datePublished'>2020-12-07T00:00:00</div><p>

<p><span class='label'>Updated</span>: <div class='metadata datetime' id='dateUpdated'>2021-11-12T00:00:00</div><p>

<p><span class='label'>Summary</span>: <div class='metadata' id='description'><div class="userstuff">
<p>This necklace - a dark blue band with a silver pendant - was a promise, more than anything. A promise Asami intended to make to Korra that meant forever. That meant no matter how often the world called on Korra to save it, Asami would be here for her, waiting and ready to tend to her wounds and find her smile again. </p><p>Asami and Korra found their happy ending. If only the world would let them enjoy it.</p>
</div></div><p>

<p><span class='label'>Fandoms</span>: <div class='metadata list' id='fandoms'><ul>
<li>Avatar: Legend of Korra</li>
</ul></div><p>

<p><span class='label'>File Extension</span>: <div class='metadata' id='formatext'>.epub</div><p>

<p><span class='label'>File Format</span>: <div class='metadata' id='formatname'>epub</div><p>

<p><span class='label'>Freeform Tags</span>: <div class='metadata list' id='freeformtags'><ul>
<li>Post-Canon</li>
<li>Future Fic</li>
<li>Takes things from the comics</li>
<li>but the comics aren't canon to the story</li>
<li>Post-Traumatic Stress Disorder - PTSD</li>
<li>Anxious Korra</li>
<li>Protective Asami</li>
<li>Minor Character Death</li>
<li>Gun Violence</li>
</ul></div><p>

<p><span class='label'>Hits</span>: <div class='metadata' id='hits'>35725</div><p>

<p><span class='label'>Kudos</span>: <div class='metadata' id='kudos'>1113</div><p>

<p><span class='label'>Langcode</span>: <div class='metadata' id='langcode'>en</div><p>

<p><span class='label'>Language</span>: <div class='metadata' id='language'>English</div><p>

<p><span class='label'>Lastupdate</span>: <div class='metadata list' id='lastupdate'><ul>
<li>Last Update Year/Month: 2021/11</li>
<li>Last Update: 2021/11/12</li>
</ul></div><p>

<p><span class='label'>Chapters</span>: <div class='metadata int' id='numChapters'>23</div><p>

<p><span class='label'>Words</span>: <div class='metadata' id='numWords'>102892</div><p>

<p><span class='label'>Python_Version</span>: <div class='metadata' id='python_version'>3.8.5 (default, Aug 27 2020, 05:10:18) [MSC v.1927 32 bit (Intel)]</div><p>

<p><span class='label'>Rating</span>: <div class='metadata' id='rating'>Mature</div><p>

<p><span class='label'>Story URL Section</span>: <div class='metadata' id='sectionUrl'>https://archiveofourown.org/works/27936862</div><p>

<p><span class='label'>Series</span>: <div class='metadata' id='series00HTML'></div><p>

<p><span class='label'>Additional Series</span>: <div class='metadata' id='series01HTML'></div><p>

<p><span class='label'>Additional Series</span>: <div class='metadata' id='series02HTML'></div><p>

<p><span class='label'>Additional Series</span>: <div class='metadata' id='series03HTML'></div><p>

<p><span class='label'>Relationships</span>: <div class='metadata list' id='ships'><ul>
<li>Korra/Asami Sato</li>
</ul></div><p>

<p><span class='label'>Publisher</span>: <div class='metadata' id='site'>archiveofourown.org</div><p>

<p><span class='label'>Site Abbrev</span>: <div class='metadata' id='siteabbrev'>ao3</div><p>

<p><span class='label'>Status</span>: <div class='metadata' id='status'>Completed</div><p>

<p><span class='label'>Story ID</span>: <div class='metadata' id='storyId'>27936862</div><p>

<p><span class='label'>Story URL</span>: <div class='metadata' id='storyUrl'>https://archiveofourown.org/works/27936862</div><p>

<p><span class='label'>Title</span>: <div class='metadata' id='title'>Impermanence</div><p>

<p><span class='label'>Titlehtml</span>: <div class='metadata' id='titleHTML'><a class='titlelink' href='https://archiveofourown.org/works/27936862'>Impermanence</a></div><p>

<p><span class='label'>Downloader Version</span>: <div class='metadata' id='version'>FanFicFare v4.12.4</div><p>

<p><span class='label'>Warnings</span>: <div class='metadata list' id='warnings'><ul>
<li>No Archive Warnings Apply</li>
</ul></div><p>

tags:
['F/F']
updatedate:
2021-11-12 00:00:00+01:00
timestamp:
2021-11-13 21:16:02.286987+01:00


Kinda OT sorry, but does anyone know if there is a way to update manually edited metadata into the Title Page, without actually editing the book by hand. I've been looking around, but so far without a result. If someone has any idea, or even to give me a clue what or where to look I'd be happy to receive your feedback. I've tried Calibre's Embed Metadata, and the the plugin Modify Epub with option Metadata, but I'm not sure what they do, but not what I want them to do ;-).
ackomb is offline   Reply With Quote
Old 05-17-2022, 07:12 PM   #7145
ownedbycats
Custom User Title
ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.
 
ownedbycats's Avatar
 
Posts: 10,993
Karma: 75337983
Join Date: Oct 2018
Location: Canada
Device: Kobo Libra H2O, formerly Aura HD
Quote:
Originally Posted by ackomb View Post
Kinda OT sorry, but does anyone know if there is a way to update manually edited metadata into the Title Page, without actually editing the book by hand.
I asked about whether the metadata could be updated from the saved column. If I remember right, Jim said that he'd considered it but it turned out to be difficult/time-expensive to actually implement. (Though that could've been a different issue. Things sometimes blur together in my head. )

I just manually edit if the story isn't available or on ffnet. It's pretty easy and Calibre's editor even lets you save search-and-replaces.

Last edited by ownedbycats; 05-18-2022 at 06:23 PM.
ownedbycats is offline   Reply With Quote
Advert
Old 05-17-2022, 09:18 PM   #7146
JimmXinu
Plugin Developer
JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.
 
JimmXinu's Avatar
 
Posts: 6,974
Karma: 4604635
Join Date: Dec 2011
Location: Midwest USA
Device: Kobo Clara Colour running KOReader
Quote:
Originally Posted by ackomb View Post
As for my previous post, I still find it kind of strange that when you ask it to look for a saved column, it still goes to check on the site. Perhaps this is some kind of fallback but, I'd rather be warned ahead to avoid unnecessary traffic to the sites or switch it off completely.
'Update metadata from saved column' came after 'Update metadata from web site'. As you figured out, FFF assumes you want to update your metadata and falls back to 'from web site' if it isn't saved.

It's pretty trivial to search or virtual library on populated saved column if you don't want that. I'm not terribly interested in changing the behavior now--it's worked that way for years.

Quote:
Originally Posted by ackomb View Post
That said, I tried testing with a few different stories that actually have saved metadata, but I keep getting the following error:
It appears to be indicating a bad replace_metadata line. But there's code that's supposed to detect and report that type of error when you save personal.ini. Did you override an error? I'd have to see your personal.ini (safe version) to say more.

Quote:
Originally Posted by ackomb View Post
Kinda OT sorry, but does anyone know if there is a way to update manually edited metadata into the Title Page, without actually editing the book by hand. I've been looking around, but so far without a result. If someone has any idea, or even to give me a clue what or where to look I'd be happy to receive your feedback. I've tried Calibre's Embed Metadata, and the the plugin Modify Epub with option Metadata, but I'm not sure what they do, but not what I want them to do ;-).
You can customize Calibre's metadata jacket, but I'm not sure to what degree. See https://manual.calibre-ebook.com/customize.html

I think there's probably an opening for a plugin that injects a customizable metadata title page or jacket, something more detailed than a cover.

I would consider calling such a plugin from FFF, but I'm not interested making it.
JimmXinu is offline   Reply With Quote
Old 05-18-2022, 02:13 PM   #7147
ackomb
Zealot
ackomb has a spectacular aura aboutackomb has a spectacular aura aboutackomb has a spectacular aura aboutackomb has a spectacular aura aboutackomb has a spectacular aura aboutackomb has a spectacular aura aboutackomb has a spectacular aura aboutackomb has a spectacular aura aboutackomb has a spectacular aura aboutackomb has a spectacular aura aboutackomb has a spectacular aura about
 
Posts: 106
Karma: 4486
Join Date: Mar 2020
Location: Netherlands
Device: i-pad
Quote:
Originally Posted by JimmXinu View Post
'Update metadata from saved column' came after 'Update metadata from web site'. As you figured out, FFF assumes you want to update your metadata and falls back to 'from web site' if it isn't saved.

It's pretty trivial to search or virtual library on populated saved column if you don't want that. I'm not terribly interested in changing the behavior now--it's worked that way for years.
Oh, I'm not that bothered by it, I just mentioned it in case it wasn't intentional.

Quote:
Originally Posted by JimmXinu View Post

It appears to be indicating a bad replace_metadata line. But there's code that's supposed to detect and report that type of error when you save personal.ini. Did you override an error? I'd have to see your personal.ini (safe version) to say more.
I was actually trying to clean-up my personal.ini so for testing purposes and speed I kind stripped it for the most part as you can see here.

Spoiler:
## [defaults] section applies to all formats and sites but may be overridden at several levels.
[defaults]

## Some sites also require the user to confirm they are adult for adult content.
is_adult:true
include_titlepage: true


##================================================ ==========================## AO3 - ARCHIVE OF OUR OWN ##================================================ =======
##================================================ ==========================## AO3 - ARCHIVE OF OUR OWN ##================================================ =======
##================================================ ==========================## AO3 - ARCHIVE OF OUR OWN ##================================================ =======


[www.archiveofourown.org]
use_basic_cache:true
always_login:true
username:XXXXXXXX
password:XXXXXXXX

## AO3 offers a page to download the entire work at once. This can save a lot of network traffic if there are many chapters. True by built-in default.
use_view_full_work:true

add_to_extra_valid_entries:,ships2chars,relationsh ips,crossovers
include_in_ships2chars:ships
include_in_relationships:ships
include_in_crossovers:genre

## Tells which tags from the site to include in calibre
include_subject_tags:extratags,ao3categories
include_in_characters:characters,ships2chars
include_in_genre:freeformtags,crossovers
include_in_category:fandoms

add_to_replace_metadata:
ships2chars=>[ ]*/[ ]*=>/
ships2chars=>[ ]*&amp;[ ]*=>/
ships2chars=>([^/]+)/([^/]+)/([^/]+)/(.+)=>\1\,\2\,\3\,\4
ships2chars=>([^/]+)/([^/]+)/(.+)=>\1\,\2\,\3
ships2chars=>([^/]+)/(.+)=>\1\,\2


add_to_include_subject_tags:,tagsfromtitle.SPLIT

## Tells which tags from the site to include on the titlepage
titlepage_entries:series,category,crossovers,genre ,characters,relationships,status,datePublished,dat eUpdated,rating,numChapters,numWords,site,descript ion,

## Tells which tags goes to which customs column
custom_columns_settings:
genre=>#ao3tags_genre,a
category=>#ao3tags_show,a
fandom=>=>#ao3tags_show,a
crossovers=>#ao3tags_show,a
characters=>#ao3tags_char,a
relationships=>#ao3tags_pair,a

replace_metadata:
genre,category,characters=> &amp; =>/
genre,category,characters=>&amp;=>/
ships,genre,category,characters=> \| =>/

##================================================ ==========================## SHOWS ##================================================ ==========================
##================================================ ==========================## SHOWS ##================================================ ==========================


##================================================ ==========================## CHARACTERS ##================================================ =====================
##================================================ ==========================## CHARACTERS ##================================================ =====================


##================================================ ==========================## SHIPS ##================================================ ==========================
##================================================ ==========================## SHIPS ##================================================ ==========================


##================================================ ==========================## GENRE ##================================================ ==========================
##================================================ ==========================## GENRE ##================================================ ==========================



#################### LEAVE THIS ######################
## Do not overwrite this when updating, is meant to be at the end!!!


## Includes/Excludes tags regardless
exclude_metadata_pre:

include_metadata_post:

exclude_metadata_post:


## This section will override anything in the system defaults or other
## sections here.
[overrides]
## default varies by site. Set true here to force all sites to collect series.
collect_series: true
extratags:

## Add this to genre if there's more than one category.
add_genre_when_multi_category: #Crossover


It was getting quite long and clogged up with things that weren't necessarily up to date anymore. To many singular lines address one specific tag, and keeping up with the posts here has taught me a few new trick I trying out.

Quote:
Originally Posted by JimmXinu View Post
'Update metadata from saved column' came after 'Update metadata from web site'. As you figured out, FFF assumes you want to update your metadata and falls back to 'from web site' if it isn't saved.


You can customize Calibre's metadata jacket, but I'm not sure to what degree. See https://manual.calibre-ebook.com/customize.html

I think there's probably an opening for a plugin that injects a customizable metadata title page or jacket, something more detailed than a cover.

I would consider calling such a plugin from FFF, but I'm not interested making it.
Thanks, I will look at the Jacket some more.
I wasn't acctually expecting it to be part of FFF. And have been searching for a separate plugin to do just that.
Modify Epub looked promising, but it just isn't working the way I expected it to.
ackomb is offline   Reply With Quote
Old 05-18-2022, 06:17 PM   #7148
JimmXinu
Plugin Developer
JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.
 
JimmXinu's Avatar
 
Posts: 6,974
Karma: 4604635
Join Date: Dec 2011
Location: Midwest USA
Device: Kobo Clara Colour running KOReader
Quote:
Originally Posted by ackomb View Post
## Tells which tags goes to which customs column
custom_columns_settings:
genre=>#ao3tags_genre,a
category=>#ao3tags_show,a
fandom=>=>#ao3tags_show,a
crossovers=>#ao3tags_show,a
characters=>#ao3tags_char,a
relationships=>#ao3tags_pair,a
custom_columns_settings expects only one =>.

I should have said "It appears to be indicating a bad custom_columns_settings line," not replace_metadata; sorry.

And it turns out I never implemented config checking for custom_columns_settings lines. I'll look into that.
JimmXinu is offline   Reply With Quote
Old 05-18-2022, 06:45 PM   #7149
ownedbycats
Custom User Title
ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.
 
ownedbycats's Avatar
 
Posts: 10,993
Karma: 75337983
Join Date: Oct 2018
Location: Canada
Device: Kobo Libra H2O, formerly Aura HD
Question: For the fanfic description in the Calibre metadata, does FFF change leading dashes at all? e.g.

Code:
<p>"Blah blah blah, quote text here."</p>
<p>- Quote author</p>
In a few of my fics, I noticed the second bit got replaced with a <li>Quote author</li> (Calibre description only; the title page summary is fine) and I'm trying to identify what's doing it.

Last edited by ownedbycats; 05-18-2022 at 06:51 PM.
ownedbycats is offline   Reply With Quote
Old 05-18-2022, 06:59 PM   #7150
JimmXinu
Plugin Developer
JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.
 
JimmXinu's Avatar
 
Posts: 6,974
Karma: 4604635
Join Date: Dec 2011
Location: Midwest USA
Device: Kobo Clara Colour running KOReader
New Test Version Posted

2022-05-18
- Add config check parsing for custom_columns_settings on personal.ini save.
JimmXinu is offline   Reply With Quote
Old 05-18-2022, 07:24 PM   #7151
JimmXinu
Plugin Developer
JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.
 
JimmXinu's Avatar
 
Posts: 6,974
Karma: 4604635
Join Date: Dec 2011
Location: Midwest USA
Device: Kobo Clara Colour running KOReader
Quote:
Originally Posted by ownedbycats View Post
Question: For the fanfic description in the Calibre metadata, does FFF change leading dashes at all? e.g.

Code:
<p>"Blah blah blah, quote text here."</p>
<p>- Quote author</p>
In a few of my fics, I noticed the second bit got replaced with a <li>Quote author</li> (Calibre description only; the title page summary is fine) and I'm trying to identify what's doing it.
I would speculate Calibre is 'helping' with what it thinks is markdown(MD).

If you have keep_summary_html:false, calibre.library.comments.sanitize_comments_html is called on the description before it's given to Calibre. There's a comment about MD text, and FFF has been calling sanitize_comments_html since at least 2014 and probably longer. It gets hard to trace exactly.

Code:
            if not self.getConfig('keep_summary_html'):
                ## because of the html->MD text->html dance, text only
                ## (or MD/MD-like) descs come out better.
                description = sanitize_comments_html(description)
If you have keep_summary_html:true (which is likely, given your snippet), FFF 'strains' the description HTML through BeautifulSoup(again), but otherwise leaves it alone before passing to Calibre.

There are some indications the html5lib knows about MD, but if that were the cause, it would show up in FFF's title page too.
JimmXinu is offline   Reply With Quote
Old 05-19-2022, 03:07 PM   #7152
ackomb
Zealot
ackomb has a spectacular aura aboutackomb has a spectacular aura aboutackomb has a spectacular aura aboutackomb has a spectacular aura aboutackomb has a spectacular aura aboutackomb has a spectacular aura aboutackomb has a spectacular aura aboutackomb has a spectacular aura aboutackomb has a spectacular aura aboutackomb has a spectacular aura aboutackomb has a spectacular aura about
 
Posts: 106
Karma: 4486
Join Date: Mar 2020
Location: Netherlands
Device: i-pad
Quote:
Originally Posted by JimmXinu View Post
custom_columns_settings expects only one =>.

I should have said "It appears to be indicating a bad custom_columns_settings line," not replace_metadata; sorry.

And it turns out I never implemented config checking for custom_columns_settings lines. I'll look into that.
No need to be sorry, without all the work you put in to this we'd all be lost. I'm just grateful for all the effort you put in to this.
ackomb is offline   Reply With Quote
Old 05-19-2022, 11:20 PM   #7153
sandmanwake
Enthusiast
sandmanwake has learned how to buy an e-book online
 
Posts: 26
Karma: 82
Join Date: Aug 2018
Device: none
Webnovel's giving errors again:

Spoiler:


EpubMerge: DEBUG: 2022-05-19 23:16:44,939: calibre_plugins.epubmerge.epubmerge_plugin(155): macmenuhack file_path:C:\Users\username\AppData\Roaming\calibr e\plugins\fanficfare_macmenuhack.txt
FFF: DEBUG: 2022-05-19 23:16:44,950: calibre_plugins.fanficfare_plugin.fff_plugin(218): Plugin FanFicFare macmenuhack file_path:C:\Users\username\AppData\Roaming\calibr e\plugins\fanficfare_macmenuhack.txt
calibre Debug log
calibre 5.42 embedded-python: True is64bit: False
Windows-10-10.0.19041 Windows ('32bit', 'WindowsPE')
32bit process running on 64bit windows
('Windows', '10', '10.0.19041')
Python 3.8.5
Windows: ('10', '10.0.19041', '', 'Multiprocessor Free')
Interface language: None
Successfully initialized third party plugins: EpubMerge (2, 13, 0) && EpubSplit (3, 3, 0) && FanFicFare (4, 12, 10)
calibre 5.42 embedded-python: True is64bit: False
Windows-10-10.0.19041 Windows ('32bit', 'WindowsPE')
32bit process running on 64bit windows
('Windows', '10', '10.0.19041')
Python 3.8.5
Windows: ('10', '10.0.19041', '', 'Multiprocessor Free')
Interface language: None
Successfully initialized third party plugins: EpubMerge (2, 13, 0) && EpubSplit (3, 3, 0) && FanFicFare (4, 12, 10)
Turning on automatic hidpi scaling
devicePixelRatio: 1.0
logicalDpi: 96.0 x 96.0
physicalDpi: 94.14671814671814 x 94.07407407407408
Using calibre Qt style: True
[0.00] Starting up...
[0.00] Showing splash screen...
[0.56] splash screen shown
[0.56] Initializing db...
[2.50] db initialized
[2.50] Constructing main UI...
[4.91] main UI initialized...
[4.91] Hiding splash screen
Starting QuickView
FFF: DEBUG: 2022-05-19 23:16:52,329: calibre_plugins.fanficfare_plugin.dialogs(497): self.extraoptions['anthology_url']:NOT FOUND
FFF: DEBUG: 2022-05-19 23:16:52,331: calibre_plugins.fanficfare_plugin.fff_plugin(1129) : FanFicFare v4.12.10
FFF: INFO: 2022-05-19 23:16:52,350: calibre_plugins.fanficfare_plugin.prefs(214): Using default settings
FFF: DEBUG: 2022-05-19 23:16:52,366: configurable.py(1034): use_browser_cache:
FFF: DEBUG: 2022-05-19 23:16:52,366: configurable.py(1048): use_basic_cache:true
FFF: DEBUG: 2022-05-19 23:16:52,368: adapter_webnovelcom.py(85): https://www.webnovel.com/book/22552918606279505
FFF: DEBUG: 2022-05-19 23:16:52,369: fetcher.py(234):
========== MISS (GET) BasicCache
https://www.webnovel.com/book/22552918606279505
FFF: DEBUG: 2022-05-19 23:16:52,369: fetcher.py(469):
---------- REQ (GET) RequestsFetcher
https://www.webnovel.com/book/22552918606279505
FFF: DEBUG: 2022-05-19 23:16:52,930: fetcher.py(482): response code:200
FFF: DEBUG: 2022-05-19 23:16:52,930: requestable.py(50): Encoding:utf8
FFF: DEBUG: 2022-05-19 23:16:53,100: fetcher.py(234):
========== MISS (GET) BasicCache
https://www.webnovel.com/go/pcm/chap...05&pageIndex=0
FFF: DEBUG: 2022-05-19 23:16:53,100: fetcher.py(469):
---------- REQ (GET) RequestsFetcher
https://www.webnovel.com/go/pcm/chap...05&pageIndex=0
FFF: DEBUG: 2022-05-19 23:16:53,220: fetcher.py(482): response code:200
FFF: DEBUG: 2022-05-19 23:16:53,220: requestable.py(50): Encoding:utf8
FFF: ERROR: 2022-05-19 23:16:53,220: calibre_plugins.fanficfare_plugin.dialogs(714): Exception: {'title': 'Unknown', 'author_sort': ['Unknown'], 'author': ['Unknown'], 'comments': '', 'good': False, 'status': 'Error', 'showerror': True, 'calibre_id': None, 'begin': None, 'end': None, 'comment': "'volumeItems'", 'url': 'https://www.webnovel.com/book/22552918606279505', 'site': 'www.webnovel.com', 'series': '', 'added': False, 'pubdate': None, 'publisher': None, 'uniqueurl': 'https://www.webnovel.com/book/22552918606279505[None-None]', 'listorder': 0, 'collision': 'Update EPUB if New Chapters'}:'volumeItems'
Traceback (most recent call last):
File "calibre_plugins.fanficfare_plugin.dialogs", line 701, in do_loop
File "calibre_plugins.fanficfare_plugin.fff_plugin" , line 1343, in prep_download_loop
File "calibre_plugins.fanficfare_plugin.fff_plugin" , line 1215, in get_story_metadata_only
File "C:\Users\username\AppData\Roaming\calibre\plugins \FanFicFare.zip\fanficfare\adapters\base_adapter.p y", line 308, in getStoryMetadataOnly
File "C:\Users\username\AppData\Roaming\calibre\plugins \FanFicFare.zip\fanficfare\adapters\adapter_webnov elcom.py", line 165, in doExtractChapterUrlsAndMetadata
KeyError: 'volumeItems'
[7.86] splash screen hidden
[7.86] Started up in 7.86 seconds with 14303 books

sandmanwake is offline   Reply With Quote
Old 05-20-2022, 11:19 AM   #7154
JimmXinu
Plugin Developer
JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.
 
JimmXinu's Avatar
 
Posts: 6,974
Karma: 4604635
Join Date: Dec 2011
Location: Midwest USA
Device: Kobo Clara Colour running KOReader
Quote:
Originally Posted by sandmanwake View Post
Webnovel's giving errors again:
Also reported here #843.

What said there applies:
Quote:
Instead of giving the chapter list in clear text JSON, the site is now giving some long base64 in a JSON packet that looks like this:
Code:
{
  "code":0,
  "data":{
    "content":"R2HNZ.....BNBEJPvc1lBS6rSUxYcQ==",
    "encryptKeyPool":"ujh.....BM4Xf6hC/l3x6u6/tw==",
    "encryptType":1,
    "encryptVersion":1652956258
  },
  "msg":"Success"
}
...except encryptKeyPool is ~80 chars long and content is 200k long.

The names suggest there's now some encryption going on. Presumably there's code in the browser or javascript loaded by the page that decrypts it.

This may or may not be fixable in FFF, depending on what's going on. I'm not going to be very interested in supporting it in FFF if it ends up requiring a bunch of crypto libraries.
JimmXinu is offline   Reply With Quote
Old 05-20-2022, 12:49 PM   #7155
ownedbycats
Custom User Title
ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.ownedbycats ought to be getting tired of karma fortunes by now.
 
ownedbycats's Avatar
 
Posts: 10,993
Karma: 75337983
Join Date: Oct 2018
Location: Canada
Device: Kobo Libra H2O, formerly Aura HD
I'm curious as to why they'd be encrypting the chapter list to begin with. That just seems... odd. Unless the point is to block tools like FFF, but there's probably better ways to go about it.
ownedbycats is offline   Reply With Quote
Reply

Tags
fanfiction


Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
[GUI Plugin] KindleUnpack - The Plugin DiapDealer Plugins 523 07-15-2025 06:45 PM
[GUI Plugin] Open With kiwidude Plugins 404 02-21-2025 05:42 AM
[GUI Plugin] Marvin XD Philantrop Plugins 126 01-29-2017 12:48 PM
[GUI Plugin] KiNotes -axel- Plugins 0 07-14-2013 06:39 PM
[GUI Plugin] Plugin Updater **Deprecated** kiwidude Plugins 159 06-19-2011 12:27 PM


All times are GMT -4. The time now is 02:47 PM.


MobileRead.com is a privately owned, operated and funded community.