[GUI Plugin] FanFicFare - Page 134

agumon1123 · 04-17-2017, 01:49 PM

Is it just me or is fanficfare not downloading the covers for stories from fanfiction.net now?

example urls:
https://www.fanfiction.net/s/9060154/1/
https://www.fanfiction.net/s/8872460/1/
https://www.fanfiction.net/s/10868522/1/

JimmXinu · 04-17-2017, 02:45 PM

Quote:

Originally Posted by agumon1123

Is it just me or is fanficfare not downloading the covers for stories from fanfiction.net now?

example urls:
https://www.fanfiction.net/s/9060154/1/
https://www.fanfiction.net/s/8872460/1/
https://www.fanfiction.net/s/10868522/1/

Hmmm. They want a Referer for cover fetches. Looks like that may have become broken with the fetch refactoring. I'll have a fix in the next test version.

Thanks for catching that.

JimmXinu · 04-17-2017, 03:19 PM

New test version posted

2017-04-17
- Fix ffnet referer for cover images.
- New site www.webnovel.com - thanks GComyn

MerlinMama · 04-17-2017, 03:28 PM

I haven't been able to download anything from AO3 today. It won't connect. It's possible it's my end, because I can get the website fine, then after FFF fails, I can't reach the website anymore for about half an hour or so. Still figure I should mention.

Spoiler:

I have checked AO3 Status, and there's nothing mentioned

JimmXinu · 04-17-2017, 03:36 PM

Quote:

Originally Posted by MerlinMama

I haven't been able to download anything from AO3 today. It won't connect. It's possible it's my end, because I can get the website fine, then after FFF fails, I can't reach the website anymore for about half an hour or so. Still figure I should mention.

Spoiler:

I have checked AO3 Status, and there's nothing mentioned

Works fine for me.

Could be AO3 blocking you if you've done a lot of downloads lately.

I believe we've also seen problems like that from over-zealous anti-virus programs/firewalls blocking Calibre from making connections.

MerlinMama · 04-17-2017, 04:01 PM

Quote:

Originally Posted by JimmXinu

Works fine for me.

Could be AO3 blocking you if you've done a lot of downloads lately.

I believe we've also seen problems like that from over-zealous anti-virus programs/firewalls blocking Calibre from making connections.

I'll have to check my AV/Firewall, because I haven't been doing much downloading lately. Past six months, 1-2 updated fics from AO3 a week at most, and sometimes none for a few weeks.

Oddly, I downloaded just a random AO3 fic and it worked fine, went back to the ones I was trying and back to timing out. Weird.

mongognom · 04-17-2017, 04:32 PM

Norwegian. I'll try switching to English and see if it works.

Edit: All problems solved. Thank you for the help!

MerlinMama · 04-19-2017, 02:25 PM

Quote:

Originally Posted by MerlinMama

I haven't been able to download anything from AO3 today. It won't connect. It's possible it's my end, because I can get the website fine, then after FFF fails, I can't reach the website anymore for about half an hour or so. Still figure I should mention.

Spoiler:

I have checked AO3 Status, and there's nothing mentioned

This still isn't working for me. Do you have any idea what else I can check for this? I realize it must be a problem on my end, because no one else has reported anything, through the plugin or the site, but I hope someone might have an easy idea.

I'm hesitant to think it's my AV/Firewall (even though I did check what I could), since everything else related to Calibre and FFF seems to be working without a problem. Only downloading a fic from AO3.

My last updated fic from there was Tuesday, April 11th, but I didn't have a problem then. I didn't have any other interaction with the site until Monday of this week, when I first had the downloading problem.

JimmXinu · 04-19-2017, 03:37 PM

Quote:

Originally Posted by MerlinMama

This still isn't working for me. Do you have any idea what else I can check for this?
...

I assume you've already tried restarting calibre and rebooting your computer.

Another idea would be to flush your DNS cache--how to do that depends on your OS--I've occasionally seen Windows and different browsers on Windows caching DNS separately (and differently).

If you can post a debug log, there's a chance that may give some more clues.

It's also vaguely possible that posting/sending me your personal.ini(safe version) to try might help.

MerlinMama · 04-19-2017, 04:06 PM

Quote:

Originally Posted by JimmXinu

I assume you've already tried restarting calibre and rebooting your computer.

Another idea would be to flush your DNS cache--how to do that depends on your OS--I've occasionally seen Windows and different browsers on Windows caching DNS separately (and differently).

If you can post a debug log, there's a chance that may give some more clues.

It's also vaguely possible that posting/sending me your personal.ini(safe version) to try might help.

I have tried restarting Calibre and rebooting (good to mention though, I've been known to overlook this in the past).

I flushed the DNS Cache as well as clearing browser cache, cookies, history, and running CCleaner. I also turned off Firewall temporarily.

Here is debug log:

Spoiler:

Code:

devicePixelRatio: 1.0
logicalDpi: 96.0 x 96.0
physicalDpi: 91.7381489842 x 91.8072289157
Starting up...
DEBUG:    0.0 KoboUtilites::action.py - loading translations
DEBUG:    0.0 KoboUtilites::dialogs.py - loading translations
DEBUG:    0.0 KoboUtilites::action.py - loading translations
FFF: DEBUG: 2017-04-19 15:45:52,872: calibre_plugins.fanficfare_plugin.fff_plugin(207): Plugin FanFicFare macmenuhack file_path:C:\Users\Wenpip\AppData\Roaming\calibre\plugins\fanficfare_macmenuhack.txt
Calibre, and hence Job Spy, was gracefully shut down last time?  True
Last time daemon started:  Wed Jun 01 18:51:54 2016
Last time daemon failed:  never
Total daemon starts inception_to_date:  1
Total daemon failures inception-to-date:  0
source2 missing from uuid dict:  SpanishNormal
source2 missing from uuid dict:  QuarantineAndScrub_Spanish
target2 missing from uuid dict:  CalibreJobSpyTest1
target2 missing from uuid dict:  CalibreJobSpyTest2
target2 missing from uuid dict:  Normal1
target2 missing from uuid dict:  Normal2
target2 missing from uuid dict:  CalibreNormal
target2 missing from uuid dict:  CalibreLibraryCodes
Job Spy has finished initialization...
DEBUG:    2.3 No Kobo Touch, Glo or Mini appears to be connected
DEBUG:    2.3 rebuild_menus - self.supports_ratings=None, self.supports_tiles=None
DEBUG:    2.3 KoboUtilities:set_toolbar_button_tooltip - start: text='None'
DEBUG:    2.3 device_driver_name - could not load extended driver. Exception= No module named kobotouch_extended.device.driver
DEBUG:    2.3 KoboUtilities:set_toolbar_button_tooltip - setting to text='Utilities to use with Kobo ereaders

Driver: KoboTouch'
DEBUG:    2.3 device_driver_name - could not load extended driver. Exception= No module named kobotouch_extended.device.driver
EpubMerge: DEBUG: 2017-04-19 15:45:53,279: calibre_plugins.epubmerge.epubmerge_plugin(152): macmenuhack file_path:C:\Users\Wenpip\AppData\Roaming\calibre\plugins\fanficfare_macmenuhack.txt
Started up in 4.28 seconds with 139 books

Personal.ini (safe)

Spoiler:

Code:

[defaults]
include_subject_tags: category,ships
titlepage_entries: category,characters,status,datePublished,dateUpdated,calibre_cust_added,numChapters,numWords,calibre_cust_pages,site,description
titlepage_no_title_entry:
 <hr/>${value}<br/>

category_label:Fandom
characters_label:Pairing
ships_label:Pairing
description_label:
calibre_cust_added_label:Packaged

datePublished_format: %%B %%d, %%Y
dateUpdated_format:	%%B %%d, %%Y
calibre_cust_added_format: %%B %%d, %%Y

keep_style_attr:false
replace_tags_with_spans:big,small

is_adult:true

add_genre_when_multi_category: Crossover

[epub]
output_css:body { background-color: #%(background_color)s;
        text-align: justify;
        margin: 2%%; }
 pre { font-size: x-small; }
 sml { font-size: small; }
 h1 { text-align: center; }
 h2 { text-align: center; }
 h3 { text-align: center; }
 h4 { text-align: center; 
  font-size: 120%% }
 h5 { text-align: center; }
 h6 { text-align: center; }
 a { text-decoration: none; }
 font1 { font-family: 'Segoe Print';
  font-size: 90%% }
 font2 { font-family: 'Dion';
  font-size: 130%% }
 news { font-family: 'Kirsty'; 
  font-style: italic; }
 flash { font-weight: bold; }
 dream { font-style: italic; }
 vision { font-weight: bold; 
  font-style: italic; }
 dbl { text-decoration: underline; 
  border-bottom: 1px solid black; }
 .CI {
     text-align:center;
     margin-top:0px;
     margin-bottom:0px;
     padding:0px;
     }
 .center   {text-align: center;}
 .centered {text-align: center;
            font-weight: bold;}
 .cover    {text-align: center;}
 .full     {width: 100%%; }
 .quarter  {width: 25%%; }
 .smcap    {font-variant: small-caps;}
 .u        {text-decoration: underline;}
 .bold     {font-weight: bold;}
 .lini     {margin-left:25px;
            margin-right:35px;}

newline:

# You can't see it, but there's a space on the line above. 

chapter_start:<?xml version='1.0' encoding='utf-8'?>
 <html xmlns="http://www.w3.org/1999/xhtml">
 <head>
 <title>${chapter}</title>
 <link href="stylesheet.css" type="text/css" charset="UTF-8" rel="stylesheet"/>
 </head>
 <body>
 <h3>${chapter}</h3>
 %(newline)s
 %(newline)s

# Next is Titlepage.
titlepage_start:<?xml version="1.0" encoding="UTF-8"?>
 <html xmlns="http://www.w3.org/1999/xhtml">
 <head>
 <title>${title} by ${author}</title>
 <link href="stylesheet.css" type="text/css" charset="UTF-8" rel="stylesheet"/>
 </head>
 <body>
 <h2><a href="${storyUrl}">${title}</a><br/> by<br/> ${authorHTML}</h2>
 <div><br/>
 %(newline)s

[www.squidge.org/peja]
category_label:Fandom
titlepage_entries: category,ships,status,datePublished,dateUpdated,calibre_cust_added,numChapters,numWords,calibre_cust_pages,site,description

[archiveofourown.org]
username:XXXXXXXX
password:XXXXXXXX
is_adult:true
keep_summary_html:true
titlepage_entries: category,ships,status,datePublished,dateUpdated,calibre_cust_added,numChapters,numWords,calibre_cust_pages,site,description
replace_metadata:
 category=>Harry Potter - J\. K\. Rowling=>Harry Potter
 category=>Harry Potter - Fandom=>Harry Potter
 category=>Twilight Series - All Media Types=>Twilight
 ships=>Draco Malfoy/Harry Potter=>Draco &amp; Harry\,Drarry
 ships=>Severus Snape/Harry Potter=>Severus &amp; Harry\,Snarry
 
[www.tthfanfic.org]
is_adult:true

username:XXXXXXXX
password:XXXXXXXX

[fanfiction.net]
include_subject_tags: category,ships,characters

## Note that ini format requires % to be escaped as %%.
dateCreated_format:	%%B %%d, %%Y
datePublished_format: %%B %%d, %%Y
dateUpdated_format:	%%B %%d, %%Y

slow_down_sleep_time:4

never_make_cover: true

[overrides]
## default varies by site.  Set true here to force all sites to
## collect series.
#collect_series: true

I might try waiting a week before trying again...unless you have any other ideas from the logs.

JimmXinu · 04-19-2017, 05:40 PM

Quote:

Originally Posted by MerlinMama

I have tried restarting Calibre and rebooting (good to mention though, I've been known to overlook this in the past).

Here is debug log:

Personal.ini (safe)

I might try waiting a week before trying again...unless you have any other ideas from the logs.

Ah. I meant a debug log while trying to do a download that fails. And the debug log usually reports OS details at the beginning, too. You can send it to me in PM if you prefer.

Although seeing Job Spy does suggest another long-shot possibility: plugin conflict. You might try disabling all your other plugins. Or try using a portable Calibre install with only FFF.

I tried your personal.ini--worked fine, so nothing wrong there.

Atherton25 · 04-19-2017, 10:36 PM

Alright! Another Translation/Xianxia site! Thanks GComyn!

There are a couple of small things that need to be noted:
#1: I got no description/synopsis at all, not sure why, it's available on the site. I verified it wasn't my problem using my test calibre database with a blank/default personal.ini to make sure it wasn't something from one of my changes.

The next two items are more nit-picky than anything, so feel free to ignore them. I may be able to solve them myself (with some help from JimmXinu) and in the future it would be nice to have the information.

#2: All chapters are given the label 'chapter' (incorrectly capitalized).
#3: Apparently if the chapters aren't titled their title becomes 'Blank Title' on the website, so for example:
https://www.webnovel.com/book/6831845202000705 every chapter is called Blank Title on the website, and so they are labeled that in the download.

I did try to write a metadata correction to simply fix these, but I don't know the correct location/name so I can relabel chapters. I thought it might be something like
chapter_title=>chapter=>Chapter
chapter_title=>Blank Title=>
(I did some variations with regex but that was the starting point) It didn't work. I'm guessing I'm calling/using the wrong identifier, but I could just be writing it wrong. Based on defaults.ini I know that chapter_title_strip_pattern strips out certain things. I had considered modifying the strip pattern to include 'Blank Title' but I'd rather leave that pattern alone instead of breaking it over and over again trying to get it right.

Also, and this has nothing to do with the new adapter, just a question that's been niggling at the back of my brain for a while. @JimmXinu, why did you decide to use a title page instead of a custom metadata jacket? Technically all the information is metadata, even though it's not normal metadata. And using a custom jacket would mean making changes to the title page may be easier. It's nothing I'm trying to ask for or anything, just a curious thought that wont leave me alone.

JimmXinu · 04-20-2017, 01:40 PM

Quote: