[GUI Plugin] FanFicFare - Page 17

McHunter · 05-30-2015, 02:34 PM

Hello...

Quote:

Originally Posted by JimmXinu

My level of sympathy for 'I can't do more than 400 updates at once' is rather limited. .

I don't select my whole library, which are 3500+ Books and "only" 2500+ are on FFN, but with 1000 I could start the Update for my most recent FFN Books and do somthing other (like grocery shopping), and I do this mostly onca a month, and for this 1000 are mostly 75-150 Updated.
And for most of them I don't get Alerts...

The other 1000+ Books are on other sites.

Quote:

Originally Posted by JimmXinu

With the addition of the feature to pass Calibre metadata into FanFicFare while it's processing, you may have bumped into a limit on the amount of data that can be passed to a background process.

Since downgrading to FFF 2.2.5 raises the limit to again 1000, I doubt that this is the case.

Quote:

Originally Posted by JimmXinu

But IMO, it's also possible one of the sites you're hitting so hard is adding slow downs.

Possible, but 240+ Minutes? And only after the Metadata Download?

Quote:

Originally Posted by JimmXinu

If you want, you can try going back to an earlier version.

Thanks, Done That, Works.

cryzed · 05-30-2015, 02:47 PM

Just because only 75-150 stories are actually updating, doesn't mean that there isn't any traffic caused for the other stories. There are hard coded sleep times for some adapters, IIRC fanfiction.net is one of them -- if there weren't, you would very quickly see errors regarding incomplete responses, aborted connections or other HTTP errors.

Instead of wasting several hours of time and gigabytes of traffic once or twice a month, you might as well take the time to set up alerts for all your fanfiction.net stories once, and then only take 1/10000 of the time and traffic when using the email feature.

Several pages back I've also published a script that should be able to do this process for you unsupervised in a fraction of the time. If you are really interested in getting this to work, figure out how to use it. The downgrading "solution" will only work for the amount of time that fanfiction.net's site layout doesn't change (or any of the other sites you are using FFF for), as soon as it does you are out of luck, unless you want to manually update your own branch of FFF.

Amalthia · 05-30-2015, 05:30 PM

Quote:

Originally Posted by JimmXinu

You could use replace_metadata and/or include/exclude_metadata_pre/post to reduce the categorys down to one. IE, replace both Avengers and Captain America with Marvel.

Hi, I kind of want to keep Captain America and Avengers and Marvel, I just don't need the Crossover genre for anything tagged with Marvel.

I'll give this some thought.

Thanks!

AmberDragon · 05-30-2015, 08:59 PM

I'm having a little bit of trouble with replace_metadata in my defaults. It was working fine, but I added

Code:

add_genre_when_multi_category: Crossover
include_metadata_pre:
 genre==Crossover

to my defaults and now the replace_metadata in the defaults sections doesn't seem to be working. It does work from the site-specific sections, though. My defaults section looks like this:

Code:

[defaults]
replace_hr: true
include_subject_tags: category,genre
add_genre_when_multi_category: Crossover
titlepage_entries: series,category,genre,dateUpdated,rating,warnings,numChapters,numWords,site,description
do_update_hook:true
include_metadata_pre:
 genre==Crossover
replace_metadata:
 category=>(.*) [Ss]eries=>\1
 category=>(.*), [Tt]he=>The \1
 category=>(Crossovers|X-over|X-overs|Xover)=>Crossover
 category=>(Buffy\: the vampire slayer|BTVS)=>Buffy the Vampire Slayer
 category=>Stargate(.*)=>Stargate
 The Devil Wears Prada=>Devil Wears Prada
 The Fast and the Furious=>Fast and the Furious

Any ideas on where I went wrong?

JimmXinu · 05-30-2015, 10:37 PM

Quote:

Originally Posted by AmberDragon

I'm having a little bit of trouble with replace_metadata in my defaults. It was working fine, but I added

Code:

add_genre_when_multi_category: Crossover
include_metadata_pre:
 genre==Crossover

...
Any ideas on where I went wrong?

If that's the only include_medata line you have, you've stripping everything out: 'Include' in this case really means 'include ONLY these.' If a metadata type has lines in include, only those values will be kept, all others are discarded.(wiki page)

AmberDragon · 05-30-2015, 11:49 PM

Ok, I thought that specifying "genre" would limit the include to only the genre data. What do I need to add to keep the category data passing through?

JimmXinu · 05-31-2015, 12:01 PM

Quote:

Originally Posted by AmberDragon

Ok, I thought that specifying "genre" would limit the include to only the genre data. What do I need to add to keep the category data passing through?

Good point. You're right, it could only effect genre. My apologies, I didn't look at your posted replace_metadata very closely--I just assumed you were talking about genre.

I don't know why add_genre_when_multi_category or include_metadata_pre would effect your category replace_metadata. They don't when I test it.

But, if you have site specific replace_metadata clauses, they will override the one in [defaults]. To include both use add_to_replace_metadata in [defaults] instead.

AmberDragon · 05-31-2015, 02:58 PM

Thanks, the add_to_replace_metadata fixed it.

MerlinMama · 06-01-2015, 03:40 PM

This has probably been addressed, but I did a search and couldn't find anything.

I get 2 'errors' when i save personal.ini file, I believe it is 2 sites that are no longer supported. Do I simply erase them completely from my ini file, or do something else with them.

[www.twiwrite.net] and [www.thewriterscoffeeshop.com]

if I do erase the, i assume it is the entire block? Thanks in advance, and sorry if this has been answered before.

JimmXinu · 06-01-2015, 04:35 PM

Quote:

Originally Posted by MerlinMama

...
I get 2 'errors' when i save personal.ini file, I believe it is 2 sites that are no longer supported. Do I simply erase them completely from my ini file, or do something else with them.
...
if I do erase the, i assume it is the entire block?

Yes and yes.

MerlinMama · 06-01-2015, 06:16 PM

Thanks. I'd rather double check as I once did the 'obvious' on something (unrelated) and it was the totally wrong thing

Elaine T · 06-03-2015, 07:35 PM

This is my first time on this forum, I'm looking for help with downloads from AO3.

The error in the log is:

http://archiveofourown.org/works/780061?view_adult=true

Status
Title
Author
Comment
URL
Bad
Unknown
Unknown
IncompleteRead(0 bytes read, 2 more expected)
http://archiveofourown.org/works/780061

I'm running FFF v 2.2.6. Any ideas? I tried to figure out how to get the download to run in debug mode, but couldn't find it easily. Help?

i also couldn't find a way to post a brand new post without replying. IF this should be elsewhere, I apologize.

JimmXinu · 06-03-2015, 08:57 PM

Quote:

Originally Posted by Elaine T

...
IncompleteRead(0 bytes read, 2 more expected)
http://archiveofourown.org/works/780061

I'm running FFF v 2.2.6. Any ideas? I tried to figure out how to get the download to run in debug mode, but couldn't find it easily. Help?

I don't have any problems downloading that story.

You can run in debug mode from the Calibre Preferences drop down. See attached.

Firedancer885 · 06-04-2015, 05:21 AM

Jimm, help please. I got this error:
Állapot
Cím
Szerző
Comment
URL
Bad
Unknown
Unknown
sequence item 0: expected string, list found
http://voracity2.e-fic.com/viewstory.php?sid=90

Spoiler:

Code:

calibre 2.29  isfrozen: True is64bit: False
Linux-3.19.0-18-generic-i686-athlon-with-debian-jessie-sid Linux ('32bit', 'ELF')
('Linux', '3.19.0-18-generic', '#18-Ubuntu SMP Tue May 19 18:30:59 UTC 2015')
Python 2.7.9
Linux: ('debian', 'jessie/sid', '')
Successfully initialized third party plugins: View Manager && EpubSplit && Reading List && Count Pages && Generate Cover && FanFicFare && EpubMerge
Starting up...
libpng warning: iCCP: Not recognizing known sRGB profile that has been edited
FFF:DEBUG:calibre_plugins.fanficfare_plugin.fff_plugin(184):Plugin FanFicFare macmenuhack file_path:/home/hosszu/.config/calibre/plugins/fanficfare_macmenuhack.txt
macmenuhack file_path:/home/hosszu/.config/calibre/plugins/fanfictiondownloader_macmenuhack.txt
Started up in 27.73 seconds with 7445 books

FFF:DEBUG:calibre_plugins.fanficfare_plugin.fff_plugin(828):FanFicFare v2.2.6
FFF:DEBUG:calibre_plugins.fanficfare_plugin.fff_plugin(864):url:http://voracity2.e-fic.com/viewstory.php?sid=90
FFF:DEBUG:calibre_plugins.fanficfare_plugin.fanficfare.adapters.base_adapter(294):#####################################
pagecache MISS: http://voracity2.e-fic.com/viewstory.php?sid=90&index=1
FFF:DEBUG:calibre_plugins.fanficfare_plugin.fanficfare.adapters.base_adapter(294):#####################################
pagecache MISS: http://voracity2.e-fic.com/viewstory.php?sid=90&index=1&ageconsent=ok&warning=4
FFF:ERROR:calibre_plugins.fanficfare_plugin.dialogs(653):Exception: {u'author': [u'Unknown'], u'end': None, u'comments': u'', u'listorder': 0, u'site': 'voracity2.e-fic.com', u'pubdate': None, u'begin': None, u'title': u'Unknown', u'calibre_id': None, u'good': False, u'comment': u'sequence item 0: expected string, list found', u'added': False, u'all_metadata': {'series': '', 'formatname': '', 'readings': u'276', 'rating': u'FRT', 'authorHTML': u"<a class='authorlink' href='http://voracity2.e-fic.com/viewuser.php?uid=615'>Voracity2</a>", 'siteabbrev': u'voe', 'dateCreated': u'2015-06-04 10:12:12', 'seriesUrl': '', 'language': '', 'formatext': '', 'site': u'voracity2.e-fic.com', 'category': u'Buffy Stories & Crossovers, Avengers, Other Movie', 'ships': u'', 'storyUrl': u'http://voracity2.e-fic.com/viewstory.php?sid=90', 'cover_image': '', 'authorId': u"[u'615']", 'status': u'Completed', 'authorUrl': u'http://voracity2.e-fic.com/viewuser.php?uid=615', 'numWords': u'136,863', 'langcode': '', 'description': u"Xander Harris and Darcy Lewis have insane lives.  Adding in some...higher beings with ideas about their bloodlines isn't going to make it any less insane.", 'warnings': u'', 'characters': u'', 'storyId': u'90', 'output_css': '', 'extratags': u'FanFiction', 'dateUpdated': u'2015-06-03', 'numChapters': u'8', 'seriesHTML': '', 'author': u'Voracity2', 'datePublished': u'2015-06-03', 'reviews': u'0', 'lastupdate': u'Last Update Year/Month: 2015/06, Last Update: 2015/06/03', 'genre': u'', 'version': u'FanFicFare v2.2.6', 'title': u'Weird Old Lines.'}, u'url': 'http://voracity2.e-fic.com/viewstory.php?sid=90', u'author_sort': [u'Unknown']}:sequence item 0: expected string, list found
Traceback (most recent call last):
  File "calibre_plugins.fanficfare_plugin.dialogs", line 643, in do_loop
  File "calibre_plugins.fanficfare_plugin.fff_plugin", line 1026, in prep_download_loop
  File "calibre_plugins.fanficfare_plugin.fanficfare.story", line 566, in dump_html_metadata
TypeError: sequence item 0: expected string, list found

JimmXinu · 06-04-2015, 10:59 AM

Quote:

Originally Posted by Firedancer885

Jimm, help please. I got this error:
Állapot
Cím
Szerző
Comment
URL
Bad
Unknown
Unknown
sequence item 0: expected string, list found
http://voracity2.e-fic.com/viewstory.php?sid=90

Spoiler:

Code:

calibre 2.29  isfrozen: True is64bit: False
Linux-3.19.0-18-generic-i686-athlon-with-debian-jessie-sid Linux ('32bit', 'ELF')
('Linux', '3.19.0-18-generic', '#18-Ubuntu SMP Tue May 19 18:30:59 UTC 2015')
Python 2.7.9
Linux: ('debian', 'jessie/sid', '')
Successfully initialized third party plugins: View Manager && EpubSplit && Reading List && Count Pages && Generate Cover && FanFicFare && EpubMerge
Starting up...
libpng warning: iCCP: Not recognizing known sRGB profile that has been edited
FFF:DEBUG:calibre_plugins.fanficfare_plugin.fff_plugin(184):Plugin FanFicFare macmenuhack file_path:/home/hosszu/.config/calibre/plugins/fanficfare_macmenuhack.txt
macmenuhack file_path:/home/hosszu/.config/calibre/plugins/fanfictiondownloader_macmenuhack.txt
Started up in 27.73 seconds with 7445 books

FFF:DEBUG:calibre_plugins.fanficfare_plugin.fff_plugin(828):FanFicFare v2.2.6
FFF:DEBUG:calibre_plugins.fanficfare_plugin.fff_plugin(864):url:http://voracity2.e-fic.com/viewstory.php?sid=90
FFF:DEBUG:calibre_plugins.fanficfare_plugin.fanficfare.adapters.base_adapter(294):#####################################
pagecache MISS: http://voracity2.e-fic.com/viewstory.php?sid=90&index=1
FFF:DEBUG:calibre_plugins.fanficfare_plugin.fanficfare.adapters.base_adapter(294):#####################################
pagecache MISS: http://voracity2.e-fic.com/viewstory.php?sid=90&index=1&ageconsent=ok&warning=4
FFF:ERROR:calibre_plugins.fanficfare_plugin.dialogs(653):Exception: {u'author': [u'Unknown'], u'end': None, u'comments': u'', u'listorder': 0, u'site': 'voracity2.e-fic.com', u'pubdate': None, u'begin': None, u'title': u'Unknown', u'calibre_id': None, u'good': False, u'comment': u'sequence item 0: expected string, list found', u'added': False, u'all_metadata': {'series': '', 'formatname': '', 'readings': u'276', 'rating': u'FRT', 'authorHTML': u"<a class='authorlink' href='http://voracity2.e-fic.com/viewuser.php?uid=615'>Voracity2</a>", 'siteabbrev': u'voe', 'dateCreated': u'2015-06-04 10:12:12', 'seriesUrl': '', 'language': '', 'formatext': '', 'site': u'voracity2.e-fic.com', 'category': u'Buffy Stories & Crossovers, Avengers, Other Movie', 'ships': u'', 'storyUrl': u'http://voracity2.e-fic.com/viewstory.php?sid=90', 'cover_image': '', 'authorId': u"[u'615']", 'status': u'Completed', 'authorUrl': u'http://voracity2.e-fic.com/viewuser.php?uid=615', 'numWords': u'136,863', 'langcode': '', 'description': u"Xander Harris and Darcy Lewis have insane lives.  Adding in some...higher beings with ideas about their bloodlines isn't going to make it any less insane.", 'warnings': u'', 'characters': u'', 'storyId': u'90', 'output_css': '', 'extratags': u'FanFiction', 'dateUpdated': u'2015-06-03', 'numChapters': u'8', 'seriesHTML': '', 'author': u'Voracity2', 'datePublished': u'2015-06-03', 'reviews': u'0', 'lastupdate': u'Last Update Year/Month: 2015/06, Last Update: 2015/06/03', 'genre': u'', 'version': u'FanFicFare v2.2.6', 'title': u'Weird Old Lines.'}, u'url': 'http://voracity2.e-fic.com/viewstory.php?sid=90', u'author_sort': [u'Unknown']}:sequence item 0: expected string, list found
Traceback (most recent call last):
  File "calibre_plugins.fanficfare_plugin.dialogs", line 643, in do_loop
  File "calibre_plugins.fanficfare_plugin.fff_plugin", line 1026, in prep_download_loop
  File "calibre_plugins.fanficfare_plugin.fanficfare.story", line 566, in dump_html_metadata
TypeError: sequence item 0: expected string, list found

The latest posted test version already fixes that problem.

06-03-2015, 07:35 PM	#252
Elaine T Connoisseur Posts: 71 Karma: 33664 Join Date: Jun 2015 Device: kindle paperwhite	Problem with download from AO3 This is my first time on this forum, I'm looking for help with downloads from AO3. The error in the log is: http://archiveofourown.org/works/780061?view_adult=true Status Title Author Comment URL Bad Unknown Unknown IncompleteRead(0 bytes read, 2 more expected) http://archiveofourown.org/works/780061 I'm running FFF v 2.2.6. Any ideas? I tried to figure out how to get the download to run in debug mode, but couldn't find it easily. Help? i also couldn't find a way to post a brand new post without replying. IF this should be elsewhere, I apologize.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
[GUI Plugin] KindleUnpack - The Plugin	DiapDealer	Plugins	527	08-15-2025 02:36 PM
[GUI Plugin] Open With	kiwidude	Plugins	404	02-21-2025 06:42 AM
[GUI Plugin] Marvin XD	Philantrop	Plugins	126	01-29-2017 01:48 PM
[GUI Plugin] KiNotes	-axel-	Plugins	0	07-14-2013 07:39 PM
[GUI Plugin] Plugin Updater Deprecated	kiwidude	Plugins	159	06-19-2011 01:27 PM

05-30-2015, 02:47 PM	#242
cryzed Evangelist Posts: 408 Karma: 1050547 Join Date: Mar 2011 Device: Kindle Oasis 2	Just because only 75-150 stories are actually updating, doesn't mean that there isn't any traffic caused for the other stories. There are hard coded sleep times for some adapters, IIRC fanfiction.net is one of them -- if there weren't, you would very quickly see errors regarding incomplete responses, aborted connections or other HTTP errors. Instead of wasting several hours of time and gigabytes of traffic once or twice a month, you might as well take the time to set up alerts for all your fanfiction.net stories once, and then only take 1/10000 of the time and traffic when using the email feature. Several pages back I've also published a script that should be able to do this process for you unsupervised in a fraction of the time. If you are really interested in getting this to work, figure out how to use it. The downgrading "solution" will only work for the amount of time that fanfiction.net's site layout doesn't change (or any of the other sites you are using FFF for), as soon as it does you are out of luck, unless you want to manually update your own branch of FFF.

05-30-2015, 11:49 PM	#246
AmberDragon Member Posts: 20 Karma: 10 Join Date: May 2012 Device: Nook Color	Ok, I thought that specifying "genre" would limit the include to only the genre data. What do I need to add to keep the category data passing through?

05-31-2015, 02:58 PM	#248
AmberDragon Member Posts: 20 Karma: 10 Join Date: May 2012 Device: Nook Color	Thanks, the add_to_replace_metadata fixed it.

06-01-2015, 03:40 PM	#249
MerlinMama Evangelist Posts: 498 Karma: 32554 Join Date: May 2014 Location: Canada Device: Kobo Sage	This has probably been addressed, but I did a search and couldn't find anything. I get 2 'errors' when i save personal.ini file, I believe it is 2 sites that are no longer supported. Do I simply erase them completely from my ini file, or do something else with them. [www.twiwrite.net] and [www.thewriterscoffeeshop.com] if I do erase the, i assume it is the entire block? Thanks in advance, and sorry if this has been answered before.

06-01-2015, 06:16 PM	#251
MerlinMama Evangelist Posts: 498 Karma: 32554 Join Date: May 2014 Location: Canada Device: Kobo Sage	Thanks. I'd rather double check as I once did the 'obvious' on something (unrelated) and it was the totally wrong thing

Advert

Advert