New test version posted
2017-05-26
- wattpad - Separate category from tags, include rags in genre by default, use larger cover image. - Thanks, botmtl!
- wattpad - need to pass utf8FromSoup a soup...
- Add comma_entries option to add commas to metadata entries--numbers only.
- wattpad - add defaults slow_down_sleep_time:2 and comma reads.
- Update translations.
@Atherton25 - I think your slow downs and other download issues had to do with throttling and large numbers of images. I've added a default
slow_down_sleep_time for wattpad.
BTW, yes, I generally appreciate debug logs with problems, but +500M of log is a bit excessive when editing quotes. At least attach it as a file next time it's that large, please.
Quote:
Originally Posted by Atherton25
I have noticed that I despise the way they allow weird ass characters in titles - like check marks and smiley faces.
|
replace_metadata is your friend.
You might consider sharing your solution when you have one you are happy with.
Quote:
Originally Posted by Atherton25
One other thing I was going to ask - and if it's a complicated thing ignore it, I'll write some regex for the conversion to fix it - but can the read count have commas in it?
|
I've added a
comma_entries ini option. I'll default wattpad
reads to use it. Not sure why I haven't done that before.
Quote:
Originally Posted by Atherton25
Just for my info, is that a sleep between stories, chapters? and is it seconds or something else?
|
Seconds, between network requests. It's not applied when a previously fetched/cached page is used again.
Quote:
Originally Posted by Atherton25
As for the one that I put in the log, what's strange is I'm pretty sure when I had that cover option turned off it downloaded pretty quickly. Only when I turned it on did it take forever to download. I'll have to recheck that, and why would just a forced update redownload all those pictures if it already had the story? Shouldn't it just get the cover (which it didn't have) and the metadata, since all the chapters were already there? I didn't actually check the story before doing a forced update. Hmm. and no, not an overwrite, just forced update.
|
The lines like this:
Code:
FFF: DEBUG: 2017-05-26 11:54:16,605: story.py(156): no_convert_image url:https://em.wattpad.com/excessivelylongnumber - no known extension
...don't indicate downloading again, just that without a file extension FFF is going to assume jpg. Only occurs when
no_image_processing:true Otherwise, images are converted to jpg because that's all some reader devices understand.
I'm not seeing anything unexpected happen now, but then I'm only looking at the first few chapters.
So I'm chalking what ever else was going on up to throttling effects; unless you can replicate it when you're not being throttled.