Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre > Plugins

Notices

Reply
 
Thread Tools Search this Thread
Old 01-17-2020, 08:27 AM   #3916
Tanjamuse
Wizard
Tanjamuse , Klaatu Barada Niktu!Tanjamuse , Klaatu Barada Niktu!Tanjamuse , Klaatu Barada Niktu!Tanjamuse , Klaatu Barada Niktu!Tanjamuse , Klaatu Barada Niktu!Tanjamuse , Klaatu Barada Niktu!Tanjamuse , Klaatu Barada Niktu!Tanjamuse , Klaatu Barada Niktu!Tanjamuse , Klaatu Barada Niktu!Tanjamuse , Klaatu Barada Niktu!Tanjamuse , Klaatu Barada Niktu!
 
Posts: 1,282
Karma: 5306
Join Date: Jan 2014
Device: none
Quote:
Originally Posted by Inanis View Post
erm the issue is still happening? like it been 2 months and Ao3 staff tells me they no longer have issues but im still getting "HTTP Error 429: Too Many Requests"

PS: does slow_down_sleep_time actually does something noticeable? i started at 1.. but then 3 , 5 , 10 and started doubling it.. till i reached 50000 and i didnt see any increase in the time it took to download stuff? (increased it to 1500000 to see if it did anything.. was thinking it would take like 4 hrs to download.. but it took the same time as 5? and still had the same error)

been thinking they might have a cap per day?maybe i should try using a VPN.
Let me know what you find out ... I haven't heard anything about solving the server problems.
Tanjamuse is online now   Reply With Quote
Old 01-17-2020, 09:26 AM   #3917
Inanis
Junior Member
Inanis began at the beginning.
 
Posts: 2
Karma: 10
Join Date: Jan 2020
Device: none
Quote:
Originally Posted by Tanjamuse View Post
Let me know what you find out ... I haven't heard anything about solving the server problems.

they posted on dec 19 that it got fixed and i didnt have any issues unless downloading 82+ links for an anthology

PS: using a VPN actually allowed me to download w/o any errors.. so my guess is that they have either a daily/weekly/something-else cap per IP..
Inanis is offline   Reply With Quote
Advert
Old 01-17-2020, 12:44 PM   #3918
JimmXinu
Plugin Developer
JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.
 
JimmXinu's Avatar
 
Posts: 3,919
Karma: 637184
Join Date: Dec 2011
Location: Midwest USA
Device: Kindle Voyage
What you can do in FFF is limited. All of that, given the network errors, is really outside of FFF's direct control.

Given the reports, I agree that it is possible, even probable, that AO3 is throttling users in some way--which is honestly their right.

slow_down_sleep_time as previously discussed, adds a sleep between network requests. Inanis, I suspect you're putting slow_down_sleep_time in the wrong section or something.

Another key setting for AO3 specifically that is on by default is use_view_full_work:

Code:
[archiveofourown.org]
## AO3 offers a page to download the entire work at once.  This can
## save a lot of network traffic if there are many chapters.
## True by built-in default.
use_view_full_work:true
Turning use_view_full_work off will cause more network requests--one per chapter, but fewer database hits per network request. If AO3 is throttling by network hits, turning this off will only make it worse, but it is something to be aware of. IE, this is why total download time isn't at least sleep time X number of chapters for AO3 downloads.

With AO3 you also have the option of downloading an EPUB directly from the site. So if you have troubles downloading with FFF, that's always an option.
JimmXinu is offline   Reply With Quote
Old 01-17-2020, 09:49 PM   #3919
JimmXinu
Plugin Developer
JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.
 
JimmXinu's Avatar
 
Posts: 3,919
Karma: 637184
Join Date: Dec 2011
Location: Midwest USA
Device: Kindle Voyage
New Test Version Posted

2020-01-17
- Use storyUrl from metadata for checking library, for those sites that make canonical storyUrl difficult, like adapter_literotica. Closes #461. Plugin only.
JimmXinu is offline   Reply With Quote
Old 01-21-2020, 01:41 PM   #3920
unicornzvi
Junior Member
unicornzvi began at the beginning.
 
Posts: 4
Karma: 10
Join Date: Nov 2019
Device: Linux laptop
An issue I noted with some threads on Spacebattles.com is that when selecting "download from URLs" it will appear to download the metadata and start downloading the story normally, but then stall at 1% and not do anything. An example of a thread with this problem is https://forums.spacebattles.com/thre...amurai.653374/

Does anyone know why this happens? Or ideally how to get the thread to ePub despite the issue?
unicornzvi is offline   Reply With Quote
Advert
Old 01-21-2020, 02:06 PM   #3921
JimmXinu
Plugin Developer
JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.
 
JimmXinu's Avatar
 
Posts: 3,919
Karma: 637184
Join Date: Dec 2011
Location: Midwest USA
Device: Kindle Voyage
unicornzvi - The percent meter only advances by whole downloads--not how far along an individual download is. So when downloading one book, it's going to stay at 1% until it jumps to done.

How long are you waiting for it?

That thread has some 600+ threadmarks spread across several categories, it's going to take time to download. It took ~10 minutes for me.
JimmXinu is offline   Reply With Quote
Old 01-21-2020, 02:25 PM   #3922
unicornzvi
Junior Member
unicornzvi began at the beginning.
 
Posts: 4
Karma: 10
Join Date: Nov 2019
Device: Linux laptop
Quote:
Originally Posted by JimmXinu View Post
How long are you waiting for it?
20+ minutes. I'll give it another try, maybe I didn't give it enough time.
unicornzvi is offline   Reply With Quote
Old 01-21-2020, 02:43 PM   #3923
JimmXinu
Plugin Developer
JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.
 
JimmXinu's Avatar
 
Posts: 3,919
Karma: 637184
Join Date: Dec 2011
Location: Midwest USA
Device: Kindle Voyage
Quote:
Originally Posted by unicornzvi View Post
20+ minutes. I'll give it another try, maybe I didn't give it enough time.
60 minutes wouldn't surprise me.

There are lots of factors--slow_down_sleep_time setting, site traffic, time of day, network latency, possible throttling on the site, if you turner off use_reader_mode, it will be ~10x longer, if you are using always_login and have change posts per reader page, etc etc.

Also, run Calibre in debug mode (Preferences drop down) to get more output.
JimmXinu is offline   Reply With Quote
Old 01-23-2020, 11:10 AM   #3924
Mymy
Junior Member
Mymy began at the beginning.
 
Posts: 1
Karma: 10
Join Date: Jan 2020
Device: Kindle Paperwhite 4
Hi Jimm!

I've been meaning to ask - there seems to be an ongoing problem getting stories from wattpad.com. Whenever I enter a URL (e.g: https://www.wattpad/XXXXXX) I get an error message which says "No valid story URLs entered", even though the link is in the correct format.

I have the latest Calibre and FanFicFare versions.

Thank you kindly!
Mymy is offline   Reply With Quote
Old 01-23-2020, 12:55 PM   #3925
JimmXinu
Plugin Developer
JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.
 
JimmXinu's Avatar
 
Posts: 3,919
Karma: 637184
Join Date: Dec 2011
Location: Midwest USA
Device: Kindle Voyage
Quote:
Originally Posted by Mymy View Post
I've been meaning to ask - there seems to be an ongoing problem getting stories from wattpad.com. Whenever I enter a URL (e.g: https://www.wattpad/XXXXXX) I get an error message which says "No valid story URLs entered", even though the link is in the correct format.
I'm not seeing that happen with a story from my 'test all sites' library and a randomly selected story:

https://www.wattpad.com/story/91590087
https://www.wattpad.com/story/193424...ce-paul-lahote

Please include at least one story URL that exhibits the problem whenever reporting any issue.
JimmXinu is offline   Reply With Quote
Old 01-23-2020, 09:46 PM   #3926
greenskye
Junior Member
greenskye began at the beginning.
 
Posts: 2
Karma: 10
Join Date: Feb 2010
Device: none
What would be the chances of getting patreon as a supported site? Just taking the contents of each post as a chapter. I'm supporting several authors who post all chapters early on patreon. I want to keep supporting them, but the reading experience isn't great.
greenskye is offline   Reply With Quote
Old 01-23-2020, 11:03 PM   #3927
JimmXinu
Plugin Developer
JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.
 
JimmXinu's Avatar
 
Posts: 3,919
Karma: 637184
Join Date: Dec 2011
Location: Midwest USA
Device: Kindle Voyage
Re: Patreon

Unlikely. I am very hesitant to do anything with pay sites.

Generally, I figure that anyone writing fanfiction doesn't have a lot of moral ground to complain about somebody scraping a copy of their writing to save.

Original works, especially ones you are paying for, are another matter.

I don't even want to talk about the can of worms that is people taking payment for writing fanfic. Is there enough disconnect between Patreon donations and making money off other people's IP? I don't know, but I fear that will end in tears lawsuits some day.

Plus, my experience is that fanfic writers supported by Patreon donations still post to open sites.
JimmXinu is offline   Reply With Quote
Old 01-24-2020, 11:54 PM   #3928
jemimahj
Junior Member
jemimahj began at the beginning.
 
Posts: 4
Karma: 10
Join Date: Dec 2019
Device: ipad 7th gen
I'm having an issue with the "no valid story urls" as well, only on ao3. It's happening with any links that are www.ao3.org but www.archiveofourown.org links work without any issues. An example of a fit that doesn't work for me is https://www.ao3.org/works/22396777 Both Calibre and the plugin are updated to latest version. Thanks!
jemimahj is offline   Reply With Quote
Old 01-25-2020, 12:32 AM   #3929
JimmXinu
Plugin Developer
JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.
 
JimmXinu's Avatar
 
Posts: 3,919
Karma: 637184
Join Date: Dec 2011
Location: Midwest USA
Device: Kindle Voyage
New Test Version Posted

2020-01-24
- Add more domains for AO3.

Quote:
Originally Posted by jemimahj View Post
I'm having an issue with the "no valid story urls" as well, only on ao3. It's happening with any links that are www.ao3.org but www.archiveofourown.org links work without any issues. ...
Probably because I've never seen ao3.org before and therefore FFF doesn't support it. When did that become a thing?

NSLookup shows ao3.org is indeed owned by archiveofourown.org.

And from their SSL cert (and past experience), it looks like we should accept:
  • archiveofourown.org
  • archiveofourown.com
  • archiveofourown.net
  • download.archiveofourown.org
  • ao3.org

Plus www. versions. archiveofourown.net is also new.

Interestingly, ao3.org immediately redirects to archiveofourown.org. But it seems to be the only one--not even www.ao3.org does it. Which means your cookies and login, etc don't carry over between domain names.
JimmXinu is offline   Reply With Quote
Old 01-25-2020, 07:42 PM   #3930
jemimahj
Junior Member
jemimahj began at the beginning.
 
Posts: 4
Karma: 10
Join Date: Dec 2019
Device: ipad 7th gen
Quote:
Probably because I've never seen ao3.org before and therefore FFF doesn't support it. When did that become a thing?
Not sure when it started, this is the first time I've seen it, seems like it's hit or miss? I tried doing a find/replace with ao3/archiveofourown before I posted and it didn't work, but I had the www. in there. I just tried without the www and was able to download from Calibre with no issues. Thanks!
jemimahj is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
[GUI Plugin] KindleUnpack - The Plugin DiapDealer Plugins 423 02-01-2020 08:40 PM
[GUI Plugin] Open With kiwidude Plugins 336 09-11-2019 09:54 PM
[GUI Plugin] Marvin XD Philantrop Plugins 126 01-29-2017 01:48 PM
[GUI Plugin] KiNotes -axel- Plugins 0 07-14-2013 07:39 PM
[GUI Plugin] Plugin Updater **Deprecated** kiwidude Plugins 159 06-19-2011 01:27 PM


All times are GMT -4. The time now is 02:33 AM.


MobileRead.com is a privately owned, operated and funded community.