01-03-2013, 10:20 PM | #1126 | ||
Plugin Developer
Posts: 6,286
Karma: 3966249
Join Date: Dec 2011
Location: Midwest USA
Device: Kindle Paperwhite(10th)
|
Quote:
Quote:
Code:
Now with cover art from [url=http://insanityrainbow.deviantart.com[/url] Code:
replace_metadata: description=>\[/url=> |
||
01-03-2013, 10:27 PM | #1127 | ||
Plugin Developer
Posts: 6,286
Karma: 3966249
Join Date: Dec 2011
Location: Midwest USA
Device: Kindle Paperwhite(10th)
|
Quote:
Quote:
|
||
Advert | |
|
01-03-2013, 11:31 PM | #1128 |
Connoisseur
Posts: 89
Karma: 706
Join Date: Nov 2012
Device: Kobo Touch
|
The main story page gives the following chapter list:
The API gives the following chapter list: {"id":39733,"title":"The Prolouge","words":717,"views":1963,"link":"http:\/\/www.fimfiction.net\/story\/13070\/1\/sweetie-belles-secret\/the-prolouge","date_modified":1331509540}, {"id":43012,"title":"Hey Sis...?","words":1161,"views":1386,"link":"http:\/\/www.fimfiction.net\/story\/13070\/2\/sweetie-belles-secret\/hey-sis","date_modified":1331509495}, {"id":46187,"title":"B-b-but...","words":1151,"views":1279,"link":"http:\/\/www.fimfiction.net\/story\/13070\/3\/sweetie-belles-secret\/bbbut","date_modified":1331509438}, {"id":110478,"title":"Author Update","words":200,"views":542,"link":"http:\/\/www.fimfiction.net\/story\/13070\/4\/sweetie-belles-secret\/author-update","date_modified":1341075370}, {"id":216887,"title":"Another Day of School","words":529,"views":246,"link":"http:\/\/www.fimfiction.net\/story\/13070\/5\/sweetie-belles-secret\/another-day-of-school","date_modified":1355586884} The fourth chapter in the API is something called "Author Update", which an old copy of the fanfic I've got kicking around from before reveals is a short note from the author talking about his update schedule. When I unescape and paste that link into my browser now, however, it takes me to the current fourth chapter, "Another Day of School." The problem is that the API's entry is telling FFDL that that chapter is called "Author Update", rather than "Another Day of School", and so it's titling the chapter that in the epub it generates. It seems that FiMFiction is basically ignoring all of the story URL beyond the chapter number, I can type whatever garbage I want in there and it still takes me to the fourth published chapter. I guess the way to fix this would be to have FFDL parse the chapter title out of the page's text rather than trusting the API. Something like that could also fix the rare accented-character-in-title-gives-null-title-in-API bug too. But that seems like a bit of a hassle and a kludge to fix something that's properly FiMFiction's responsibility to correct. My only real concern is that if one of my epubs gets an incorrect chapter title inside it there's no error message letting me know it needs to be fixed, but I guess that's a pretty minor flaw. Edit: it just occurred to me that this could potentially cause a bigger problem under the right circumstances, if there's a 50-chapter epic and the first chapter is "hidden" every chapter afterward would have its name shifted down to the next chapter. That's kinda annoying. *makes a new years' resolution to keep bugging Knighty at regular intervals* As for the IPv6 URL problem, thanks for spotting the root cause. I figured it'd be something weirdly specific to this one story, I've used FFDL on a heck of a lot of FiMFiction stories and that's the first time I ever saw that error. I actually picked that one at random to download when I grabbed a few thousand story URLs a while back to try to find cases reproducing the other bugs. Maybe I should read it anyway since I've paid so much attention to it now. Last edited by FaceDeer; 01-03-2013 at 11:44 PM. Reason: linking the API's URL for chapter 4 for convenience, and noting a worst-case scenario that just came to mind |
01-03-2013, 11:40 PM | #1129 |
Connoisseur
Posts: 89
Karma: 706
Join Date: Nov 2012
Device: Kobo Touch
|
Ah yes. I actually ditched AVG because every time I updated Calibre to a new version it would identify calibre-parallel.exe as a virus and remove it from my system, requiring me to add a new exception. Super annoying. But if AVG was going to insist "there's only room on this computer for Calibre or me!" the decision was easy enough.
|
01-04-2013, 02:50 PM | #1130 | |
Plugin Developer
Posts: 6,286
Karma: 3966249
Join Date: Dec 2011
Location: Midwest USA
Device: Kindle Paperwhite(10th)
|
Quote:
Then I made a version that does as you suggest. For fimfic (and only fimfic), it would update the chapter titles with the chapters that are downloaded. But that's still broken in the face of an update to the 50 chapter epic--because it wouldn't be downloading all those chapters again, it would use the titles from the API for old chapters. So really, the only compromise left is to scrape the chapter list. Turns out FFDL already has to scrape to get the characters, so it wasn't a huge hardship. But the fimf API does become less and less useful. Try the attached version and see how it works for you. If you have an example for the 'accented-character-in-title-gives-null-title-in-API bug', I'd be interested to hear about it, too. UPDATE Jan 04, 2012 -- Remove obsolete beta versions Last edited by JimmXinu; 01-04-2013 at 05:15 PM. Reason: Remove obsolete beta versions |
|
Advert | |
|
01-04-2013, 04:27 PM | #1131 |
Connoisseur
Posts: 89
Karma: 706
Join Date: Nov 2012
Device: Kobo Touch
|
These three are the only ones I know of with the null title problem: http://www.fimfiction.net/api/story.php?story=51241, http://www.fimfiction.net/api/story.php?story=52367 and http://www.fimfiction.net/api/story.php?story=65474. It's really rare but I kept careful track of them because if one of these gets included in a batch update the whole batch fails.
I'll try out the attached version and let you know how it works for me. I haven't updated any of my actual I'm-reading-these stories with the latest update, just test data I can throw away, so there shouldn't be any misnamed chapters hidden away in hard-to-find places. Oh, an unrelated question while on the subject of scraping pages and whatnot. FiMFiction stories sometimes have "sex" and "gore" tags on their description pages (http://www.fimfiction.net/story/74526/ and http://www.fimfiction.net/story/58005/ are examples). Does FFDL pick those up? I thought I'd hooked up all the metadata to custom columns but I've overlooked stuff like this before. Edit: Just tried it out on that test case from yesterday and it worked great, chapters are titled correctly. Thanks! I wish you'd put a "donate" link on your plugin, after all this hassle "thanks" seems like such a minor reward. Last edited by FaceDeer; 01-04-2013 at 04:36 PM. Reason: fix the URLs of those three null title stories |
01-04-2013, 05:14 PM | #1132 | |||
Plugin Developer
Posts: 6,286
Karma: 3966249
Join Date: Dec 2011
Location: Midwest USA
Device: Kindle Paperwhite(10th)
|
Quote:
I'd started writing a whole rant about how if they can't fix their API, it's just going to stay broken, but that's hardly fair. There's lots of other sites that FFDL has to scrape more than one page to collect all the necessary data. Quote:
The attached version, I believe, addresses all of the issues so far raised. Quote:
If you look on the Plugin Index and search for JimmXinu, you'll see that I do have 3 other plugins that I do have donation links for. UPDATE Jan 05 2013 -- Remove obsolete beta versions Last edited by JimmXinu; 01-05-2013 at 01:01 PM. Reason: Remove obsolete beta versions |
|||
01-04-2013, 05:47 PM | #1133 |
Junior Member
Posts: 3
Karma: 10
Join Date: Jan 2013
Device: none
|
Thank you for your help. I use Comodo and until the last two updates for Calibre and at least the last one from FFDL i hadn't any trouble. I'll find someone to help me with it. I'm on the level to download and install but not play around with settings. Cheers. :P
|
01-04-2013, 06:07 PM | #1134 |
Connoisseur
Posts: 89
Karma: 706
Join Date: Nov 2012
Device: Kobo Touch
|
I wouldn't be paying for the fanfics, though, I'd be paying for the handy convenient tool that helped me download and format them. Like how I paid money for an ebook reader that I mainly use to read fanfics. The stories themselves are still free, the ebook reader just makes it more convenient to read them.
Fortunately, it seems I've used your epubmerge plugin a couple of times before. And I recall it being very helpful, certainly worth donating to support. So there, everything's all nice and morally pure. |
01-05-2013, 12:16 AM | #1135 | |
Junior Member
Posts: 3
Karma: 10
Join Date: Jan 2013
Device: none
|
Quote:
If someone else encounter the same problem it might be possible to pinpoint the culprit. Thanks anyway. Cheers. |
|
01-05-2013, 01:01 PM | #1136 |
Plugin Developer
Posts: 6,286
Karma: 3966249
Join Date: Dec 2011
Location: Midwest USA
Device: Kindle Paperwhite(10th)
|
New Test version with:
I'd appreciate some feedback on whether this settles all the known fimfic problems and how well restrictedsection.org works. Feedback on other new sites also welcome. Thanks. UPDATE Jan 05 2013 -- Remove obsolete beta versions Last edited by JimmXinu; 01-05-2013 at 05:44 PM. Reason: Remove obsolete beta versions |
01-05-2013, 04:39 PM | #1137 |
Connoisseur
Posts: 89
Karma: 706
Join Date: Nov 2012
Device: Kobo Touch
|
I ran this version and updated my complete collection of "problem" FiMFiction stories and none of them gave errors. I spot-checked a few and the epub matches what's on the site (except for one case I found which I believe was a situation where the author had done edits to a previously-downloaded chapter - "overwrite always" fixed it). So I think this got 'em all.
|
01-05-2013, 05:41 PM | #1138 |
Plugin Developer
Posts: 6,286
Karma: 3966249
Join Date: Dec 2011
Location: Midwest USA
Device: Kindle Paperwhite(10th)
|
Version 1.7.3 - 05 Jan 2013
Last edited by JimmXinu; 01-05-2013 at 05:43 PM. |
01-08-2013, 12:35 AM | #1139 |
Zealot
Posts: 139
Karma: 21754
Join Date: Mar 2011
Device: Kindle 3, Icarus Illumina XL, Boyue T62
|
I've noticed that if server is unreachable, such as 502 error that pops up when ao3 is under maintenance, or server overloaded error on ficbook.net that shows up from time to time, when the dialog box appears saying x good and y bad updates were found, that link isn't listed.
|
01-08-2013, 12:16 PM | #1140 | |
Plugin Developer
Posts: 6,286
Karma: 3966249
Join Date: Dec 2011
Location: Midwest USA
Device: Kindle Paperwhite(10th)
|
Quote:
|
|
Thread Tools | Search this Thread |
|
Similar Threads | ||||
Thread | Thread Starter | Forum | Replies | Last Post |
[GUI Plugin] Resize Cover | kiwidude | Plugins | 95 | 03-16-2024 11:55 PM |
[GUI Plugin] Open With | kiwidude | Plugins | 402 | 03-16-2024 11:44 PM |
[GUI Plugin] Find Duplicates | kiwidude | Plugins | 1096 | 03-16-2024 11:28 PM |
[GUI Plugin] Count Pages | kiwidude | Plugins | 1744 | 03-02-2024 05:39 PM |
[GUI Plugin] Plugin Updater **Deprecated** | kiwidude | Plugins | 159 | 06-19-2011 12:27 PM |