[GUI Plugin] FanFictionDownLoader

[Tanjamuse]

Thanks Mini.

@ JimmXinu.

I have troubles with the stories from AO3.

I got the following error:

calibre, version 2.0.0
ERROR: Unhandled exception: <b>HTTPError</b>:HTTP Error 404: Not Found

calibre 2.0 isfrozen: True is64bit: False
Windows-7-6.1.7601-SP1 Windows ('32bit', 'WindowsPE')
32bit process running on 64bit windows
('Windows', '7', '6.1.7601')
Python 2.7.8
Windows: ('7', '6.1.7601', 'SP1', 'Multiprocessor Free')
Successfully initialized third party plugins: FanFictionDownLoader && Generate Cover && User Category && Find Duplicates && Reading List && Wiki Reader
Traceback (most recent call last):
File "calibre_plugins.fanfictiondownloader_plugin.ffdl_ plugin", line 403, in get_urls_from_page_menu
File "calibre_plugins.fanfictiondownloader_plugin.ffdl_ plugin", line 419, in get_urls_from_page
File "calibre_plugins.fanfictiondownloader_plugin.fanfi cdownloader.geturls", line 50, in get_urls_from_page
File "calibre_plugins.fanfictiondownloader_plugin.fanfi cdownloader.adapters.base_adapter", line 336, in _fetchUrl
HTTPError: HTTP Error 404: Not Found

[JimmXinu]

Originally Posted by Tanjamuse

Thanks Mini.

@ JimmXinu.

I have troubles with the stories from AO3.

I got the following error:
...
HTTPError: HTTP Error 404: Not Found
...

Please post the URL and explain what you were trying to do at the time. With the information given, all I can do is say your URL wasn't found.

[Tanjamuse]

http://archiveofourown.org/works?utf8=%E2%9C%93&work_search[sort_column]=revised_at&work_search[other_tag_names]=&work_search[query]=&work_search[language_id]=1&work_search[complete]=0&commit=Sort+and+Filter&tag_id=Sabrina+the+Teena ge+Witch+%28TV%29 - This is the link. I was just trying to download the stories from "Get story URLs from Web Page"

[JimmXinu]

Originally Posted by Tanjamuse

http://archiveofourown.org/works?utf8=%E2%9C%93&work_search[sort_column]=revised_at&work_search[other_tag_names]=&work_search[query]=&work_search[language_id]=1&work_search[complete]=0&commit=Sort+and+Filter&tag_id=Sabrina+the+Teena ge+Witch+%28TV%29 - This is the link. I was just trying to download the stories from "Get story URLs from Web Page"

Good heavens.

Well, the problem has to with AO3 links and automatically adding view_adult=true to the URL.

The attached version should work better.

[m_g]

Originally Posted by JimmXinu

Go to the Introduction to plugins thread and search down to How do I install a plugin manually?.

Thanks. That's what I needed.

[iakovl]

small help, might be an issue with FFDL
https://www.mobileread.com/forums/sho...d.php?t=246214

it will sound stupid but i still don't know how to add a simple column like "added"

i have normal 2 : "modified" and "published" both help me
published = original publish date from the epub
modified = as i read fanfiction i can see when i updated something

but i lack simple "added" that will show me when i added the story to calibre,
"date" has the same date as "modified"for me
should be:
"modified">"date" for updated story
"modified"="date" for new story

[JimmXinu]

Originally Posted by iakovl

...
but i lack simple "added" that will show me when i added the story to calibre,
"date" has the same date as "modified"for me
should be:
"modified">"date" for updated story
"modified"="date" for new story

By default, FFDL updates calibre's 'date' column (internally 'timestamp') when ever the book is updated.

However, it's very easy for you to configure FFDL to not update the Date field. Go into FFDL config, on the Standard Columns tab and check 'New Only' for Date. See attached image.

I believe that will make it work the way you want.

[Firedancer885]

.....

[iakovl]

Originally Posted by JimmXinu

By default, FFDL updates calibre's 'date' column (internally 'timestamp') when ever the book is updated.

However, it's very easy for you to configure FFDL to not update the Date field. Go into FFDL config, on the Standard Columns tab and check 'New Only' for Date. See attached image.

I believe that will make it work the way you want.

thanks, i will see if it will solve the issue
but should it be the default option to avoid the problem in the first place
there shouldn't be a reason for DATE to be updated on updating a file that is in calibre

[JimmXinu]

Originally Posted by iakovl

thanks, i will see if it will solve the issue
but should it be the default option to avoid the problem in the first place
there shouldn't be a reason for DATE to be updated on updating a file that is in calibre

It is arguable that FFDL does it wrong. But I, for one, expect it to work the way that it does. FFDL is a rare case in calibre--books that are expected to be updated and changed. And IMO, that should bump the Date.

Since it's as easy as clicking a checkbox for the users who disagree--and a mob hasn't formed with torches and pitchforks clamoring for it--I'm not very inclined to change it.

[iakovl]

a mob starts with one man and an idea :P
the whole point is having:
"modified" = date of update
"date" = date of adding to calibre
"published" = date of publishing

[chrnno]

To put my two cents in I don't even use the default column called 'date' since it seemed so out of place. Might be because English is not my native language and thus I use calibre in another but considering we have a published and modified dates I don't see the logic in have a date only field. I mean date of what?

I can see the use of an updated field which could filled by when the text itself is altered* which is rather rare but I have done it a few times** instead of including metadata like modified and an added field for when it was included in the library but having a 'date' field when you have both 'modified (date)' and 'published (date)'...?

*It would then be used to drop the update from fanfics sites for FFDL specifically

**A sad, sad day when the free translations over the web are better than the one you paid for, by this point I either buy e-books in English or dead tree format if in Portuguese. Though I acknowledge it is less that it really makes a difference and more that I got so annoyed that one time that it wrongly prejudiced me against the whole.

[iakovl]

Originally Posted by chrnno

I can see the use of an updated field which could filled by when the text itself is altered* which is rather rare but I have done it a few times** instead of including metadata like modified and an added field for when it was included in the library but having a 'date' field when you have both 'modified (date)' and 'published (date)'...?

the date is "added date" the day you added the fic to calibre
for me it's an easy way to see what new storys i have not only updated ones

for all the dates
published < date (added) < modified

[turtle7761]

I'm trying to turn this series into an anthology https://www.squidge.org/peja/cgi-bin...p?seriesid=572 but when I try I get a story called Site Map by peja & Amejisuto. The first story consists of all the stuff from the site like fandoms and various assorted things. The actual first story shows up as being the second in the series. Any ideas?

[JimmXinu]

Originally Posted by turtle7761

I'm trying to turn this series into an anthology https://www.squidge.org/peja/cgi-bin...p?seriesid=572 but when I try I get a story called Site Map by peja & Amejisuto. The first story consists of all the stuff from the site like fandoms and various assorted things. The actual first story shows up as being the second in the series. Any ideas?

I'm going to point my finger at squidge.org/peja and say it's their fault.

The 'Story URLs from Web Page' feature looks at all the URLs on a page and grabs the ones that match known story URL patterns.

The problem is that squidge.org/peja uses a story for their 'Site Map' link. Literally.

If you look at the list of collected story URLs, and go to the first one, it's the Site Map and the second is the first story of the series:

https://www.squidge.org/peja/cgi-bin....php?sid=47746
https://www.squidge.org/peja/cgi-bin....php?sid=42785
...

Ideally, we'd get squidge.org/peja to stop doing that. But that's outside our control. So instead I've changed the squidge.org/peja adapter in FFDL to reject that one story ID (47746). I think. It seems to work anyway.

Try the attached test version and let me know.