[GUI Plugin] FanFictionDownLoader

[Firedancer885]

Originally Posted by Amalthia

Hi,

I'm still receiving the cannot update metadata on ebooks and I think part of the problem is if there is no data to fill in the custom column (like if a story has 0 reviews) I receive the error message that the metadata couldn't be updated. However, the other custom columns filled in information.

Here is the message I recieved on the latest story download.

calibre, version 1.3.0
ERROR: Error Updating Metadata: <p>An error has occurred while FFDL was updating calibre's metadata for <a href='http://archiveofourown.org/works/968232'>The Victory of Shadows</a>.</p>The ebook has been updated, but the metadata has not.

Traceback (most recent call last):
File "calibre_plugins.fanfictiondownloader_plugin.ffdl_ plugin", line 1113, in update_books_loop
File "calibre_plugins.fanfictiondownloader_plugin.ffdl_ plugin", line 1512, in update_metadata
File "site-packages\calibre\db\legacy.py", line 646, in set_custom
File "site-packages\calibre\db\cache.py", line 939, in set_field
File "site-packages\calibre\db\write.py", line 509, in set_books
File "site-packages\calibre\db\write.py", line 509, in <dictcomp>
File "site-packages\calibre\db\write.py", line 88, in adapt_number
ValueError: invalid literal for int() with base 10: ''

The latest official release of FFDL should have a fix for it.

[Sefiriot]

Originally Posted by JimmXinu

Sefiriot -

If it's anthologies you're trying to use Generate Cover for, I use this:

[defaults]
add_to_generate_cover_settings:
 ${anthology}=>true=>FFDL

Where FFDL is the name of the GC settings.

Yep, that's what I was angling for and it works; thank you very much!

ETA: Updating metadata for some books, got this message. Not sure what I did here that it's throwing these messages.

calibre, version 1.3.0
ERROR: Error Updating Metadata: <p>An error has occurred while FFDL was updating calibre's metadata for <a href='http://archiveofourown.org/works/748104'>Please Act Now To Claim Your Prize</a>.</p>The ebook has been updated, but the metadata has not.

Traceback (most recent call last):
  File "calibre_plugins.fanfictiondownloader_plugin.ffdl_plugin", line 1113, in update_books_loop
  File "calibre_plugins.fanfictiondownloader_plugin.ffdl_plugin", line 1444, in update_metadata
  File "site-packages\calibre\db\legacy.py", line 794, in func
  File "site-packages\calibre\db\cache.py", line 56, in ans
  File "site-packages\calibre\db\cache.py", line 922, in set_field
  File "site-packages\calibre\db\backend.py", line 1348, in windows_check_if_files_in_use
  File "site-packages\calibre\utils\filenames.py", line 340, in __init__
IOError: [Errno 13] File is open in another process: u'd:\\library\\antarctica_or_bust\\please act now to claim your prize (5266)\\please act now to claim your prize - antarctica_or_bust.epub'

Story Details:
comment:
     Metadata collected.
status:
     Meta
pubdate:
     2013-04-04 00:00:00+08:00
listorder:
     19
all_metadata:
     rating:
          General Audiences
     author:
          Antarctica_or_bust
     series:
          
     numWords:
          388
     site:
          archiveofourown.org
     datePublished:
          2013-04-04
     authorHTML:
          <a class='authorlink' href='http://archiveofourown.org/users/Antarctica_or_bust/pseuds/Antarctica_or_bust'>Antarctica_or_bust</a>
     storyUrl:
          http://archiveofourown.org/works/748104
     storyId:
          748104
     bookmarks:
          2
     category:
          The Hobbit (Jackson movies)
     lastupdate:
          Last Update Year/Month: 2013/04, Last Update: 2013/04/04
     formatext:
          
     title:
          Please Act Now To Claim Your Prize
     ships:
          
     seriesHTML:
          
     comments:
          1
     fandoms:
          The Hobbit (Jackson movies)
     formatname:
          
     version:
          FanFictionDownLoader v1.7.44
     collections:
          
     series02:
          
     dateUpdated:
          2013-04-04
     numChapters:
          1
     status:
          Completed
     langcode:
          
     description:
          <blockquote class="userstuff"><p>The Shire isn't the first place Thorin and Co. went looking for a burglar. The problem is, their quest sounds a lot like a Nigerian Prince con.</p><p>"You know, I am a displaced prince soon to come into my inheritance, lend me $$$/fund my quest/come with and I will repay you with 1/14th share of my mountain of gold."</p><p>Thorin is 100% legitimate but his sales pitch is so bad NOBODY believes him.</p></blockquote>
     warnings:
          No Archive Warnings Apply
     series03Url:
          
     dateCreated:
          2013-09-16 18:47:39
     output_css:
          
     series00Url:
          
     authorId:
          Antarctica_or_bust
     series01Url:
          
     authorUrl:
          http://archiveofourown.org/users/Antarctica_or_bust/pseuds/Antarctica_or_bust
     characters:
          Elrond, Thorin Oakenshield
     series02Url:
          
     genre:
          Crack, Gen, Miscommunication, Nigerian Prince Scam, POV Minor Character, Preseries
     hits:
          292
     kudos:
          38
     language:
          
     seriesUrl:
          
     series03:
          
     ao3categories:
          Gen
     series01:
          
     series00:
          
     siteabbrev:
          ao3
     freeformtags:
          Crack, Miscommunication, Nigerian Prince Scam, POV Minor Character, Preseries
     extratags:
          FanFiction
     freefromtags:
          Crack, Miscommunication, Nigerian Prince Scam, POV Minor Character, Preseries
added:
     False
end:
     None
author:
     [u'Antarctica_or_bust']
comments:
     <blockquote><p>The Shire isn't the first place Thorin and Co. went looking for a burglar. The problem is, their quest sounds a lot like a Nigerian Prince con.</p><p>"You know, I am a displaced prince soon to come into my inheritance, lend me $$$/fund my quest/come with and I will repay you with 1/14th share of my mountain of gold."</p><p>Thorin is 100% legitimate but his sales pitch is so bad NOBODY believes him.</p></blockquote>
calibre_id:
     5266
series:
     
username:
     <was set, removed for security>
begin:
     None
good:
     True
tags:
     ['', u'Thorin Oakenshield', u'POV Minor Character', u'Preseries', u'Miscommunication', u'The Hobbit (Jackson movies)', u'Completed', u'Crack', u'Nigerian Prince Scam', u'FanFiction', u'Gen', u'Elrond']
timestamp:
     2013-09-16 18:47:39.324000+08:00
is_adult:
     False
updatedate:
     2013-04-04 00:00:00+08:00
password:
     
icon:
     metadata.png
publisher:
     archiveofourown.org
url:
     http://archiveofourown.org/works/748104
author_sort:
     [u'Antarctica_or_bust']
title:
     Please Act Now To Claim Your Prize

[JimmXinu]

Originally Posted by Sefiriot

ETA: Updating metadata for some books, got this message. Not sure what I did here that it's throwing these messages.

...
  File "site-packages\calibre\db\backend.py", line 1348, in windows_check_if_files_in_use
  File "site-packages\calibre\utils\filenames.py", line 340, in __init__
IOError: [Errno 13] File is open in another process: u'd:\\library\\antarctica_or_bust\\please act now to claim your prize (5266)\\please act now to claim your prize - antarctica_or_bust.epub'

Pretty much exactly what it says. calibre 1.X checks for other processes with open handles on its files now. This isn't an FFDL problem. Are you running DropBox or something on your library?

[Sefiriot]

The library isn't in Dropbox or anything that cloud syncs, and the books weren't open either, that's why I was surprised. Whole thing was strange, since later when I updated the books one by one the problem disappeared.

[Amalthia]

Originally Posted by Firedancer885

The latest official release of FFDL should have a fix for it.

I'm using the latest release.

[JimmXinu]

Originally Posted by Amalthia

I'm using the latest release.

(post describing the problem)

My bad, I only fixed this for the GUI defined custom columns, not ini defined mappings.

(In my defense, by the time I went to test it using that story, it didn't have 0 reviews anymore. It took 10x longer to setup a test case than it did to fix the problem.)

Anyway, attached is a version that fixes ini mapped custom float/int columns when there's no value to set.

UPDATE Sep 25, 2013 - Remove obsolete beta versions

[Amalthia]

Originally Posted by JimmXinu

(post describing the problem)

My bad, I only fixed this for the GUI defined custom columns, not ini defined mappings.

(In my defense, by the time I went to test it using that story, it didn't have 0 reviews anymore. It took 10x longer to setup a test case than it did to fix the problem.)

Anyway, attached is a version that fixes ini mapped custom float/int columns when there's no value to set.

Thanks!

[KayLee]

would like to see some support for this site
http://www.gayauthors.org/index.html

[JimmXinu]

Originally Posted by KayLee

would like to see some support for this site
http://www.gayauthors.org/index.html

We did for a while. They asked us to stop.

[BaconButty]

I seem to be having a problem with fanfiction.net.

I get an error message when I try to update several fics:

http://www.fanfiction.net/s/9533161/1/
http://www.fanfiction.net/s/8620508/1/
and a couple of others...

Title: Unknown
Author: Unknown
Comment: 'NoneType' object has no attribute '__getitem__'


Edit: please ignore this post. FFN is offline.

"09/20/2013: Repairs in progress….
We are working to bring service back online after some hardware failures have resulted in downtime. We apologize for the disruption. Please stay tuned for further updates."

[Firedancer885]

Jimm, please check this:

Status
Title
Author
Comment
URL
Error
ST13 : BUD/s Forever
Tenhawk
IncompleteRead(6135 bytes read)
http://fanfiction.tenhawkpresents.com/viewstory.php?sid=13

http://fanfiction.tenhawkpresents.co...ory.php?sid=13

I have a suspicion, it's because the slash in the title, but I'm not sure.

[seabream]

Hi JimmXinu,

Minor strange issue with the formatting of a story on AO3:
http://archiveofourown.org/works/924346/

In the version of Chapter 2 downloaded with FFDL, after the second instance of "John Keats's Attitude to Lawyers", everything is italicised until the end of the chapter, which it isn't if viewed on the webpage. And it's not just a single absent trailing </i> tag. I opened it with Tweak Book, and every paragraph after "Lawyers" has <i class="calibre8"></i> around it.

It happens again at Chapter 11, after "You said the entire".

I can manually correct it, but it does seem to be a bug that might be generalizable.

I'm using the version of FFDL posted in #1896. Still with Calibre 0.9.40.

Sorry I haven't gotten back to you yet about the AO3 series testing. While it mostly works, there are some things about the implementation that I'm trying to work out in terms of use cases that I haven't been able to get to work, but I may yet. Also I've been a bit occupied with other things.

[JimmXinu]

Originally Posted by seabream

Minor strange issue with the formatting of a story on AO3:
http://archiveofourown.org/works/924346/

In the version of Chapter 2 downloaded with FFDL, after the second instance of "John Keats's Attitude to Lawyers", everything is italicised until the end of the chapter, which it isn't if viewed on the webpage. And it's not just a single absent trailing </i> tag. I opened it with Tweak Book, and every paragraph after "Lawyers" has <i class="calibre8"></i> around it. ...

GIGO. There are italic tags are in the source HTML. Multiple times, in fact. AO3 has CSS that sets "font-style: normal;" on an <i> tag inside an <i> tag.

Looking at it, I presume the author made a mistake with "<i>text<i>" instead of "<i>text</i>" and some helpful program tried to fix it.

Spoiler:

<p>"I overheard her and another girl," John tells her.  "In 
the library basement, the other day.  I was looking for 
<i>John Keats's Attitude to Lawyers<i> and I just sort 
of stumbled onto them."  </i></i></p>
<p>
  <i>
    <i>Beca doesn't say anything.  </i>
  </i>
</p>
...
<p>
  <i>
    <i>
      <i>
        <i>John almost hopes that Beca brings her girlfriends 
              to their next lunch; it will make for a very interesting 
              meal.</i>
      </i>
    </i>
  </i>
</p>

After it gets into FFDL, all the nested <i> tags get folded down into one.

The class="calibre8" on the <i> tags isn't from FFDL. That leads me to suspect you did a epub->epub conversion on it, too.

[seabream]

Originally Posted by JimmXinu

GIGO. There are italic tags are in the source HTML. Multiple times, in fact.

That's embarrassing. I should have thought to check the source and not just the display. Something to remember for next time. Sorry for the bother.

Originally Posted by JimmXinu

After it gets into FFDL, all the nested <i> tags get folded down into one.

The class="calibre8" on the <i> tags isn't from FFDL. That leads me to suspect you did a epub->epub conversion on it, too.

I did. I prefer book formatting with paragraph breaks as indents rather than vertical gaps for the space efficiency.

Thanks for looking into it, though you shouldn't have had to.

[JimmXinu]

Originally Posted by seabream

That's embarrassing. I should have thought to check the source and not just the display. Something to remember for next time. Sorry for the bother.

No problem. I've come to expect weird cruft in story HTML. It's amazing what you'll find sometimes.