Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre > Plugins

Notices

Closed Thread
 
Thread Tools Search this Thread
Old 10-09-2012, 02:27 PM   #916
evamaria
Junior Member
evamaria began at the beginning.
 
Posts: 8
Karma: 10
Join Date: Dec 2009
Device: Sony PRS-505
This seems to fix it - thanks so much for your patience and help!

Quote:
Originally Posted by JimmXinu View Post
If you haven't already, go create a file named /Users/evamaria/Library/Preferences/calibre/plugins/fanfictiondownloader_macmenuhack.txt

See http://code.google.com/p/fanficdownl...rCalibrePlugin regarding "Mac User Crashes"
evamaria is offline  
Old 10-10-2012, 11:30 AM   #917
Firedancer885
Occassional Beta Tester
Firedancer885 can teach chickens to fly.Firedancer885 can teach chickens to fly.Firedancer885 can teach chickens to fly.Firedancer885 can teach chickens to fly.Firedancer885 can teach chickens to fly.Firedancer885 can teach chickens to fly.Firedancer885 can teach chickens to fly.Firedancer885 can teach chickens to fly.Firedancer885 can teach chickens to fly.Firedancer885 can teach chickens to fly.Firedancer885 can teach chickens to fly.
 
Posts: 283
Karma: 3516
Join Date: Nov 2010
Location: Hungary
Device: Samsung Galaxy Tab 4 (wifi only)
Hi Jim,

is there a way to make FFDL strip the extra indentation of the comments for the AO3 stories without also stripping all the other html?
Firedancer885 is offline  
Old 10-10-2012, 11:36 AM   #918
JimmXinu
Plugin Developer
JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.
 
JimmXinu's Avatar
 
Posts: 6,262
Karma: 3675895
Join Date: Dec 2011
Location: Midwest USA
Device: Kindle Paperwhite(10th)
Quote:
Originally Posted by Firedancer885 View Post
is there a way to make FFDL strip the extra indentation of the comments for the AO3 stories without also stripping all the other html?
Can you give an example?

I know that Ida wrote the AO3 adapter and went to some effort to include the leading and trailing author notes and chapter summaries. I'd probably have just left them off.
JimmXinu is offline  
Old 10-10-2012, 11:42 AM   #919
iatheia
Zealot
iatheia has become one with the cosmosiatheia has become one with the cosmosiatheia has become one with the cosmosiatheia has become one with the cosmosiatheia has become one with the cosmosiatheia has become one with the cosmosiatheia has become one with the cosmosiatheia has become one with the cosmosiatheia has become one with the cosmosiatheia has become one with the cosmosiatheia has become one with the cosmos
 
Posts: 139
Karma: 21754
Join Date: Mar 2011
Device: Kindle 3, Icarus Illumina XL, Boyue T62
Ah, those things. AO3 formats all the notes and summaries as blockquote. When including them I haven't changed that tag to anything else. Shouldn't be terribly hard to change it to p or div instead, but personally I think it is more charming this way.
iatheia is offline  
Old 10-10-2012, 12:58 PM   #920
Firedancer885
Occassional Beta Tester
Firedancer885 can teach chickens to fly.Firedancer885 can teach chickens to fly.Firedancer885 can teach chickens to fly.Firedancer885 can teach chickens to fly.Firedancer885 can teach chickens to fly.Firedancer885 can teach chickens to fly.Firedancer885 can teach chickens to fly.Firedancer885 can teach chickens to fly.Firedancer885 can teach chickens to fly.Firedancer885 can teach chickens to fly.Firedancer885 can teach chickens to fly.
 
Posts: 283
Karma: 3516
Join Date: Nov 2010
Location: Hungary
Device: Samsung Galaxy Tab 4 (wifi only)
Quote:
Originally Posted by iatheia View Post
Ah, those things. AO3 formats all the notes and summaries as blockquote. When including them I haven't changed that tag to anything else. Shouldn't be terribly hard to change it to p or div instead, but personally I think it is more charming this way.
Could you or Jimm add something to either FFDL or the specific adapter to let people choose? I don't like the blockquotes. I don't mind them in the chapter summaries, only in the comments.
Firedancer885 is offline  
Old 10-10-2012, 06:08 PM   #921
JimmXinu
Plugin Developer
JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.
 
JimmXinu's Avatar
 
Posts: 6,262
Karma: 3675895
Join Date: Dec 2011
Location: Midwest USA
Device: Kindle Paperwhite(10th)
Quote:
Originally Posted by Firedancer885 View Post
Could you or Jimm add something to either FFDL or the specific adapter to let people choose? I don't like the blockquotes. I don't mind them in the chapter summaries, only in the comments.
How about giving yourself a personalized output_css and adding:

Code:
blockquote {margin-left: 0px; margin-right: 0px;}
JimmXinu is offline  
Old 10-10-2012, 07:01 PM   #922
AmberDragon
Member
AmberDragon began at the beginning.
 
Posts: 18
Karma: 10
Join Date: May 2012
Device: Nook Color
I think I just ran into either a bug or an unforseen case on AO3. Apparently, on AO3, a story can be tagged with multiple series tags, but FFDL only picks up the first series listed when you try to grab the story. You can find a series with several stories that have secondary series tags at http://archiveofourown.org/series/25185 for testing.
AmberDragon is offline  
Old 10-10-2012, 07:37 PM   #923
JimmXinu
Plugin Developer
JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.
 
JimmXinu's Avatar
 
Posts: 6,262
Karma: 3675895
Join Date: Dec 2011
Location: Midwest USA
Device: Kindle Paperwhite(10th)
Quote:
Originally Posted by AmberDragon View Post
I think I just ran into either a bug or an unforseen case on AO3. Apparently, on AO3, a story can be tagged with multiple series tags, but FFDL only picks up the first series listed when you try to grab the story. You can find a series with several stories that have secondary series tags at http://archiveofourown.org/series/25185 for testing.
As I recall, there are a few sites that allow a story to be in multiple series. But what would FFDL do with them?

Calibre defines one standard series column. You can add custom series columns, the issue is populating them. Even if FFDL had a feature for an extra series, can one story have three series? Four?

And even if the 'second' series for a story is put in a custom series column, if the next story in the same two series list them in the opposite order, it's still not useful. You could end up with:
Code:
Series     Custom Series
A[1]        A[2]
B[2]        B[1]
You couldn't sort correctly on either column.

That said, I am willing to add some AO3 specific extra_valid_entries for additional series. But you'd have to use custom_columns_settings to get them into your secondary custom series column(s). (Another alternative would be to include them in tags or tags-like custom column.)

Does that sound useful to anyone?
JimmXinu is offline  
Old 10-10-2012, 08:28 PM   #924
AmberDragon
Member
AmberDragon began at the beginning.
 
Posts: 18
Karma: 10
Join Date: May 2012
Device: Nook Color
I didn't even think to check whether Calibre supported multiple series or not, I just assumed it did. Sorry. If it's a Calibre thing, I'll just keep on using the Manage Series plugin to keep everything neat and tidy.
AmberDragon is offline  
Old 10-14-2012, 03:41 PM   #925
JimmXinu
Plugin Developer
JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.
 
JimmXinu's Avatar
 
Posts: 6,262
Karma: 3675895
Join Date: Dec 2011
Location: Midwest USA
Device: Kindle Paperwhite(10th)
I could use some help testing a new version again.

This one is mostly internal changes and minor fixes rather than new features for plugin users.
  • Changes to logging to make more logging visible in PI and the same across web/CLI/PI.
  • Don't number chapter when only one chapter even if add_chapter_numbers is on.
  • Fix a bug where cover could still be partially set to an image in the cover_exclusion_regexp list.
  • Add include_images support for HTML format. (I can't imagine that anyone uses HTML output with calibre--this is for CLI users.)
  • Preserve id and class attributes on the HTML tags. (This is for users who want more control from output_css. I don't *think* it will hurt anything for other users...)

If I could get a few users to use this version for a few days, I'd appreciate it, thanks.

Update Oct 15, 2012: There's a newer test version with one change a few posts down.

Last edited by JimmXinu; 10-15-2012 at 10:43 AM. Reason: Remove obsolete beta versions
JimmXinu is offline  
Old 10-14-2012, 05:49 PM   #926
iatheia
Zealot
iatheia has become one with the cosmosiatheia has become one with the cosmosiatheia has become one with the cosmosiatheia has become one with the cosmosiatheia has become one with the cosmosiatheia has become one with the cosmosiatheia has become one with the cosmosiatheia has become one with the cosmosiatheia has become one with the cosmosiatheia has become one with the cosmosiatheia has become one with the cosmos
 
Posts: 139
Karma: 21754
Join Date: Mar 2011
Device: Kindle 3, Icarus Illumina XL, Boyue T62
Quote:
Originally Posted by JimmXinu View Post
I could use some help testing a new version again.[*]Don't number chapter when only one chapter even if add_chapter_numbers is on.
What about adding new chapters to a recently published one-chaptered ongoing story? Would that add the number to the first chapter?
iatheia is offline  
Old 10-14-2012, 06:54 PM   #927
ilovejedd
hopeless n00b
ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.
 
ilovejedd's Avatar
 
Posts: 5,111
Karma: 19597086
Join Date: Jan 2009
Location: in the middle of nowhere
Device: PW4, PW3, Libra H2O, iPad 10.5, iPad 11, iPad 12.9
Found an issue with grangerenchanted.com. For some reason, it's not getting the chapter text for chapters that have an image at the beginning of the chapter.

Here's one of the affected fics:
http://grangerenchanted.com/enchant/...y.php?sid=2956

the following chapters to be specific:
http://grangerenchanted.com/enchant/...956&chapter=42
http://grangerenchanted.com/enchant/...956&chapter=44
http://grangerenchanted.com/enchant/...956&chapter=51
http://grangerenchanted.com/enchant/...956&chapter=54

This is using the latest version of FFDL (1.6.14). Issue also exists on older versions I've tried (1.6.08, 1.6.09, 1.6.12 & 1.6.13).
ilovejedd is offline  
Old 10-15-2012, 09:27 AM   #928
JimmXinu
Plugin Developer
JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.
 
JimmXinu's Avatar
 
Posts: 6,262
Karma: 3675895
Join Date: Dec 2011
Location: Midwest USA
Device: Kindle Paperwhite(10th)
Quote:
Originally Posted by iatheia View Post
What about adding new chapters to a recently published one-chaptered ongoing story? Would that add the number to the first chapter?
Yes. Update actually reads the chapter text from each existing chapter and uses it as if it had just been downloaded. So the chapter titles are rewritten, adding numbers when there's more than one.
JimmXinu is offline  
Old 10-15-2012, 10:25 AM   #929
JimmXinu
Plugin Developer
JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.
 
JimmXinu's Avatar
 
Posts: 6,262
Karma: 3675895
Join Date: Dec 2011
Location: Midwest USA
Device: Kindle Paperwhite(10th)
Quote:
Originally Posted by ilovejedd View Post
Found an issue with grangerenchanted.com. For some reason, it's not getting the chapter text for chapters that have an image at the beginning of the chapter.
Try this version. It uses a different, more forgiving version of the HTML parser.

Update: There's a newer test version later on incorporating this change.

Last edited by JimmXinu; 10-16-2012 at 05:07 PM. Reason: Remove obsolete beta versions
JimmXinu is offline  
Old 10-16-2012, 06:57 AM   #930
Dylan Tomorrow
Connoisseur
Dylan Tomorrow ought to be getting tired of karma fortunes by now.Dylan Tomorrow ought to be getting tired of karma fortunes by now.Dylan Tomorrow ought to be getting tired of karma fortunes by now.Dylan Tomorrow ought to be getting tired of karma fortunes by now.Dylan Tomorrow ought to be getting tired of karma fortunes by now.Dylan Tomorrow ought to be getting tired of karma fortunes by now.Dylan Tomorrow ought to be getting tired of karma fortunes by now.Dylan Tomorrow ought to be getting tired of karma fortunes by now.Dylan Tomorrow ought to be getting tired of karma fortunes by now.Dylan Tomorrow ought to be getting tired of karma fortunes by now.Dylan Tomorrow ought to be getting tired of karma fortunes by now.
 
Posts: 62
Karma: 640495
Join Date: Jul 2012
Location: Germany
Device: Kindle Touch, Android smartphone w/ FBReader
Lightbulb Broken indexing, missing TOC on my Kindle and my conversion workaround

Hi everyone,

as before, I've been reading loads of fanfics and mostly been very happy with this plugin and the fabulous job it is doing at downloading them, but two issues remained when reading the MOBI files on my Kindle Touch, for which I can at least share how I fixed them using two other tools (Howto at the end):

1. Broken indexing: This only happened to some of the MOBIs created by FFDL—most of them do index without hassle—but it was very annoying as the Kindle tries to index those files into eternity, draining the battery while doing so. This one I got to the bottom of—for 3 files—in trying to solve it. I figured it must be some non.validating HTML tag soup responsible for it and turned out, that was the case for those three. I let FFDL download the EPUB and then converted that to MOBI with the official KindleGen command line tool.

It was the three fictionalley stories I mentioned before, all of them had not properly closed <p> tags which KindleGen corrected (well, for two of them. One of them it just gave me an "Enhanced Mobi building failure" ). None of the other files gave me errors, but most of the MOBIs KindleGen put out indexed where they did not before, no matter what I tried (adding only one unindexed file at a time, restarting the kindle, etc.). So I figure they must have had some invalid syntax somewhere.

2. Missing Table of Contents: This is an issue with all of the MOBIs created by FFDL. This one is also peculiar. The MOBIs FFDL creates evidently have TOCs, as Calibre can navigate through them in the sidebar of its EbookViewer. But they never showed up in the Go to menu. You could only jump to the TOC that was part of the ebook itself via Go to, which made it more cumbersome.

But the TOC does show up in the Go to menu for ebooks converted with KindleGen. To be more exact, it does in the KF8 part. You see, MobiGen creates a hybrid MOBI file that contains both the old Mobipocket/mobi7 format and the new KF8/AZW3 format. Because there were still some files refusing to index, I used MobiUnpack to split the MOBIs into .mobi (mobi7) and .azw3 (KF8) files. The .azw3 files are the ones where the TOC works as expected, and all of them index without a problem and also superfast.

Summary/Conversion Workaround Howto:
So to anyone else who has problems with indexing/missing TOCs on Kindle eReaders (Warning: This will probably break your previous notes and marks from the old MOBIs, so I'd advise to only do it for the non-indexing files and files without notes/marks):

Preparations (only necessary before first conversion):
  • configure duplicates automerge: Go to Preferences -> Import/Export -> Adding Books: Activate checkbox Automerge added books if they already exist in the calibre library, choose Create new record for each duplicate format in the dropdown menu, click Apply.
  • Download & Install KindleGen [mr wiki page] (install under linux: execute sudo cp kindlegen/ /opt/kindlegen/ ; sudo ln -s /opt/kindlegen/kindlegen /usr/local/bin/kindlegen), download & unpack MobiUnpack [mr wiki page] (use the python script in Mobi_Unpack_v054.zip, as the calibre plugin can unfortunately not work on more than one file at a time), copy contents of /lib/ subfolder into your conversion folder
The conversion workaround itself:
  1. Download Fanfics as EPUB with FFDL
  2. Export EPUBs (and MOBIs, if any) to conversion directory with calibre's save to disk in a single directory action (delete OPF files (you can keep the old MOBIs, if any, as backup, but move them to another directory so that KindleGen won't overwrite them))
  3. batch convert EPUBs with kindlegen; e.g. in the Linux bash console:
    Code:
    for b in *.epub ; do kindlegen "$b" -c2 -verbose && echo "$b converted successfully." ; done
  4. batch split .azw3s from .mobis, collect .azw3s from MobiUnpack-created subfolders in conversion folder and delete leftover MU subfolders; e.g. in the Linux bash console:
    Code:
    for b in *.mobi ; do python mobi_unpack.py -s -d "$b" && echo "$b converted successfully." ; done
    mv */*.azw3 .
    rm -rf ./*/
  5. delete any left-over old MOBIs from the fanfics with the remove files of a specific format from selected books action
  6. import .azw3 files from your conversion directory into calibre via drag&drop or the Add Books menu entry, they are automagically added to the right entries
  7. Put files onto Kindle, experience working TOC and indexing
  8. PROFIT!

EDIT:
I did not save the KindleGen log for the first conversion batch I did, so I just ran the corresponding FFDL created EPUBs through Epubcheck 3. Almost all of them had similar HTML syntax errors (mostly unclosed <p> tags, also faulty element nesting, missing attributes, inline elements in the <body> outside of an enclosing block element). ALL of the fictionalley fanfics produced errors, also some from AO3 and fanfiction.net.

So I assume those kinds of syntax errors explain why the files did not index. If only all fanfic sites enforced proper XHTML/HTML5!

Last edited by Dylan Tomorrow; 10-16-2012 at 03:10 PM. Reason: +Epubcheck results
Dylan Tomorrow is offline  
Closed Thread

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
[GUI Plugin] Resize Cover kiwidude Plugins 95 03-16-2024 11:55 PM
[GUI Plugin] Open With kiwidude Plugins 402 03-16-2024 11:44 PM
[GUI Plugin] Find Duplicates kiwidude Plugins 1096 03-16-2024 11:28 PM
[GUI Plugin] Count Pages kiwidude Plugins 1744 03-02-2024 05:39 PM
[GUI Plugin] Plugin Updater **Deprecated** kiwidude Plugins 159 06-19-2011 12:27 PM


All times are GMT -4. The time now is 05:21 AM.


MobileRead.com is a privately owned, operated and funded community.