Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Formats > Workshop

Notices

Reply
 
Thread Tools Search this Thread
Old 12-23-2008, 08:01 PM   #46
erayd
Zealot
erayd doesn't littererayd doesn't litter
 
Posts: 128
Karma: 146
Join Date: Apr 2008
Device: Sony PRS-650
Quote:
Originally Posted by Amalthia View Post
Thanks for getting back to me so quickly.
No problem :-). Of course, those instructions are for integrating that recipe into fflag, which I'm not sure is exactly what you wanted...
erayd is offline   Reply With Quote
Old 01-06-2009, 12:24 AM   #47
Iteria
Member
Iteria ought to be getting tired of karma fortunes by now.Iteria ought to be getting tired of karma fortunes by now.Iteria ought to be getting tired of karma fortunes by now.Iteria ought to be getting tired of karma fortunes by now.Iteria ought to be getting tired of karma fortunes by now.Iteria ought to be getting tired of karma fortunes by now.Iteria ought to be getting tired of karma fortunes by now.Iteria ought to be getting tired of karma fortunes by now.Iteria ought to be getting tired of karma fortunes by now.Iteria ought to be getting tired of karma fortunes by now.Iteria ought to be getting tired of karma fortunes by now.
 
Posts: 14
Karma: 492360
Join Date: Jan 2009
Device: Sony PRS-T1, Nook Simple Touch, Nook HD+
Is there anyway that you could adapt this for Fictionpress.com? It's the sister-site to Fanfiction.net, so I don't think it would be too much trouble. I like to read there too and it would be cool to carry those stories with me also.
Iteria is offline   Reply With Quote
Old 01-12-2009, 07:31 PM   #48
ilovejedd
hopeless n00b
ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.
 
ilovejedd's Avatar
 
Posts: 2,312
Karma: 5761596
Join Date: Jan 2009
Location: in the middle of nowhere
Device: PRS-350, Nexus S, Galaxy S, Nook Color, iPhone4, iPT4, iPad 2012
Just discovered FLAG today and it's been pretty awesome so far (at least the online version is). Thank you very much for sharing this with us.

I used to do something similar semi-manually back in 2001~02 (I've got a working mirror of my favorite authors and fanfics on my hard drive) but the new system made it hard for me to use my semi-automated workflow. I was looking for an easier way to archive fanfics to my PC and read them on my iPhone (hooray for Stanza; Safari crashes way too often) and I'm happy to have found this.

Now, I'm going to have to find some way to install Apache/PHP on my PC as having command-line access to this program for batch processing will be absolutely invaluable.

Thanks again for this wonderful program!
ilovejedd is offline   Reply With Quote
Old 01-13-2009, 01:53 AM   #49
ilovejedd
hopeless n00b
ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.
 
ilovejedd's Avatar
 
Posts: 2,312
Karma: 5761596
Join Date: Jan 2009
Location: in the middle of nowhere
Device: PRS-350, Nexus S, Galaxy S, Nook Color, iPhone4, iPT4, iPad 2012
Quote:
Originally Posted by rgibbo View Post
Here's my version of flag for windows, it is now similar to the original so it *might* run on linux (i dont have linux so i don't know if it works on there or not)
Thanks for this. I have it installed on a virtual machine (VPC 2007: XP Pro SP2, Apache 2.2.11 and PHP 5.2.8) and it seems to be working quite well. I just copied the code from lrf.codec.php to get epub.codec.php working. Lol, I was too impatient to wait for an "official" fix. I had to copy all the contents of the calibre (v0.4.128) install directory to lib\calibre because of all the python dependencies of html2epub, though.

I appreciate how modular the code for this program is. It makes it very easy to add new sources and output file types. I'm currently attempting to make source files for other sites. I've also customized ffnet.source.php, epub.codec.php and lrf.codec.php to include the fanfic summary/description.

I'm attaching a source file for Fictionpress.com as well as codec and Fanfiction.net source files modified from rgibbo's Windows version. I reckon the source files should work for both Linux and Windows. I checked ffnet.source.php from the tarball and rgibbo's zip and there didn't seem to be any difference between the two.
Attached Files
File Type: zip fpress.zip (1.3 KB, 238 views)
File Type: zip ffnet.zip (1.3 KB, 242 views)
File Type: zip codec.zip (3.2 KB, 218 views)

Last edited by ilovejedd; 01-19-2009 at 05:04 PM.
ilovejedd is offline   Reply With Quote
Old 02-10-2009, 04:09 PM   #50
malkie13
Member
malkie13 began at the beginning.
 
Posts: 14
Karma: 10
Join Date: Feb 2009
Device: PRS-505
Love this app, though thanks to it, I'll be lucky to see my PRS505 anytime soon (the missus reads heaps off of fanfiction.net). That said, is there anyway to incorporate this functionality into Calibre directly (I'm runnin OSX, Calibre is how I dump all my books onto the reader)?

Edit: I'm using the web version to do the converting, not the actual app on a machine.
malkie13 is offline   Reply With Quote
Old 02-10-2009, 04:53 PM   #51
erayd
Zealot
erayd doesn't littererayd doesn't litter
 
Posts: 128
Karma: 146
Join Date: Apr 2008
Device: Sony PRS-650
Yes, but not by me - you'd need to ask kovidgoyal for this functionality, as it would need to be rewritten as a calibre grabber.
erayd is offline   Reply With Quote
Old 02-10-2009, 04:57 PM   #52
malkie13
Member
malkie13 began at the beginning.
 
Posts: 14
Karma: 10
Join Date: Feb 2009
Device: PRS-505
*nod* I've posted on the forum for Calibre as well, figured I'd hit both and see what people have done.

*chuckle* My wife absently asked me about reading FF on the Reader and I stumbled on your app totally on accident. It may well lead to me buying another Reader just for her because of this apps functionality.
malkie13 is offline   Reply With Quote
Old 02-14-2009, 01:56 PM   #53
ilovejedd
hopeless n00b
ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.
 
ilovejedd's Avatar
 
Posts: 2,312
Karma: 5761596
Join Date: Jan 2009
Location: in the middle of nowhere
Device: PRS-350, Nexus S, Galaxy S, Nook Color, iPhone4, iPT4, iPad 2012
Help! I want to create a PHP script that returns just the cleaned-up html output of the story. How do I go about doing that?

For example:

URL
Quote:
http://localhost/getstory.php?source=ffnet&storyid=1234567
I assume I'll have to create a getstory.php based on a modified fflag? Not exactly sure how I go about modifying fflag, though. Also, some modules (e.g. ffnet.source.php) display status messages. I assume I just need to comment out all printMsg lines from those modules?

EDIT:
Nevermind, got it working. Just had to edit two sections in the main fflag program. Didn't even need to touch any of the other modules. Additions in green, deletions in red.
Code:

	//Config
	require_once('inc/args.inc.php');
	//$config = getCliArgs();
	$config['source'] = $_GET['source'];
	$config['format'] = 'html';
	$config['storyid'] = $_GET['storyid'];

	//Output the compiled file if it the story is valid
	if ($story['meta']['title'] != "") {

		//Output file
		echo $output;

		/*
		//Create the file name
		if (!isset($config['outfile']) || $config['outfile'] == '-') $config['outfile'] = getBaseFilename($story) . ".{$config['format']}";
		
		//Output the file
		printMsg("\t* Writing output to {$config['outfile']}...\n");
		file_put_contents($config['outfile'], $output);
		*/
Oh yeah, just had a root canal so I had a bit of free time today. Just putting the finishing touches on a SugarQuill source.php and it'll be ready for posting.

Alas, porting the ffnet source to a Calibre/Python recipe is taking much longer. I have a working recipe used in conjuction with a local Apache+PHP server (that's the reason why I needed variable passing via URL) but converting the chapter retrieval and page cleaning logic to Python escapes me.

Last edited by ilovejedd; 02-17-2009 at 08:52 PM.
ilovejedd is offline   Reply With Quote
Old 02-18-2009, 09:10 PM   #54
ilovejedd
hopeless n00b
ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.
 
ilovejedd's Avatar
 
Posts: 2,312
Karma: 5761596
Join Date: Jan 2009
Location: in the middle of nowhere
Device: PRS-350, Nexus S, Galaxy S, Nook Color, iPhone4, iPT4, iPad 2012
Here are some source files, including a fixed version of fpress.source.php. The one I posted earlier had a bug in it, I think. Also included are sources for The Sugar Quill, A Single Spark (http://sesskag.com) and there's a whole folder for AdultFanFiction.net (darn their multiple subdomains). For the AFF sources, I couldn't figure out a way to pass the subdomain without creating additional command-line arguments, hence, I created a source for each archive/subdomain. I've only tested the source file for the InuYasha archive but I think the logic should work for all the other subdomains.

For the The Sugar Quill source.php, the original plan was to adapt moggie's TheSugarQuill Calibre recipe. I eventually scrapped it since I can't make sense of the Python syntax. I'm going to need to do some heavy reading on Python before I can program anything more complex than Hello World. Thankfully, PHP is remarkably similar to C/C++ which is what I'm familiar with. All source files are based heavily on erayd's ffnet.source.php.

Again, a really big erayd for releasing this program and making the source code available and rgibbo for porting it to Windows. It's become one of my most used programs as of late.
Attached Files
File Type: zip source.zip (38.9 KB, 248 views)
ilovejedd is offline   Reply With Quote
Old 02-22-2009, 06:12 AM   #55
ilovejedd
hopeless n00b
ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.
 
ilovejedd's Avatar
 
Posts: 2,312
Karma: 5761596
Join Date: Jan 2009
Location: in the middle of nowhere
Device: PRS-350, Nexus S, Galaxy S, Nook Color, iPhone4, iPT4, iPad 2012
Doesn't fetch story id: 4880173. Meta information is detected correctly but the chapter content is blank.
ilovejedd is offline   Reply With Quote
Old 02-22-2009, 06:30 AM   #56
erayd
Zealot
erayd doesn't littererayd doesn't litter
 
Posts: 128
Karma: 146
Join Date: Apr 2008
Device: Sony PRS-650
Quote:
Originally Posted by ilovejedd View Post
Doesn't fetch story id: 4880173. Meta information is detected correctly but the chapter content is blank.
Thanks - will investigate and try to figure out why this is.
erayd is offline   Reply With Quote
Old 02-22-2009, 06:36 AM   #57
erayd
Zealot
erayd doesn't littererayd doesn't litter
 
Posts: 128
Karma: 146
Join Date: Apr 2008
Device: Sony PRS-650
Quote:
Originally Posted by ilovejedd View Post
Help! I want to create a PHP script that returns just the cleaned-up html output of the story. How do I go about doing that?
What exactly did you mean here? Are you aware that fflag can output cleaned HTML already as a standard output format?

Currently supported outputs are HTML, RTF, BBeB (LRF) and ePub.

To everyone submitting modified source:Would it be useful if I gave you guys commit access to the subversion repo for this? Alternatively, is there any chance you'd be willing to submit patches to the original source? That way we can keep everything in sync and stop it diverging too much - the way things are, there's a chance we could end up with way too many branches spread all over the thread in various zipfiles.

And finally, thanks heaps to all the users and modifiers in this thread :-). It's great to see people taking an interest, I must admit I thought for a while there that there were only a couple of people interested.

Second final note: To all who have contributed code, are you happy with your changes being integrated into trunk and released under GPLv2?

Last edited by erayd; 02-22-2009 at 06:59 AM.
erayd is offline   Reply With Quote
Old 02-22-2009, 08:48 AM   #58
Valloric
Created Sigil, FlightCrew
Valloric ought to be getting tired of karma fortunes by now.Valloric ought to be getting tired of karma fortunes by now.Valloric ought to be getting tired of karma fortunes by now.Valloric ought to be getting tired of karma fortunes by now.Valloric ought to be getting tired of karma fortunes by now.Valloric ought to be getting tired of karma fortunes by now.Valloric ought to be getting tired of karma fortunes by now.Valloric ought to be getting tired of karma fortunes by now.Valloric ought to be getting tired of karma fortunes by now.Valloric ought to be getting tired of karma fortunes by now.Valloric ought to be getting tired of karma fortunes by now.
 
Valloric's Avatar
 
Posts: 1,978
Karma: 350515
Join Date: Feb 2008
Device: Sony Reader PRS 505
Quote:
Originally Posted by erayd View Post
Would it be useful if I gave you guys commit access to the subversion repo for this?
Okay idea.

Quote:
Originally Posted by erayd View Post
Alternatively, is there any chance you'd be willing to submit patches to the original source?
Much better idea.

Quote:
Originally Posted by erayd View Post
Second final note: To all who have contributed code, are you happy with your changes being integrated into trunk and released under GPLv2?
If your original source code was release under the GPL, while it's polite to ask if you could integrate their code, they don't really have a say in this. They are bound to publicly release their changes to the code under a GPL-compatible license. Legally, you don't have to ask their permission, and they don't have to give you one. You can integrate their code either way.

I'm saying all this to make sure you are aware of your rights. I applaud the intent to ask first though.
Valloric is offline   Reply With Quote
Old 02-22-2009, 08:58 AM   #59
erayd
Zealot
erayd doesn't littererayd doesn't litter
 
Posts: 128
Karma: 146
Join Date: Apr 2008
Device: Sony PRS-650
Quote:
Originally Posted by Valloric View Post
If your original source code was release under the GPL, while it's polite to ask if you could integrate their code, they don't really have a say in this. They are bound to publicly release their changes to the code under a GPL-compatible license. Legally, you don't have to ask their permission, and they don't have to give you one. You can integrate their code either way.

I'm saying all this to make sure you are aware of your rights. I applaud the intent to ask first though.
I am aware of my rights... that said, my original code wasn't actually licensed to anyone - it was made available, but there was no provision for modification or distribution (and as such, all mods in this thread are technically illegal). I would like to rectify this and place it all under a license everyone is happy with, rather than simply catering to my own whims. I prefer GPLv2 for a number of reasons, but if others would prefer something else I'd be quite happy to consider it.

I wrote this because I wanted the functionality, and I released it because I thought others may also want it - the contributions were an unexpected surprise, but I'd like to tidy this up now while it's still easy, before there are so many changes it all gets a bit out of hand.
erayd is offline   Reply With Quote
Old 02-22-2009, 05:53 PM   #60
ilovejedd
hopeless n00b
ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.ilovejedd ought to be getting tired of karma fortunes by now.
 
ilovejedd's Avatar
 
Posts: 2,312
Karma: 5761596
Join Date: Jan 2009
Location: in the middle of nowhere
Device: PRS-350, Nexus S, Galaxy S, Nook Color, iPhone4, iPT4, iPad 2012
Quote:
Originally Posted by erayd View Post
Thanks - will investigate and try to figure out why this is.
Figured it out. Lol, I didn't sleep until it was fixed. The fanfic was fairly long and the pcre.backtrack_limit was too restrictive (100,000). Changed the value to 1,000,000 and it works now.

Quote:
Originally Posted by erayd View Post
What exactly did you mean here? Are you aware that fflag can output cleaned HTML already as a standard output format?
Yep. I was just trying to figure out a way so it returns the HTML output directly/in-line. e.g. http://localhost/getstory.php?source=ffnet&storyid=1234567 would contain the cleaned HTML source instead of creating a separate cleaned HTML file. Needed this for def print_version() in a Calibre news feed while I'm still learning Python and unable to convert the chapter detection and page cleaning logic.

Quote:
Originally Posted by erayd View Post
To everyone submitting modified source:Would it be useful if I gave you guys commit access to the subversion repo for this? Alternatively, is there any chance you'd be willing to submit patches to the original source? That way we can keep everything in sync and stop it diverging too much - the way things are, there's a chance we could end up with way too many branches spread all over the thread in various zipfiles.
Would be great except I don't know how to use subversion, although I guess I could learn. Also, I'm not sure if my submitted code is Linux compatible.

Quote:
Originally Posted by erayd View Post
And finally, thanks heaps to all the users and modifiers in this thread :-). It's great to see people taking an interest, I must admit I thought for a while there that there were only a couple of people interested.
I think one of the reasons you're not getting more hits is because people aren't aware your program/service exists. I used to archive fanfics the hard way and I eventually gave up doing that since it just takes too much effort. Alas, short of posting a link on the FanFiction.Net forums (which might result in them asking you to take down the web service), I can't think of a way for people to learn about FLAG.

Quote:
Originally Posted by erayd View Post
Second final note: To all who have contributed code, are you happy with your changes being integrated into trunk and released under GPLv2?
Sure, no problem.

Addendum:
I just read a bit about subversion. I think patches would be better so we don't accidentally break something.

Last edited by ilovejedd; 02-22-2009 at 05:59 PM.
ilovejedd is offline   Reply With Quote
Reply

Tags
converter, fanfiction, fanfiction.net, grabber, lrf

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Fanfiction.net on Kindle forkyfork Amazon Kindle 26 08-07-2011 09:42 AM
bookmarks/notes grabber Reader2 Android Developer's Corner 0 10-02-2010 10:24 AM
EASY fanfiction grabber? sherryg Workshop 19 01-08-2010 04:13 AM
FLAG (Fanfiction.net Lightweight Automated Grabber) and Calibre? malkie13 Calibre 1 02-10-2009 06:43 PM


All times are GMT -4. The time now is 02:14 PM.


MobileRead.com is a privately owned, operated and funded community.