FLAG (Fanfiction.net Lightweight Automated Grabber) - Page 2

erayd · 07-07-2008, 02:30 AM

New release is up (r23), now supports RTF output. If you wish to use RTF, you will need to have pandoc installed. I've also updated the original post.

mateo · 07-07-2008, 02:34 PM

Quote:

Originally Posted by erayd

Thanks for this - can you supply a story ID that doesn't render correctly? Which version are you using?

Sure, 4250149. I was using an SVN copy.

erayd · 07-07-2008, 10:11 PM

Quote:

Originally Posted by mateo

Sure, 4250149. I was using an SVN copy.

Thanks - this should be fixed in the latest subversion build (r24). Please let me know if you have any further issues.

whitearrow · 07-14-2008, 10:41 PM

erayd, I think the you might want to talk to these folks:

http://www.transformativeworks.org/p...s/archive.html

They are building all new archive software from the ground up, and I'm sure they'd love to have this capability from the get-go. And unlike fanfic.net, it's not going to be run for profit.

SeaWolf · 07-15-2008, 08:59 AM

Quote:

Originally Posted by whitearrow

erayd, I think the you might want to talk to these folks:

I think their software is going to written in Ruby on Rails though, rather than PHP, which might make integrating the existing work a little hard, but being able to export in eBook formats from their archive would be a great feature to have when they get up and running.

Speaking of which, any luck getting the Fanfic.net people to agree to you running a public FLAG server? I'd love to be able to give the software a try.

erayd · 07-15-2008, 11:06 PM

I haven't had a reply from ff.net yet - my current plan is if I haven't received a response by Friday evening (+12) I'll just go ahead and do it anyway, and take it down if they complain about it.

Whitearrow, adding such capability isn't hard, and if they're using RoR as SeaWolf says then my code would be rather useless to them anyway unless they're calling it as an external application. I have no problem with them using it though.

myridom · 07-15-2008, 11:18 PM

Awesome.. I have been doing a few stories by hand just to have backup material if I get bored. Glad someone started down this road.. LOL

Myridom

erayd · 07-19-2008, 12:48 AM

The fanfiction.net people haven't replied, so I've put this app online as a webservice here. Enjoy

. The webservice is currently running a subversion copy (r26). Note that not all features of the CLI version are available (--notoc, automatic file naming etc).

As always, any comments, suggestions, bug reports, etc are more than welcome.

SeaWolf · 07-20-2008, 11:41 PM

Minor bug report:
If I enter a six digit story ID I get the error

Code:

Invalid story ID (should be a 7-digit number).

But that's not true, there are certainly six digit IDs and probably shorter ones as well. It's solvable by padding the ID with a zero at the start though.

erayd · 07-21-2008, 03:00 AM

Quote:

Originally Posted by SeaWolf

Minor bug report:
If I enter a six digit story ID I get the error

Code:

Invalid story ID (should be a 7-digit number).

But that's not true, there are certainly six digit IDs and probably shorter ones as well. It's solvable by padding the ID with a zero at the start though.

Oops, I didn't realise that. It will now accept story IDs from 1-7 digits.

Lane · 11-16-2008, 02:17 AM

This tool is fantastic. I've tried other FF.net downloaders and think it's the best among them. I only wish someone would port it to windows so that I could use the software instead of the website. For now, I hope you keep the app online for those of us not running Linux. Thank you for this great tool, erayd.

erayd · 11-16-2008, 02:58 AM

Glad to hear you find it useful :-). I'd actually given up on maintaining it, as it hasn't been getting much use, but I do plan to keep it online. Are there any particular features you'd like it to have that it doesn't have currently?

ficbot · 11-16-2008, 02:13 PM

I love it! Thanks so much. The HTML one generated decent plain text when I pasted it (unformatted) into my word processor. I was easily able to save it as a word doc and convert it to ereader.

erayd · 11-17-2008, 04:38 AM

FLAG now supports output to ePub. Latest version (r29) is attached to the first post in this thread, or can be used as a web service here.

rgibbo · 12-05-2008, 11:55 AM

Hi been using your software for quite a bit now, I tried using others but none of the others are as good as this one, i've mainly been using the version hosted on erayd's server since I use a windows machine and just saved the files straight onto my reader.

Anyway, the online version has suddenly stopped working as it wouldn't download anything, I tried at least 10 different codes and they all wouldn't work. I'm a bit of a PHP developer so I downloaded it and tried it on my machine, after a lot of hacking to just get it running on my windows machine,the exact same problem came up where it wouldn't download anything.

After scanning through the code for about 15 minutes, I managed to found the problem, it lies with how the tool gets the page data from the site as it sends out a user agent string to help identify it, it seems fanfiction has blocked the tool's user agent string so the tool can't download anything. I decided to change the user agent string which was FLAG/1.0 or similar to the something differentand it seems to be working again now.

Well that's the solution if anyone else wants it.

07-07-2008, 02:30 AM	#16
erayd Zealot Posts: 134 Karma: 146 Join Date: Apr 2008 Device: Onyx Boox Poke 2	New release is up (r23), now supports RTF output. If you wish to use RTF, you will need to have pandoc installed. I've also updated the original post. Last edited by erayd; 07-07-2008 at 03:23 AM. Reason: r21 contained a bug - simpletos should not be on by default!

07-19-2008, 12:48 AM	#23
erayd Zealot Posts: 134 Karma: 146 Join Date: Apr 2008 Device: Onyx Boox Poke 2	The fanfiction.net people haven't replied, so I've put this app online as a webservice here. Enjoy . The webservice is currently running a subversion copy (r26). Note that not all features of the CLI version are available (--notoc, automatic file naming etc). As always, any comments, suggestions, bug reports, etc are more than welcome. Last edited by erayd; 07-19-2008 at 01:12 AM.

07-20-2008, 11:41 PM	#24
SeaWolf Connoisseur Posts: 63 Karma: 65091 Join Date: Jul 2008 Location: Sydney, Australia Device: Kindle Paperwhite WiFi	Minor bug report: If I enter a six digit story ID I get the error Code: Invalid story ID (should be a 7-digit number). But that's not true, there are certainly six digit IDs and probably shorter ones as well. It's solvable by padding the ID with a zero at the start though.

Thread Tools	Search this Thread
Show Printable Version Email this Page	Search this Thread: Advanced Search

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Fanfiction.net on Kindle	forkyfork	Amazon Kindle	26	08-07-2011 08:42 AM
bookmarks/notes grabber	Reader2	Android Developer's Corner	0	10-02-2010 09:24 AM
EASY fanfiction grabber?	sherryg	Workshop	19	01-08-2010 03:13 AM
FLAG (Fanfiction.net Lightweight Automated Grabber) and Calibre?	malkie13	Calibre	1	02-10-2009 05:43 PM

07-14-2008, 10:41 PM	#19
whitearrow Guru Posts: 808 Karma: 2260766 Join Date: Apr 2008 Device: Kindle Oasis 2	erayd, I think the you might want to talk to these folks: http://www.transformativeworks.org/p...s/archive.html They are building all new archive software from the ground up, and I'm sure they'd love to have this capability from the get-go. And unlike fanfic.net, it's not going to be run for profit.

07-15-2008, 11:06 PM	#21
erayd Zealot Posts: 134 Karma: 146 Join Date: Apr 2008 Device: Onyx Boox Poke 2	I haven't had a reply from ff.net yet - my current plan is if I haven't received a response by Friday evening (+12) I'll just go ahead and do it anyway, and take it down if they complain about it. Whitearrow, adding such capability isn't hard, and if they're using RoR as SeaWolf says then my code would be rather useless to them anyway unless they're calling it as an external application. I have no problem with them using it though.

07-15-2008, 11:18 PM	#22
myridom Enthusiast Posts: 41 Karma: 108 Join Date: Jul 2008 Location: Panama City, FL Device: Kindle	Awesome.. I have been doing a few stories by hand just to have backup material if I get bored. Glad someone started down this road.. LOL Myridom

11-16-2008, 02:17 AM	#26
Lane Junior Member Posts: 2 Karma: 10 Join Date: Nov 2008 Device: HP iPaq 210	This tool is fantastic. I've tried other FF.net downloaders and think it's the best among them. I only wish someone would port it to windows so that I could use the software instead of the website. For now, I hope you keep the app online for those of us not running Linux. Thank you for this great tool, erayd.

11-16-2008, 02:58 AM	#27
erayd Zealot Posts: 134 Karma: 146 Join Date: Apr 2008 Device: Onyx Boox Poke 2	Glad to hear you find it useful :-). I'd actually given up on maintaining it, as it hasn't been getting much use, but I do plan to keep it online. Are there any particular features you'd like it to have that it doesn't have currently?

11-16-2008, 02:13 PM	#28
ficbot Wizard Posts: 2,409 Karma: 4132096 Join Date: Sep 2008 Device: Kindle Paperwhite/iOS Kindle App	I love it! Thanks so much. The HTML one generated decent plain text when I pasted it (unformatted) into my word processor. I was easily able to save it as a word doc and convert it to ereader.

11-17-2008, 04:38 AM	#29
erayd Zealot Posts: 134 Karma: 146 Join Date: Apr 2008 Device: Onyx Boox Poke 2	FLAG now supports output to ePub. Latest version (r29) is attached to the first post in this thread, or can be used as a web service here.

12-05-2008, 11:55 AM	#30
rgibbo Junior Member Posts: 5 Karma: 14 Join Date: Dec 2008 Device: Sony PRS-505	Hi been using your software for quite a bit now, I tried using others but none of the others are as good as this one, i've mainly been using the version hosted on erayd's server since I use a windows machine and just saved the files straight onto my reader. Anyway, the online version has suddenly stopped working as it wouldn't download anything, I tried at least 10 different codes and they all wouldn't work. I'm a bit of a PHP developer so I downloaded it and tried it on my machine, after a lot of hacking to just get it running on my windows machine,the exact same problem came up where it wouldn't download anything. After scanning through the code for about 15 minutes, I managed to found the problem, it lies with how the tool gets the page data from the site as it sends out a user agent string to help identify it, it seems fanfiction has blocked the tool's user agent string so the tool can't download anything. I decided to change the user agent string which was FLAG/1.0 or similar to the something differentand it seems to be working again now. Well that's the solution if anyone else wants it.

Advert

Advert