Proof Of Concept - RSS->PDF

sartori · 10-06-2006, 11:23 PM

Hi, this is my first post and I'm one of the many waiting for Sony to deliver my reader so in the meantime I've been working on creating content.

If it's possible for someone who has their reader could they check out my progress and see if it works before I continue development.

If you go to http://www.mysonyreader.com you will be presented with a dynamically created pdf containing the last 20 stories from gizmodo.com.

Just save the pdf and check that it works and is readable.

I think I have it generating the correct page size for the reader although I can't check the font size, margins or contrast etc. Right now it strips all the html from the file so it's plain text.

My future plans include -

- Being able to subscribe to multiple feeds and then download one combined pdf or even have it emailed you daily

- Including the images from the original feed

- Allowing you to select the font size & style

- Actually building a site to house all of this

- Adding a developers area to above site so people can contribute to this project.

Feedback would be greatly appreciated.

If anyone is interested in playing with the code to do this, let me know and I'll post the details.

Rob

P.S. If you get a domain placeholder page - give it a few hours as I just registered the domain and it may not have propagated everywhere yet.

Greg G · 10-06-2006, 11:51 PM

I loaded it up and it looks great! Pagination is perfect!

Kaitou Ace · 10-07-2006, 12:07 AM

What language is the code in? I'd be interested in messing with it.

sartori · 10-07-2006, 12:11 AM

It's in php. I'll tidy it up a bit and add some comments and post it for you.

Rob

Kaitou Ace · 10-07-2006, 12:18 AM

Ooh, php. I like my php. Thanks, I'll see if I can get anything interesting/useful out of it.

sartori · 10-07-2006, 12:32 AM

Ok, you can see the code at :

http://www.mysonyreader.com/index_commented.phps

The actual pdf generation is about 10 lines of code in the middle.

Keep in mind I'm not a php programmer by trade so it may be a bit untidy - hopefully theres not too many wtf? parts in there

I'd been interested in taking a look at any improvements you add.

Rob

bugsbunny14 · 10-07-2006, 04:12 AM

thanks sartori , great work

Laurens · 10-07-2006, 05:03 AM

You do know that you can't just publically redistribute copyrighted content, do you?

Looks good, though.

sartori · 10-07-2006, 05:40 AM

Yes I know - The included libraries are GPL - If you google for them you can see their individual licenses. Once I get things together a little more I will include all the details in the code. I was more looking to see if it works at this point.

As far as the content - it's from the gizmodo RSS feed - not sure on the legality of distributing it - but I did not remove any copyright notices or advertising that they include in their feeds.

There are so many news aggregator services out there that have this content and all different rss feeds so I'm not sure how that works. The only difference is this code repackages it and sends it to your browser as a pdf rather than html.

I noticed that many feeds only offer a brief summary of the articles (or even just a title) so that you have to go to their site to read the whole thing. I wouldn't 'scrape' their sites to get the full content as they obviously don't want you to do that.

If anyone has a definitive answer or more opinions on that it would be good to know. If I can get the code together it might be a good idea to approach the feed suppliers to ask permission? Maybe have people suggest feeds that they would like to read?

If it turns out that it's not a good idea to use the feeds, I will probably continue to develop the code - one of my ideas for it is to interface with your own google calendar rss feed so you could take 2-3 weeks of your calendar with you on your reader.

Rob

Edit: For now I have switched the feed over to http://en.wikinews.org/wiki/Main_Page - it's a free news feed that is released under the creative commons license. I'll look into this in more detail tomorrow - after I get some sleep

Slava · 10-07-2006, 08:20 AM

Rob,

it looks nice so far. I agree that a lot depens on the content (and validity) of RSS feed itself. In ideal case I would like to see something like this

-----------------------------------
Site/RSS Name
Generation Date
TOC:
Article name 1 (<--Link)
Article name 2 (<--Link)
Article name 3 (<--Link)
...

Article name 1
Picture
Text
...

And NO comments

-----------------------------------

On practice though, it means that one needs to scan RSS feed, then follow URL to an article, and grab only meanfull part of Ads infested page. So you need to know page structure of every site and follow its changes. In Gizmodo case it would be everything between

Code:

<div class="PostText">
</div>

And that's a lot of hard work

And I believe that Sony should do it

Small thing. Instead of source code, http://www.mysonyreader.com/index_commented.phps page redirects to Gizmodo pdf.

Kaitou Ace · 10-07-2006, 08:55 AM

As Slava said, the security settings on your server seem to keep the source from being seen in the phps page.

TadW · 10-07-2006, 02:41 PM

Looks like this could turn out into a useful tool! Add me to the list of those interested to play with the code

sartori · 10-07-2006, 03:40 PM

Sorry that the link didn't work - the code is attached as a text file.

Rob

10-06-2006, 11:23 PM	#1
sartori Connoisseur Posts: 54 Karma: 29 Join Date: Oct 2006	Proof Of Concept - RSS->PDF Hi, this is my first post and I'm one of the many waiting for Sony to deliver my reader so in the meantime I've been working on creating content. If it's possible for someone who has their reader could they check out my progress and see if it works before I continue development. If you go to http://www.mysonyreader.com you will be presented with a dynamically created pdf containing the last 20 stories from gizmodo.com. Just save the pdf and check that it works and is readable. I think I have it generating the correct page size for the reader although I can't check the font size, margins or contrast etc. Right now it strips all the html from the file so it's plain text. My future plans include - - Being able to subscribe to multiple feeds and then download one combined pdf or even have it emailed you daily - Including the images from the original feed - Allowing you to select the font size & style - Actually building a site to house all of this - Adding a developers area to above site so people can contribute to this project. Feedback would be greatly appreciated. If anyone is interested in playing with the code to do this, let me know and I'll post the details. Rob P.S. If you get a domain placeholder page - give it a few hours as I just registered the domain and it may not have propagated everywhere yet.

10-07-2006, 12:11 AM	#4
sartori Connoisseur Posts: 54 Karma: 29 Join Date: Oct 2006	Language It's in php. I'll tidy it up a bit and add some comments and post it for you. Rob

10-07-2006, 12:18 AM	#5
Kaitou Ace Zealot Posts: 126 Karma: 1352743 Join Date: Oct 2002	Ooh, php. I like my php. Thanks, I'll see if I can get anything interesting/useful out of it. Last edited by Kaitou Ace; 10-07-2006 at 12:27 AM.

10-07-2006, 05:40 AM	#9
sartori Connoisseur Posts: 54 Karma: 29 Join Date: Oct 2006	Yes I know - The included libraries are GPL - If you google for them you can see their individual licenses. Once I get things together a little more I will include all the details in the code. I was more looking to see if it works at this point. As far as the content - it's from the gizmodo RSS feed - not sure on the legality of distributing it - but I did not remove any copyright notices or advertising that they include in their feeds. There are so many news aggregator services out there that have this content and all different rss feeds so I'm not sure how that works. The only difference is this code repackages it and sends it to your browser as a pdf rather than html. I noticed that many feeds only offer a brief summary of the articles (or even just a title) so that you have to go to their site to read the whole thing. I wouldn't 'scrape' their sites to get the full content as they obviously don't want you to do that. If anyone has a definitive answer or more opinions on that it would be good to know. If I can get the code together it might be a good idea to approach the feed suppliers to ask permission? Maybe have people suggest feeds that they would like to read? If it turns out that it's not a good idea to use the feeds, I will probably continue to develop the code - one of my ideas for it is to interface with your own google calendar rss feed so you could take 2-3 weeks of your calendar with you on your reader. Rob Edit: For now I have switched the feed over to http://en.wikinews.org/wiki/Main_Page - it's a free news feed that is released under the creative commons license. I'll look into this in more detail tomorrow - after I get some sleep Last edited by sartori; 10-07-2006 at 06:12 AM.

10-07-2006, 08:20 AM	#10
Slava Addict Posts: 285 Karma: 129 Join Date: Sep 2006 Location: New England Device: PRS-500 · iPad · K3	Rob, it looks nice so far. I agree that a lot depens on the content (and validity) of RSS feed itself. In ideal case I would like to see something like this ----------------------------------- Site/RSS Name Generation Date TOC: Article name 1 (<--Link) Article name 2 (<--Link) Article name 3 (<--Link) ... Article name 1 Picture Text ... And NO comments ----------------------------------- On practice though, it means that one needs to scan RSS feed, then follow URL to an article, and grab only meanfull part of Ads infested page. So you need to know page structure of every site and follow its changes. In Gizmodo case it would be everything between Code: <div class="PostText"> </div> And that's a lot of hard work And I believe that Sony should do it Small thing. Instead of source code, http://www.mysonyreader.com/index_commented.phps page redirects to Gizmodo pdf.

10-06-2006, 11:51 PM	#2
Greg G Groupie Posts: 181 Karma: 66 Join Date: Sep 2006 Location: NJ Device: Kindle Papwerwhite	I loaded it up and it looks great! Pagination is perfect!

10-07-2006, 12:07 AM	#3
Kaitou Ace Zealot Posts: 126 Karma: 1352743 Join Date: Oct 2002	What language is the code in? I'd be interested in messing with it.

10-07-2006, 12:32 AM	#6
sartori Connoisseur Posts: 54 Karma: 29 Join Date: Oct 2006	Ok, you can see the code at : http://www.mysonyreader.com/index_commented.phps The actual pdf generation is about 10 lines of code in the middle. Keep in mind I'm not a php programmer by trade so it may be a bit untidy - hopefully theres not too many wtf? parts in there I'd been interested in taking a look at any improvements you add. Rob

10-07-2006, 04:12 AM	#7
bugsbunny14 Connoisseur Posts: 64 Karma: 10 Join Date: Sep 2006	thanks sartori , great work

10-07-2006, 05:03 AM	#8
Laurens Jah Blessed Posts: 1,295 Karma: 1373 Join Date: Apr 2003 Location: The Netherlands Device: iPod Touch	You do know that you can't just publically redistribute copyrighted content, do you? Looks good, though.

10-07-2006, 08:55 AM	#11
Kaitou Ace Zealot Posts: 126 Karma: 1352743 Join Date: Oct 2002	As Slava said, the security settings on your server seem to keep the source from being seen in the phps page.

10-07-2006, 02:41 PM	#12
TadW Uebermensch Posts: 2,583 Karma: 1094606 Join Date: Jul 2003 Location: Italy Device: Kindle	Looks like this could turn out into a useful tool! Add me to the list of those interested to play with the code

Thread Tools	Search this Thread
Show Printable Version Email this Page	Search this Thread: Advanced Search

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
DR800 Shutdown mode - proof of concept	Shaggy	iRex	51	09-16-2012 06:39 AM
Proof of Concept: TeX-based ebook program in iPad	frabjous	General Discussions	2	07-09-2010 08:46 AM
English News RSS PDF	yunhaid	iRex	8	05-27-2007 10:06 PM
English News RSS PDF	yunhaid	Sony Reader	0	05-24-2007 09:30 PM
RSS to PDF?	Devlar	iRex	19	11-04-2006 03:59 AM

Advert

Advert