Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Readers > Sony Reader > Sony Reader Dev Corner

Notices

Closed Thread
 
Thread Tools Search this Thread
Old 04-16-2007, 09:49 AM   #1
alex_d
Addict
alex_d doesn't litteralex_d doesn't litter
 
Posts: 303
Karma: 187
Join Date: Dec 2006
Device: Sony Reader
RasterFarian 2.5 Beta

Editor's Note: These files have been blocked from being downloaded because it has been brought to our attention that the application as it currently exists includes an expired trial version of commercial software.


That's right. Back from a developmental hiatus, I bring to you the latest version of PDFRasterFarian (now renamed simply RasterFarian)!

Many new things have changed. Gone is AutoImager. Gone even, gasp, is ImageMagick! Also, RasterFarian now behaves much more like a normal Windows program. This probably isn't a big deal to those of you who were able to figure out how to get it to work before, but it'll be a big help to newbies. Lastly, it's now possible to start converting multiple files, have them get into a neat queue, and finish automatically. Very helpful, given the amount of time a single conversion takes.


On the image quality front, I've discovered a wonderful packaged called NetPBM. This thing sits in imagemagick's shadow, but it's actually much more powerful and has a much better interface. Not to mention MUCH, MUCH better documentation. Seriously, WTF is up with imagemagick's documentation?

NetPBM is able to do dilate. It also fixes some nagging issues with how imagemagick did the shrink and color reduction operations. E.g., instead of refiltering the image before resizing, netpbm lets me do a simple pixel mix. The output ends up sharper. Also, imagemagick had this perverse color-reducing scheme where it would try to find the four "optimal" colors for a palettized image. There was no way to tell it "no, you stupid crap, the sony reader can only use THESE four colors." With NetPBM, it's easy to specify this explicitly. So gone is the gray snow you'd sometimes see on backgrounds which should've been white (and on fields that should've been black).

NetPBM also introduced me to a new image processing step. Instead of sharpening, I now do "edge enhancement." The results are considerably sharper and clearer compared to 2.1.


On the installation front, we now have an installer that copies files into Program Files and sets up Start Menu entries. If you call pdfrasterfarian from the start menu, you'll get an ordinary windows dialog box asking you which file you'd like to convert and where to save it. Hopefully, in the future no one will try to install and ask "how do i install it" or "how do i run it."


On the queueing front, you can now start multiple conversions. They won't all happen simultaneously (taking up a huge amount of ram). Rather, they'll wait for eachother to complete and then start by themselves. It all happens automatically (with a couple new questions) when you run multiple instances simultaneously. Previously, if you tried that the program would simply fail.


Many things on the original "todo" and "known issues" are getting crossed off. I'm still planning to add DjVu support (probably before this comes out of beta) and also raw images input support (this one will be a tad more annoying). One-click profiles are in the works (but i'm not sure if there's a point since you can just configure the defaults and use Simple mode. try this, it's nice.)
Attachments Pending Approval
File Type: rar RasterFarian 2.5.1.part1.rar
File Type: rar RasterFarian 2.5.1.part2.rar
File Type: rar RasterFarian 2.5.1.part3.rar
File Type: rar RasterFarian 2.5.1.part4.rar

Last edited by alex_d; 06-04-2007 at 03:11 AM.
alex_d is offline  
Old 04-16-2007, 01:14 PM   #2
Bob Russell
Recovering Gadget Addict
Bob Russell ought to be getting tired of karma fortunes by now.Bob Russell ought to be getting tired of karma fortunes by now.Bob Russell ought to be getting tired of karma fortunes by now.Bob Russell ought to be getting tired of karma fortunes by now.Bob Russell ought to be getting tired of karma fortunes by now.Bob Russell ought to be getting tired of karma fortunes by now.Bob Russell ought to be getting tired of karma fortunes by now.Bob Russell ought to be getting tired of karma fortunes by now.Bob Russell ought to be getting tired of karma fortunes by now.Bob Russell ought to be getting tired of karma fortunes by now.Bob Russell ought to be getting tired of karma fortunes by now.
 
Bob Russell's Avatar
 
Posts: 5,319
Karma: 590687
Join Date: May 2004
Location: Pittsburgh, PA
Device: Note3, MacBook Air, iPad Air
I can't wait to try it! This is great news as I've been drooling over (PDF)RasterFarian and seem to be in the "newbie" category because I couldn't get the previous version to install. (It seems to have been a conflict with another program.) PDFs open up a whole new world on the Reader, so I am excited to see what they look like, and whether or not I will need magnifying lenses to read them.

Most amazing will be the ability to scan material into a PDF file on my all-in-one printer/scanner/copier and then read it on the Reader.

Woohoo!
Bob Russell is offline  
Old 04-16-2007, 02:12 PM   #3
Jary
Member
Jary began at the beginning.
 
Posts: 11
Karma: 10
Join Date: Apr 2007
Device: PRS-500
hey alex. Install is quite nice.

I might have found a little bug, not sure, when converting a file, it checks "..RasterFarian\\software\...". I tought those double "\" was typoo, no big deal, seems to work still.

Good job, I appreciate it.
Jary is offline  
Old 04-16-2007, 07:08 PM   #4
Azayzel
Cache Ninja!
Azayzel ought to be getting tired of karma fortunes by now.Azayzel ought to be getting tired of karma fortunes by now.Azayzel ought to be getting tired of karma fortunes by now.Azayzel ought to be getting tired of karma fortunes by now.Azayzel ought to be getting tired of karma fortunes by now.Azayzel ought to be getting tired of karma fortunes by now.Azayzel ought to be getting tired of karma fortunes by now.Azayzel ought to be getting tired of karma fortunes by now.Azayzel ought to be getting tired of karma fortunes by now.Azayzel ought to be getting tired of karma fortunes by now.Azayzel ought to be getting tired of karma fortunes by now.
 
Azayzel's Avatar
 
Posts: 643
Karma: 1002300
Join Date: Jan 2007
Location: Tokyo, Japan
Device: PRS-500, HTC Shift, iPod Touch, iPaq 4150, TC1100, Panasonic WordsGear
Glad to see a new revision Alex! I've actually held off on upgrading to the latest and greatest due to the amount of problems people had been reporting (besides, I was satisfied with how 2.1.4 worked so couldn't complain). Now that tax season is past and I have a bit more free time to do conversions, I plan on testing this out on some *hardly* readable comics I have & other PDF's that, in their current state, aren't Reader-friendly.
Azayzel is offline  
Old 04-16-2007, 07:21 PM   #5
Bob Russell
Recovering Gadget Addict
Bob Russell ought to be getting tired of karma fortunes by now.Bob Russell ought to be getting tired of karma fortunes by now.Bob Russell ought to be getting tired of karma fortunes by now.Bob Russell ought to be getting tired of karma fortunes by now.Bob Russell ought to be getting tired of karma fortunes by now.Bob Russell ought to be getting tired of karma fortunes by now.Bob Russell ought to be getting tired of karma fortunes by now.Bob Russell ought to be getting tired of karma fortunes by now.Bob Russell ought to be getting tired of karma fortunes by now.Bob Russell ought to be getting tired of karma fortunes by now.Bob Russell ought to be getting tired of karma fortunes by now.
 
Bob Russell's Avatar
 
Posts: 5,319
Karma: 590687
Join Date: May 2004
Location: Pittsburgh, PA
Device: Note3, MacBook Air, iPad Air
I can tell you that all my install problems are gone, and I'm up and running!!
Probably was stupid to pick a large document for my first trial... 422 pages on my slow PC, but I'll let it run and am anxious to see what comes of it.

So far, I have to say it's very clear how to get started also. I don't know what it will look like, but you sure made the process easy Alex.

Also noticed that you have a special option for two column documents so it can be split in quarters. That looks like a fantastic option.

Will let people know how it goes for me, and I'll probably try a document with columns next. Also maybe a document I tried first time I touched a Reader that was unreadable. That one is a tough test, but who knows, with a little help from RF... maybe "miracles" are possible.
Bob Russell is offline  
Old 04-17-2007, 12:21 AM   #6
eskin
Member
eskin began at the beginning.
 
Posts: 17
Karma: 10
Join Date: Apr 2007
Device: Sony Reader
Thank you, this is really wonderful stuff!
eskin is offline  
Old 04-17-2007, 02:29 AM   #7
gdxf
Enthusiast
gdxf began at the beginning.
 
Posts: 48
Karma: 27
Join Date: Oct 2006
Device: Sony Reader PRS-500
This is a great program! I have been eagerly waiting for testing this new release because with the development of this software I see endless advantages of reading pdf on the Sony Reader over other file formats. However, on my computer, it won't split the pdf page into half in the processed lrf file, even though it can split the page into quarters. The scenario is like this: A4 sized pdf input results in A4 sized lrf output. It's so strange. Another problem I encounter is that if I use auto-crop option, the produced lrf file will only contain blank pages. These problems also existed in previous versions. I don't know if anyone else has encountered these problems. Anyone can tell me what could be the cause? Thanks.
gdxf is offline  
Old 04-17-2007, 05:47 AM   #8
alex_d
Addict
alex_d doesn't litteralex_d doesn't litter
 
Posts: 303
Karma: 187
Join Date: Dec 2006
Device: Sony Reader
gdxf, to have the program "split the page in half" after you select the appropriate conversion mode, you have to switch the Sony Reader into Landscape. Do this by holding the Size button for 5+ seconds.

I've now updated the UI text a bit to make this step clearer. I must say, the UI screen asking and explaining layouts has had more evolution than almost anything else. Some developers take the darwinian view that you make a useable UI and not try to baby anyone or give it a second though (e.g. many of those working in unix/linux.) Meanwhile, other developers (like Apple) take it as their primary aim to remove everything that could take an iota of brainpower to figure out (the "simplest but no simpler" that keeps getting repeated to my ire). My opinion on this is that you have to keep trying to make the interface more and more intuitive, but then use that as an opportunity to make it more sophisticated in other places. This pushes progress and maintains the Flynn Effect (no, seriously). Simplifying in order to complexify. For all the bad stuff that gets piled on microsoft, I think it's one of the few big developers that understands the inherent correctness of this approach.

anyway...

The autocropping causing blank pages, though, sounds like a serious problem. Can you a) post your PDF and b) use the "preview" function and then copy all the text that you see get printed?* That last step will tell me some debug info, but only if previewing also gives you blank pages.

*To copy text from a CMD window, right-click on the title bar (the bar that has the X and minimize buttons), choose properities, and then enable QuickEdit mode. This lets you highlight text and copy it by right-clicking on it. Copy everything, even if you have to scroll up.
alex_d is offline  
Old 04-17-2007, 12:07 PM   #9
eskin
Member
eskin began at the beginning.
 
Posts: 17
Karma: 10
Join Date: Apr 2007
Device: Sony Reader
This is really great stuff, I just picked up my Readers last Friday, and your app has made it possible for me to do what I intended with the device (portable tune books for traditional Irish music).

One feature that would be extremely helpful would be if the app could somehow respect any cropping that has been applied using Acrobat.

Here's the scenario. I have a book from a 3rd party that was printed formatted for letter size paper, and it has a massive table of contents already setup. Using Acrobat, I'm able to go into the doc, crop the margins off and save it, resulting in a book that has all the meta-data, and no margins when displayed using Acrobat Reader. I realize the crop info is just additional meta-data on the document. If I re-PDF it by printing it with either Acrobat PDF or CutePDF, I lose the table of contents.

If I bring this cropped book into RasterFarian, it doesn't appear to respect the cropping information in the PDF file when it rasterizes the page, so if the original doc has, for example crop lines and such at the edges of the pages, they show up again in the LRF file.

In a perfect world, it would be great if RasterFarian would use the crop information in the PDF file as the starting bounds rectangle for the auto-sizing functionality.

Alternatively, do you know of a way to retain the TOC meta-data when re-printing a document to PDF using the Acrobat PDF or CutePDF print drivers? That would also achieve the same results.

Another question is on how RasterFarian handles hierarchical table of contents in a PDF document, it appears to flatten the TOC in the final output LRF document. In my case, where I have over 1000 tunes with a TOC in the PDF doc that organizes them by name, key, rhythm, etc.. After running through RasterFarian, I end up with over 3000 individual TOC entries which are nearly impossible to navigate since there is no way on the reader to jump ahead when using the TOC. Any chance for support of hierarchical TOCs in a future release? That would be absolutely awesome!

Thanks,

Michael Eskin
San Diego, CA

Last edited by eskin; 04-17-2007 at 06:48 PM.
eskin is offline  
Old 04-17-2007, 07:15 PM   #10
gdxf
Enthusiast
gdxf began at the beginning.
 
Posts: 48
Karma: 27
Join Date: Oct 2006
Device: Sony Reader PRS-500
Alex, thanks for your tips! Now I finally understand the how the split page works: it only works on the Sony Reader, not on the connect software as I originally thought. I wish I could adjust the overlapping area manually, but it seems it's the limit on the part of Sony Reader. As for autocropping leading to blank page, I got the following message:


---------------------------------------
Generating preview....
...Creating EPS file from PDF page
Error: PDF version 1.6 -- xpdf supports version 1.5 (continuing anyway)
Error: Unknown character collection 'Adobe-GB1'
Error: Couldn't find 'UniGB-UCS2-H' CMap file for 'Adobe-GB1' collection
Error: Unknown CMap 'UniGB-UCS2-H' for character collection 'Adobe-GB1'
Error: Unknown character collection 'Adobe-GB1'
Error: Couldn't find 'UniGB-UCS2-V' CMap file for 'Adobe-GB1' collection
Error: Unknown CMap 'UniGB-UCS2-V' for character collection 'Adobe-GB1'
Error: Unknown character collection 'Adobe-GB1'
Error: Couldn't find 'UniGB-UCS2-H' CMap file for 'Adobe-GB1' collection
Error: Unknown CMap 'UniGB-UCS2-H' for character collection 'Adobe-GB1'
Error: Unknown character collection 'Adobe-GB1'
Error: Couldn't find 'UniGB-UCS2-V' CMap file for 'Adobe-GB1' collection
Error: Unknown CMap 'UniGB-UCS2-V' for character collection 'Adobe-GB1'
Error: Unknown font tag 'F00'
Error (2120): No font in show
Error (2171): No font in show
Error (2235): No font in show
Error (2283): No font in show
Error (2322): No font in show
Error (2373): No font in show
Error: Unknown font tag 'F00'
...Performing autocropping
gawk: C:\Program Files\RasterFarian\\modules\crop_ps.awk:19: EOF
gawk: C:\Program Files\RasterFarian\\modules\crop_ps.awk:19: ^ syntax erro

gawk: C:\Program Files\RasterFarian\\modules\crop_ps.awk:19: EOF print;
gawk: C:\Program Files\RasterFarian\\modules\crop_ps.awk:19: ^ inval
' in expression
...Rastering PNG file from EPS file (this step may take several minutes)
NOTE: Some especially complex pages may take a VERY long time (e.g. an hour)
If this seems to be occuring, you may go into the Task Manager
and kill the gswin32c.exe process. This will result in a blank page.


WARNING: The page is blank. It's OK.
However, you will probably want to preview a page that contains text.

Opening MSPaint...
Note: This image will look MUCH different on an eInk screen.
Typically, it will look smoother and less bold.

Are you satisfied with the result?
y) yes
b) go back (return to conversion options)
p) preview (to preview another page)
gdxf is offline  
Old 04-18-2007, 02:10 PM   #11
ashkulz
Addict
ashkulz will become famous soon enoughashkulz will become famous soon enoughashkulz will become famous soon enoughashkulz will become famous soon enoughashkulz will become famous soon enoughashkulz will become famous soon enoughashkulz will become famous soon enough
 
ashkulz's Avatar
 
Posts: 350
Karma: 705
Join Date: Dec 2006
Location: Mumbai, India
Device: Kindle 1/REB 1200
Quote:
Originally Posted by alex_d
I must say, the UI screen asking and explaining layouts has had more evolution than almost anything else. Some developers take the darwinian view that you make a useable UI and not try to baby anyone or give it a second though (e.g. many of those working in unix/linux.)
I know that one is directed towards me

Seriously though, it's good to see you resuming work on RasterFarian. I'd have preferred it if we could have merged our work, but looks like we have slightly different priorities (and opinions) when it comes to implementing things.
ashkulz is offline  
Old 04-18-2007, 02:22 PM   #12
ashkulz
Addict
ashkulz will become famous soon enoughashkulz will become famous soon enoughashkulz will become famous soon enoughashkulz will become famous soon enoughashkulz will become famous soon enoughashkulz will become famous soon enoughashkulz will become famous soon enough
 
ashkulz's Avatar
 
Posts: 350
Karma: 705
Join Date: Dec 2006
Location: Mumbai, India
Device: Kindle 1/REB 1200
Quote:
Originally Posted by eskin
In a perfect world, it would be great if RasterFarian would use the crop information in the PDF file as the starting bounds rectangle for the auto-sizing functionality.
alex_d, I have stopped using bbox computation via Ghostscript in PDFRead because of this very reason. A much better approach is to simply convert a page to EPS, rasterize via Ghostscript using -dEPSCrop and then finally remove all pixels which have the background color. I do that via using PIL, you can achieve the same effect by using pnmcrop from NetPBM (which you use). It will do very nice cropping, even more better than GS (You might want to leave it as an option though, like I have with --gscrop).

And there's one more advantage: Ghostscript problems on blank pages go away with this approach.

BTW, I was inadvertently using the image enhancement step you added via pgmenhance; the same thing is used internally in rbmake (which is the finally step for my 1100), so I've added it as an option in PDFRead, too

Last edited by ashkulz; 04-18-2007 at 02:26 PM.
ashkulz is offline  
Old 04-19-2007, 10:14 AM   #13
alex_d
Addict
alex_d doesn't litteralex_d doesn't litter
 
Posts: 303
Karma: 187
Join Date: Dec 2006
Device: Sony Reader
eskin, right now it seems manual cropping doesn't get respected when you use "autocrop", but it is obeyed when you turn that feature off.

"In a perfect world, it would be great if RasterFarian would use the crop information in the PDF file as the starting bounds rectangle for the auto-sizing functionality."

Agreed.

I'm going to try implementing Ashkulz's suggestion, it's very elegant (esp. since there's even a netpbm exe for this). This ought to also fix gdfx's problem (though I have no idea why gawk isn't working for him). I was a little concerned before in trying such an approach because I wanted to keep the resize operation as a neat reduce by an integer (for the best crispness). However, I doubt the difference will be noticeable.

Ashkulz, tho, why do you think the ghostscript-based cropping is still worth supporting? Also, did you understand what that third guy was doing to get autocropping that doesn't quit after the first dark pixel?


Now, as for heirchical TOC.. asfaik, the Reader's TOC doesn't support nested levels or even indentation. (They're big believers in the Apple approach, but they're much lazier and it comes out even worse.) However, the Reader does support links inside LRFs. It should be possible to do something interesting with that. The simplest would be to redo a ToC with indentations with maybe 20-50 entries per page. Slightly more complex would be to have multiple levels of links (tho it's probably not very desirable for people who don't have 3k bookmarks). Even better would be to add some sort of "navigation panel" at the start of books. Maybe it could give you links to every 5th page (since you can't fn enter page numbers). Anyway, I'm talking too much. Anybody else have better ideas?

Also, i've never done anything with LRF links. a) how do you do it? and b) can you navigate between them in 2D?
alex_d is offline  
Old 04-19-2007, 10:35 AM   #14
curiouser
Junior Member
curiouser began at the beginning.
 
Posts: 9
Karma: 10
Join Date: Jul 2006
Quote:
Also, did you understand what that third guy was doing to get autocropping that doesn't quit after the first dark pixel?
Considering it is written in python using PIL, and is packaged up in a function, it would be easy to integrate into ashkulz's work. Sorry I don't have something written in C that could be readily integrated into RasterFarian. It not only trims the extra white space, but does so while maintaining the correct aspect ratio and centering the image. As I mentioned before, for scanned documents, where you'll inevitably have some dirty pixels, it works much better than the current use of the getbbox() method. Though the code may look a little clumsy, I never noticed any real speed penalty in image processing.

My schedule clears up in a couple of weeks. Hope to play again soon.
curiouser is offline  
Old 04-20-2007, 06:38 AM   #15
alex_d
Addict
alex_d doesn't litteralex_d doesn't litter
 
Posts: 303
Karma: 187
Join Date: Dec 2006
Device: Sony Reader
thanks, curiouser. I think me or ashkultz can use py2exe to convert your function into something that can be inserted into rasterfarian. Can you make a quick stand-alone py script that takes in a range of overlap 0-to-whatever and input from the pipe and outputs to the pipe in a netpbm format? If that's that at all complicated, any sort of input/output would be fine.

P.S. I'm not sure centering the image is best. If you view the document in landscape and you only have a trailing paragraph on the page, wouldn't it be better for it to go to the middle-top?
alex_d is offline  
Closed Thread

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Beta Beta Test of Major New Features Starson17 Calibre 45 05-17-2010 10:55 AM
PRS-500 RasterFarian patched and available again! NatCh Sony Reader Dev Corner 7 05-25-2009 05:16 PM
PRS-500 rasterfarian question drogo Sony Reader Dev Corner 2 10-05-2007 10:18 AM
PRS-500 Crop in RasterFarian 2.5 ddavtian Sony Reader Dev Corner 4 08-01-2007 07:40 PM
RasterFarian V2.5 Beta for Sony Reader released Alexander Turcic Sony Reader 0 04-17-2007 02:53 PM


All times are GMT -4. The time now is 10:24 AM.


MobileRead.com is a privately owned, operated and funded community.