Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre > Conversion

Notices

Reply
 
Thread Tools Search this Thread
Old 01-24-2011, 04:20 AM   #1
diamante
piedra preciosa
diamante has a spectacular aura aboutdiamante has a spectacular aura aboutdiamante has a spectacular aura aboutdiamante has a spectacular aura aboutdiamante has a spectacular aura aboutdiamante has a spectacular aura aboutdiamante has a spectacular aura aboutdiamante has a spectacular aura aboutdiamante has a spectacular aura aboutdiamante has a spectacular aura aboutdiamante has a spectacular aura about
 
Posts: 44
Karma: 4170
Join Date: Jan 2011
Location: Boston, Massachusetts
Device: Sony PRS-350, Kindle 2, several Palm OS devices
Question Formatting issue converting eReader PDB e-books

Calibre is awesome. I'm blown away by how it can convert my old e-books and send them to either my Kindle or my Sony Reader, seamlessly handling the necessary conversions.

I've noticed one small issue with eReader PDBs that I've converted. It pertains to extra space that's used between paragraphs to indicate the passage of time or a change in point of view. Apparently how this is displayed depends on the e-reader device and software. I've noticed that even in the old eReader world, these spaces are indicated with three centered asterisks between paragraphs in older versions of eReader for Windows and Palm OS, while the most recent version of eReader for Windows simply puts more space between the paragraphs without displaying asterisks or any other "horizontal rule," if that's the correct term.

In any case, Calibre seems to ignore whatever element is used in the original PDBs when converting them to EPUB and MOBI. Is there any way to make it recognize and preserve the extra spacing, or insert some kind of horizontal rule? I know I'm being demanding here, but since Calibre is already so elegant there must be a way to do this...
diamante is offline   Reply With Quote
Old 01-24-2011, 06:59 AM   #2
user_none
Sigil & calibre developer
user_none ought to be getting tired of karma fortunes by now.user_none ought to be getting tired of karma fortunes by now.user_none ought to be getting tired of karma fortunes by now.user_none ought to be getting tired of karma fortunes by now.user_none ought to be getting tired of karma fortunes by now.user_none ought to be getting tired of karma fortunes by now.user_none ought to be getting tired of karma fortunes by now.user_none ought to be getting tired of karma fortunes by now.user_none ought to be getting tired of karma fortunes by now.user_none ought to be getting tired of karma fortunes by now.user_none ought to be getting tired of karma fortunes by now.
 
user_none's Avatar
 
Posts: 2,436
Karma: 950001
Join Date: Jan 2009
Location: Florida, USA
Device: Nook STR
Soft scene breaks are something that has come to my attention recently and there is currently no support for them in eReader, zTXT, PalmDoc, or TXT inputs. They were never accounted for because up until last week I have never seen an ebook using them.
user_none is offline   Reply With Quote
Old 01-25-2011, 04:16 AM   #3
diamante
piedra preciosa
diamante has a spectacular aura aboutdiamante has a spectacular aura aboutdiamante has a spectacular aura aboutdiamante has a spectacular aura aboutdiamante has a spectacular aura aboutdiamante has a spectacular aura aboutdiamante has a spectacular aura aboutdiamante has a spectacular aura aboutdiamante has a spectacular aura aboutdiamante has a spectacular aura aboutdiamante has a spectacular aura about
 
Posts: 44
Karma: 4170
Join Date: Jan 2011
Location: Boston, Massachusetts
Device: Sony PRS-350, Kindle 2, several Palm OS devices
Hello! Thank you for the reply. I came across your blog in my search for an answer to this question, and I thought I would try this forum before e-mailing you directly. :-)

So, they are called soft scene breaks. I'd like to put in a respectful request for support for them in eReader input. I hope this only requires a simple tweak, but I'm prepared for less favorable news.

I've noticed these scene breaks for years in eReader e-books but just thought of them as three centered asterisks until I checked yesterday and found that they are handled differently in different eReader versions. (I also found it interesting that older versions of the eReader software show additional TOC elements or levels that aren't supported in the final version of eReader for Windows; these elements were also ignored in my Calibre conversions to EPUB and MOBI.)

In any case, I am very grateful for Calibre's conversion capabilities. I gave it a try on a whim and the results far exceeded my expectations, to put it mildly. Many thanks to you!
diamante is offline   Reply With Quote
Old 01-25-2011, 06:48 AM   #4
user_none
Sigil & calibre developer
user_none ought to be getting tired of karma fortunes by now.user_none ought to be getting tired of karma fortunes by now.user_none ought to be getting tired of karma fortunes by now.user_none ought to be getting tired of karma fortunes by now.user_none ought to be getting tired of karma fortunes by now.user_none ought to be getting tired of karma fortunes by now.user_none ought to be getting tired of karma fortunes by now.user_none ought to be getting tired of karma fortunes by now.user_none ought to be getting tired of karma fortunes by now.user_none ought to be getting tired of karma fortunes by now.user_none ought to be getting tired of karma fortunes by now.
 
user_none's Avatar
 
Posts: 2,436
Karma: 950001
Join Date: Jan 2009
Location: Florida, USA
Device: Nook STR
Quote:
Originally Posted by diamante View Post
I'd like to put in a respectful request for support for them in eReader input. I hope this only requires a simple tweak, but I'm prepared for less favorable news.
No it's not a simple tweak. It's some thing that is high on my todo list though. ldolse and I spoke about it not to long ago and it is something he would like to see implemented too. Once I finish with the few things I'm currently working on I plan to bring it up with him again.
user_none is offline   Reply With Quote
Old 01-25-2011, 08:51 AM   #5
dwig
Guru
dwig ought to be getting tired of karma fortunes by now.dwig ought to be getting tired of karma fortunes by now.dwig ought to be getting tired of karma fortunes by now.dwig ought to be getting tired of karma fortunes by now.dwig ought to be getting tired of karma fortunes by now.dwig ought to be getting tired of karma fortunes by now.dwig ought to be getting tired of karma fortunes by now.dwig ought to be getting tired of karma fortunes by now.dwig ought to be getting tired of karma fortunes by now.dwig ought to be getting tired of karma fortunes by now.dwig ought to be getting tired of karma fortunes by now.
 
dwig's Avatar
 
Posts: 969
Karma: 1382338
Join Date: Dec 2004
Location: Paradise (Key West, FL)
Device: Current:Dell Venue 8 Pro, Kindle 3/WiFi - Retired:Clie UX50, T415, ...
Quote:
Originally Posted by diamante View Post
...
So, they are called soft scene breaks. I'd like to put in a respectful request for support for them in eReader input. ...
I've noticed these scene breaks for years in eReader e-books but just thought of them as three centered asterisks until I checked yesterday and found that they are handled differently in different eReader versions. ...
I've found both what I think of as "soft" scene breaks (extra blank space) and "hard" scene breaks (extra space with an ornament, three asterisks, or horizontal rule) in books. Usually a book uses one style or the other, but I've seen the occasional book, both printed and ebook, that use both, one indicating a small jump and the other a larger one.

When massaging ebook files I habitually replace the old text convention (***) with a horizontal rule (25% width is my preference) and make sure that there is a non-breaking space in the "soft" scene breaks if they are done with simple paragraph tags (<p>, I replace any <p></p> pairs with <p>&nbsp</p>) instead of a CSS style. I haven't encountered it myself, but I've read that some ereader software ignore empty paragraphs and therefore don't display the extra blank space when a simple <p></p> pair is used.
dwig is offline   Reply With Quote
Old 01-25-2011, 09:10 AM   #6
ldolse
Wizard
ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.
 
Posts: 1,337
Karma: 123455
Join Date: Apr 2009
Location: Malaysia
Device: PRS-650, iPhone
Quote:
Originally Posted by dwig View Post
I've found both what I think of as "soft" scene breaks (extra blank space) and "hard" scene breaks (extra space with an ornament, three asterisks, or horizontal rule) in books. Usually a book uses one style or the other, but I've seen the occasional book, both printed and ebook, that use both, one indicating a small jump and the other a larger one.

When massaging ebook files I habitually replace the old text convention (***) with a horizontal rule (25% width is my preference) and make sure that there is a non-breaking space in the "soft" scene breaks if they are done with simple paragraph tags (<p>, I replace any <p></p> pairs with <p>&nbsp</p>) instead of a CSS style. I haven't encountered it myself, but I've read that some ereader software ignore empty paragraphs and therefore don't display the extra blank space when a simple <p></p> pair is used.
The format scene breaks option under heuristics is doing things along these lines - although scene break detection is currently only working in a couple special cases and really needs a some more work put into it. One of the things that I'm starting to dislike about 'soft' breaks is that they really don't work with ebooks. With a printed book the publisher will always make sure a soft break winds up in the middle of the page so it's obvious to the reader. With reflowable books more often than not a soft break will wind up on a page break, and then the user won't even realize it was supposed to be a softbreak. So the idea of the format scene breaks option just replaces all scene breaks with horizontal rules.
ldolse is offline   Reply With Quote
Old 01-26-2011, 03:25 AM   #7
diamante
piedra preciosa
diamante has a spectacular aura aboutdiamante has a spectacular aura aboutdiamante has a spectacular aura aboutdiamante has a spectacular aura aboutdiamante has a spectacular aura aboutdiamante has a spectacular aura aboutdiamante has a spectacular aura aboutdiamante has a spectacular aura aboutdiamante has a spectacular aura aboutdiamante has a spectacular aura aboutdiamante has a spectacular aura about
 
Posts: 44
Karma: 4170
Join Date: Jan 2011
Location: Boston, Massachusetts
Device: Sony PRS-350, Kindle 2, several Palm OS devices
user_none, I'm glad to hear that this is high on your to-do list!

Out of curiosity, I just used eReader eBook Studio to take a look at the PML generated during the conversion process for an e-book that I'm currently reading. The soft scene breaks are of course invisible on my Sony Reader and Kindle, but in eBook Studio they are plainly visible as two blank lines, i.e. three hard returns between paragraphs (there is only one hard return separating most other paragraphs). Also, the paragraph immediately preceding each soft scene break is centered for some reason, with this attribute extending through the first of the two blank lines of each soft scene break. You mentioned that soft scene breaks are currently not supported for eReader input; pardon my ignorance, but does this mean that the PML file is generated at a stage prior to eReader input? Also, if I add an eReader PDB to Calibre and convert it first to EPUB and then to MOBI and possibly other formats, does Calibre always go back to the PMLZ to do each conversion? By the way, can you recommend a better application than eBook Studio for editing PML files?

idolse, the earlier versions of eReader for Windows, as well as the Palm OS versions, displayed the soft scene breaks I describe above as three asterisks, probably because the extra blank space would often go unnoticed on small screens. The eReader developers apparently agreed with you on soft scene breaks for e-books and decided to "harden" them for the devices in use back then. On the Kindle and other newer devices, though, I think soft scene breaks can work as well as they do in printed books. In many printed books as well as e-books there are added cues to signal a soft scene break. Sometimes a paragraph immediately following a soft scene break will be unindented while most or all other paragraphs are indented. I just checked one Kindle e-book against a printed version and found that both use this method (extra blank space followed by a non-indented paragraph). I have also noticed in printed books that soft scene breaks that occur between pages are sometimes very easy to miss unless you're aware of the change in indentation, so it seems to me that the issue really isn't that different for e-books, except of course that e-books are reflowable and therefore a specific break may be more or less conspicuous depending on the device, font size settings, etc.
diamante is offline   Reply With Quote
Old 01-26-2011, 08:11 AM   #8
user_none
Sigil & calibre developer
user_none ought to be getting tired of karma fortunes by now.user_none ought to be getting tired of karma fortunes by now.user_none ought to be getting tired of karma fortunes by now.user_none ought to be getting tired of karma fortunes by now.user_none ought to be getting tired of karma fortunes by now.user_none ought to be getting tired of karma fortunes by now.user_none ought to be getting tired of karma fortunes by now.user_none ought to be getting tired of karma fortunes by now.user_none ought to be getting tired of karma fortunes by now.user_none ought to be getting tired of karma fortunes by now.user_none ought to be getting tired of karma fortunes by now.
 
user_none's Avatar
 
Posts: 2,436
Karma: 950001
Join Date: Jan 2009
Location: Florida, USA
Device: Nook STR
Quote:
Originally Posted by diamante View Post
Out of curiosity, I just used eReader eBook Studio to take a look at the PML generated during the conversion process for an e-book that I'm currently reading. ... in eBook Studio they are plainly visible as two blank lines, ... You mentioned that soft scene breaks are currently not supported for eReader input; pardon my ignorance, but does this mean that the PML file is generated at a stage prior to eReader input?
Conversion is a three stage process: Input -> OEB -> Output. The input format is read and converted to OEB (what is inside an EPUB file and consists of XHTML, CSS and some control files). Depending on a few factors (such as having heuristic processing enabled) the OEB is then further manipulated. The OEB is then given to the output generator and it transforms the OEB into the output format.

When I said that soft scene breaks are not supported in PDB (eReader) input I mean that literally. The PML is extracted from the PDB file and then the attributes are read and transformed into XHTML equivalents. This is harder than it sounds especially because PML is a pseudo fixed layout format. Soft scene breaks at this point are just ignored. They are not transformed into any XHTML or retained in any way.

Quote:
Originally Posted by diamante View Post
Also, if I add an eReader PDB to Calibre and convert it first to EPUB and then to MOBI and possibly other formats, does Calibre always go back to the PMLZ to do each conversion?
In the upper left of the conversion screen you can select which format you want to use for the source format.

Quote:
Originally Posted by diamante View Post
By the way, can you recommend a better application than eBook Studio for editing PML files?
Nope. Other than using a plain text editor, eBook Studio is the only dedicated PML editor I know of. PDB (eReader) and PML are a dying format and have been quickly supplanted by EPUB.

Quote:
Originally Posted by diamante View Post
... the earlier versions of eReader for Windows, as well as the Palm OS versions, displayed the soft scene breaks I describe above as three asterisks...
What ldolse and I are planning is to retain soft scene breaks as soft scene breaks but have a heuristic option (I think he's already added it) that will transform them into hard scene breaks.
user_none is offline   Reply With Quote
Old 01-26-2011, 01:36 PM   #9
KevinH
Guru
KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.
 
Posts: 865
Karma: 393922
Join Date: Nov 2009
Device: many
Hi,

Might I ask, exactly how is a "soft scene break" done in the pml? Is there a specific tag for it? Is it simply double-linebreaks? I have never seen a pdb book with such a beastie?
KevinH is offline   Reply With Quote
Old 01-26-2011, 04:44 PM   #10
user_none
Sigil & calibre developer
user_none ought to be getting tired of karma fortunes by now.user_none ought to be getting tired of karma fortunes by now.user_none ought to be getting tired of karma fortunes by now.user_none ought to be getting tired of karma fortunes by now.user_none ought to be getting tired of karma fortunes by now.user_none ought to be getting tired of karma fortunes by now.user_none ought to be getting tired of karma fortunes by now.user_none ought to be getting tired of karma fortunes by now.user_none ought to be getting tired of karma fortunes by now.user_none ought to be getting tired of karma fortunes by now.user_none ought to be getting tired of karma fortunes by now.
 
user_none's Avatar
 
Posts: 2,436
Karma: 950001
Join Date: Jan 2009
Location: Florida, USA
Device: Nook STR
Quote:
Originally Posted by KevinH View Post
Might I ask, exactly how is a "soft scene break" done in the pml? Is there a specific tag for it? Is it simply double-linebreaks? I have never seen a pdb book with such a beastie?
Multiple line breaks between paragraphs. I had not seen one (eReader) or otherwise that uses them until very recently. It seems to be a growing typesetting trend with some publishers.

I purchased Deadhouse Gates by Steven Erikson today from B&N. Turns out they're selling it as an eReader file and it's littered with soft scene breaks.

I've gone ahead and made changes to calibre's PML input to account for them. Basically if there is 3+ empty lines it puts an empty paragraph in the resultant HTML. I plan to do more work on it with ldolse in the near future to make it more robust. In the mean time the next calibre release will at least keep these books readable.
user_none is offline   Reply With Quote
Old 01-26-2011, 05:09 PM   #11
diamante
piedra preciosa
diamante has a spectacular aura aboutdiamante has a spectacular aura aboutdiamante has a spectacular aura aboutdiamante has a spectacular aura aboutdiamante has a spectacular aura aboutdiamante has a spectacular aura aboutdiamante has a spectacular aura aboutdiamante has a spectacular aura aboutdiamante has a spectacular aura aboutdiamante has a spectacular aura aboutdiamante has a spectacular aura about
 
Posts: 44
Karma: 4170
Join Date: Jan 2011
Location: Boston, Massachusetts
Device: Sony PRS-350, Kindle 2, several Palm OS devices
user_none, thanks very much for answering my questions so thoroughly.

KevinH, at least in the PMLs generated by Calibre from the PDBs that I have, as viewed in eBook Studio, it is indeed just the two linebreaks between paragraphs. I'm not sure if the centering of the paragraph preceding the two linebreaks has any relevance. Again, when I open the original PDBs in an older version of eReader, these breaks appear as three asterisks. (Haven't you noticed these scene breaks in PDBs? I'm pretty sure they're in the vast majority of the many PDBs I have.) In the final version of eReader, they appear simply as extra space between paragraphs.

I just saw user_none's latest post.

user_none, I read somewhere that if you download a Barnes & Noble e-book using a computer, it's in eReader format. If you use a Nook, it's an EPUB. I'm not sure if this is true, but I thought it was interesting. And wow, you're already making the necessary modifications to support these SSBs!!
diamante is offline   Reply With Quote
Old 02-01-2011, 04:43 AM   #12
diamante
piedra preciosa
diamante has a spectacular aura aboutdiamante has a spectacular aura aboutdiamante has a spectacular aura aboutdiamante has a spectacular aura aboutdiamante has a spectacular aura aboutdiamante has a spectacular aura aboutdiamante has a spectacular aura aboutdiamante has a spectacular aura aboutdiamante has a spectacular aura aboutdiamante has a spectacular aura aboutdiamante has a spectacular aura about
 
Posts: 44
Karma: 4170
Join Date: Jan 2011
Location: Boston, Massachusetts
Device: Sony PRS-350, Kindle 2, several Palm OS devices
Talking Thank you!

Quote:
Originally Posted by user_none View Post
In the mean time the next calibre release will at least keep these books readable.
Readable? As far as I'm concerned they are perfect now. THANK YOU! Not only the soft scene breaks but also the TOC issue, fixed!!

I re-converted the e-book I had just finished reading, as a test. The soft scene breaks showed up as three centered asterisks, and the TOC had all the layers that were in the original PDB. I'm new to the Sony Reader, so I had never even seen a layered TOC in anything but the old eReader before. Impressive.

Before realizing that I would need to go back to the original PDBs and not the PMLs, I converted another e-book and noticed that only one item from the whole eReader TOC showed up in the TOC on the Sony Reader. When I reconverted the PDB, though, everything was there, and so were the soft scene breaks. Very, very nice.
diamante is offline   Reply With Quote
Old 02-01-2011, 05:40 AM   #13
ldolse
Wizard
ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.ldolse is an accomplished Snipe hunter.
 
Posts: 1,337
Karma: 123455
Join Date: Apr 2009
Location: Malaysia
Device: PRS-650, iPhone
The next release of Calibre should be a bit more robust in this department - I believe user_none is extending the list of input formats which preserve soft scene breaks on input, and heuristics has had some improvements added to attempt to detect the difference between soft scene breaks and vertical whitespace. Things which are detected as actual scene breaks will get styled so that the ereader won't break the page on a soft scene break - one of my pet peeves that I see even with professionally published ebooks. Instead (assuming the reader supports css correctly), you'll always have a couple lines of text above the scene break.

Lastly there will be an option to convert soft scene breaks to 'hard breaks' (as named in this thread) with some ornamentation. Basically either convert 'soft' breaks to hard breaks, or convert vanilla '***' style breaks to something a bit fancier of your choosing.

Last edited by ldolse; 02-01-2011 at 05:58 AM.
ldolse is offline   Reply With Quote
Old 02-05-2011, 03:44 AM   #14
diamante
piedra preciosa
diamante has a spectacular aura aboutdiamante has a spectacular aura aboutdiamante has a spectacular aura aboutdiamante has a spectacular aura aboutdiamante has a spectacular aura aboutdiamante has a spectacular aura aboutdiamante has a spectacular aura aboutdiamante has a spectacular aura aboutdiamante has a spectacular aura aboutdiamante has a spectacular aura aboutdiamante has a spectacular aura about
 
Posts: 44
Karma: 4170
Join Date: Jan 2011
Location: Boston, Massachusetts
Device: Sony PRS-350, Kindle 2, several Palm OS devices
Idolse, all this sounds great! Thanks for the update. I see the new release came out today; I can't wait to try it.
diamante is offline   Reply With Quote
Old 02-07-2011, 03:35 AM   #15
ChristopherTD
Addict
ChristopherTD ought to be getting tired of karma fortunes by now.ChristopherTD ought to be getting tired of karma fortunes by now.ChristopherTD ought to be getting tired of karma fortunes by now.ChristopherTD ought to be getting tired of karma fortunes by now.ChristopherTD ought to be getting tired of karma fortunes by now.ChristopherTD ought to be getting tired of karma fortunes by now.ChristopherTD ought to be getting tired of karma fortunes by now.ChristopherTD ought to be getting tired of karma fortunes by now.ChristopherTD ought to be getting tired of karma fortunes by now.ChristopherTD ought to be getting tired of karma fortunes by now.ChristopherTD ought to be getting tired of karma fortunes by now.
 
ChristopherTD's Avatar
 
Posts: 310
Karma: 1003456
Join Date: Apr 2008
Location: London. UK
Device: Marvin Reader on iPad Air (via Calibre)
Generally my experience has been that soft scene breaks are lost and just the normal paragraph spacing appears. But it has been a long time since I converted my PDB books, so I might wait on 0.7.45 and try some of the new goodness!

Thanks for keeping working on this!
ChristopherTD is offline   Reply With Quote
Reply

Tags
ereader pdb conversion, horizontal rule, paragraph spacing, pdb to epub, pdb to mobi

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Converting eReader books mjmcleod Calibre 73 12-18-2012 07:41 AM
PDB file (eReader) - How to keep the formatting? Juliepac Other formats 0 11-26-2010 07:38 AM
eReader (pdb) purchased ebooks on B&N eReader on iPhone? bthoven Reading and Management 5 12-23-2009 06:52 AM
Losing formatting when converting PDB Edwin Fong Calibre 5 10-25-2009 07:20 PM
Converting books to eReader with Dropbook Robotech_Master Workshop 1 12-23-2008 12:46 PM


All times are GMT -4. The time now is 11:50 PM.


MobileRead.com is a privately owned, operated and funded community.