How I Manage eBooks with calibre

[theducks]

Originally Posted by LadyKate

Had to comment. I remember setting up a fidonet on my first ibm clone with dual floppies and no hard drive. (and the first hard drive I bought for more than I spent recently on my 4TB usb was only 10meg lol).

Sorry all, just feeling my age here

My first 'Timeshare account' was on 'The Source' (readers digest?) with a 300 Baud (direct connect ) Modem,
Later I went with Compuserve (remember CIM?)

[PeterT]

Originally Posted by theducks

My first 'Timeshare account' was on 'The Source' (readers digest?) with a 300 Baud (direct connect ) Modem,
Later I went with Compuserve (remember CIM?)

75716,2377 at your service.

[theducks]

Originally Posted by PeterT

75716,2377 at your service.

74007,1015 (been closed at least 10 years now)

[BetterRed]

I had a timeshare account with GE in the early 70's - wrote a yachting regatta scoring program in Dartmouth Basic when my club hosted the (I forget) class world championships - i/o device was a Decwriter on an 300 baud acoustic coupler.

[unboggling]

Irrelevant content deleted.

[Dngrsone]

I think what was even tougher was trying to produce a presentable report on a good old-fashioned manual typewriter... one mistake and you had retype the entire sheet again.

[unboggling]

Originally Posted by Dngrsone

I think what was even tougher was trying to produce a presentable report on a good old-fashioned manual typewriter... one mistake and you had retype the entire sheet again.

Probably the most brilliant invention in the 20th century:

http://en.wikipedia.org/wiki/Liquid_Paper

[PeterT]

Originally Posted by unboggling

Probably the most brilliant invention in the 20th century:

http://en.wikipedia.org/wiki/Liquid_Paper

And what is fascinating there is the linkage to The Monkees and also to the music video.

[LadyKate]

Originally Posted by unboggling

Ah, de good ol' days.

Yep, ya snivelin' youngers pro'bly warn't aborned yet, in de good ol' days, when ya had ta walk ta school or de coal mine barefoot in de snow, o'er hill an' dale, in de boilin hot sun, wit a t'ing called paper weighin ya down. Bind paper wit glue, an' ya got a book. Yep, books usta be made outta paper, I'm tellin ya, swear it be true. Yep, an' hoss hoof too. An' skin. Books twar hefty, good fer buildin muscles, ifn ya didn't have no mule ta haul 'em.
etc ....

I am probably one of the few who actually went to a one room school with no running water, a wood stove at the back (in front of the only door so a real fire hazard), his and her outhouses, a wood shed and a well and bucket (the well water was tainted so a bit of bleach was added to each bucket to purify it)

That was in the early 60s in Ontario Canada and only 30 miles from the nations capital lol.

Hard to believe someone who started out in a school like that would end up working in computers isn't it.

[LadyKate]

One way someone could edit a document in a word processor for publication as an ebook is to put a tag around the bold and italics manually, make sure they don't use hard breaks for anything other than paragraphs and when the file looks good save it to text (that's txt not rtf).

Next step, reload the document under a test name. Search for the markers for bold and italic. Change the bold and italic to bold... italic.

Now try saving the document as html.

This should remove a great deal of the excess formatting.

[unboggling]

Originally Posted by theducks

74007,1015 (been closed at least 10 years now)

Originally Posted by PeterT

And what is fascinating there is the linkage to The Monkees and also to the music video.

Different media for different generations. But transitions from one type of media to another — and one type of communication method to another — seem to be speeding up. (Or maybe it's me slowing down.) I've paper'd, landline-phoned, sliderule'd, calculator'd, typewriter'd, audio-system'd (45, 33 lp, reel-to-reel, cassette, cd, mp3), computer'd, video-system'd (cassette, dvd), BBposted, online-service'd, IRC'd, IM'd, cellphoned, smartphoned, e-inked, tablet'd, streamed. Rarely texted. Never tweeted. Never skyped or facetimed.

Originally Posted by LadyKate

I am probably one of the few who actually went to a one room school with no running water, a wood stove at the back (in front of the only door so a real fire hazard), his and her outhouses, a wood shed and a well and bucket (the well water was tainted so a bit of bleach was added to each bucket to purify it)....

His and Her outhouses? Luxury! We had to share one.

Originally Posted by LadyKate

One way someone could edit a document in a word processor for publication as an ebook is to put a tag around the bold and italics manually, make sure they don't use hard breaks for anything other than paragraphs and when the file looks good save it to text (that's txt not rtf).

Next step, reload the document under a test name. Search for the markers for bold and italic. Change the bold and italic to bold... italic.

Now try saving the document as html.

This should remove a great deal of the excess formatting.

Sounds similar to this:

http://daringfireball.net/projects/markdown/

(mentioned in Prefs > Input Options > TXT Input > Markdown)

[LadyKate]

Originally Posted by unboggling

His and Her outhouses? Luxury! We had to share one.

Sounds similar to this:

http://daringfireball.net/projects/markdown/

(mentioned in Prefs > Input Options > TXT Input > Markdown)

I did a quick look at the markdown but... does it maintain the italics and bold settings? Didn't notice that.

The freebie tool (not around that I could find but still works well) HTML Book Fixer, strips the excess spans BUT it also manages to remove the italics if they are in a span. Most irritating.

With excess nested spans it is darn near impossible to find the matching open / close tags that refer to italics using regex and a royal pain to "eyeball" the italics in the original.

I don't know why modern word processors don't allow the option to clean up the underlying code that is used to create pdf and html files. The main reason I find pdf files so hard to clean up is because most were created in a wysiwyg program. From the underlying code I get in the html it is usually word or a word clone that uses the horrid "<p class=MsoNormal><span style='mso-fareast-font-family:"MS Mincho"'>" often skipping the quotes around the class name. (note the font family/name is whatever font the doc used.)

I think all that excess code can lead to problems in conversions when nested too deep. I had one problem caused by not cleaning up a file because I had not noticed that one of the nested div tags was class="chapter" and around the entire chapter and another was class="chapterHead" and around the Chapter whatever.

[unboggling]

Originally Posted by LadyKate

I did a quick look at the markdown but... does it maintain the italics and bold settings? Didn't notice that.

I don't use it. I noticed in its documentation that it handles emphasis, but I don't know how that translates to bold or italic, if at all.

Originally Posted by LadyKate

The freebie tool (not around that I could find but still works well) HTML Book Fixer, strips the excess spans BUT it also manages to remove the italics if they are in a span. Most irritating.

With excess nested spans it is darn near impossible to find the matching open / close tags that refer to italics using regex and a royal pain to "eyeball" the italics in the original.

I don't know why modern word processors don't allow the option to clean up the underlying code that is used to create pdf and html files. The main reason I find pdf files so hard to clean up is because most were created in a wysiwyg program. From the underlying code I get in the html it is usually word or a word clone that uses the horrid "<p class=MsoNormal><span style='mso-fareast-font-family:"MS Mincho"'>" often skipping the quotes around the class name. (note the font family/name is whatever font the doc used.)

I think all that excess code can lead to problems in conversions when nested too deep. I had one problem caused by not cleaning up a file because I had not noticed that one of the nested div tags was class="chapter" and around the entire chapter and another was class="chapterHead" and around the Chapter whatever.

Okay, I think I've been misunderstanding what you're trying to do. Do I have the following sequence straight?

1. You are starting with a PDF. (That is a horrible format to start with.)
2. Then you convert it to what? HTML? EPUB? RTF? I'm confused about that.
3. Then you want to clean up the underlying code. Why? Is it just to eliminate formatting problems that interfere with readability? Or are you producing books for other people that must have clean code?
4. At the end of the sequence, what format should it be, EPUB?

This is what I do, using the "ignore underlying code" approach. If bold or italic formatting are there to begin with in the original format, that formatting is preserved all the way through to the end (except in cases where I fix inappropriate use of bold or italic).

1. I buy a book. Say AZW3. In this case it's file size is 405 KB. Add it to calibre, download metadata.
2. Convert to EPUB. This EPUB is 315 KB. Assess it in calibre Viewer. It has some bold and italic formatting. If annoying formatting problems don't exist enter format quality rating, else:
3. Convert to RTF. Fix formatting problems in Word, without worrying about underlying code. Deal with Word's presentation of markup (if necessary replace interfering linefeeds with pilcrows, replace awkward page breaks with nothing, etc.) Fix other annoying problems. The appropriate bold and italic formatting is still there. Save as DOCX.
4. Convert to EPUB. This EPUB is 310 KB. Assess in Viewer, enter format quality rating. The bold and italic are still there. Then just read the book, unbothered by any extra unnecessary code in the format because it is invisible and irrelevant while reading.

Now, instead of starting with AZW3, let's say I start with PDF. I would do the same sequence: convert to EPUB and assess it, convert to RTF, fix in Word (get rid of headers/footers, deal with Word's presentation of markup, fix other annoying problems) and save as DOCX, convert to EPUB, assess it in Viewer. Ignore the underlying code. Read the book.

Admittedly this works best for simply formatted text-based books, starting from EPUB, AZW3, or MOBI. PDF conversions usually have more problems so for me they're more trouble than they're worth.

[LadyKate]

Originally Posted by unboggling

I don't use it. I noticed in its documentation that it handles emphasis, but I don't know how that translates to bold or italic, if at all.

Okay, I think I've been misunderstanding what you're trying to do. Do I have the following sequence straight?

1. You are starting with a PDF. (That is a horrible format to start with.)
2. Then you convert it to what? HTML? EPUB? RTF? I'm confused about that.
3. Then you want to clean up the underlying code. Why? Is it just to eliminate formatting problems that interfere with readability? Or are you producing books for other people that must have clean code?
4. At the end of the sequence, what format should it be, EPUB?

Ok, I tend to look at things as starting to cleanup with HTML.

HTML can be obtained by opening an ePub, or a Mobi file from Calibre. Saving an rtf, doc or docx file as html in some kind of editor that handles it.

Converting a pdf file to HTM or HTML using Acrobat Pro (I only have version 7 lol. don't use it enough to buy a newer version), a word processor that can translate to HTML or mobipocket creator which as part of the process of translating the prc generates an html file.

In other words. Using any method I can find I translate my original document to HTML. Perhaps even taking an old text file and going through and adding tags to it. (I can't find the php files I had that used a bunch of rules for creating paragraphs out of a flat txt file. It took me quite a while to write it and figure out the regex for finding all the characters found in a paragraph)

Sometimes, if it is horrid with nested spans and garbage I will strip all coding from it and using the original file do a search for italics and bold or strong. Using two editors I will search in the original file for the tag. Copy enough of the text in or around the tag to find the text in the "clean" copy and put in clean tags.

ARgh I must apologize. This has become much too rambling and probably has bored everyone.

[unboggling]

Originally Posted by LadyKate

Ok, I tend to look at things as starting to cleanup with HTML.

HTML can be obtained by opening an ePub, or a Mobi file from Calibre. Saving an rtf, doc or docx file as html in some kind of editor that handles it.

Converting a pdf file to HTM or HTML using Acrobat Pro (I only have version 7 lol. don't use it enough to buy a newer version), a word processor that can translate to HTML or mobipocket creator which as part of the process of translating the prc generates an html file.

In other words. Using any method I can find I translate my original document to HTML. Perhaps even taking an old text file and going through and adding tags to it. (I can't find the php files I had that used a bunch of rules for creating paragraphs out of a flat txt file. It took me quite a while to write it and figure out the regex for finding all the characters found in a paragraph) ...

You want to clean the HTML, or generate clean HTML. This thread isn't the best place to discuss that. Like I said, I generally ignore the code level (HTML/XHTML, CSS). My knowledge/skills there are at the low end of the learning curve. I fix formatting problems interfering with readability if they are quickly fixable, but for my purpose, reading books for enjoyment, I don't care if the underlying code is clean or not.

btw, take a look at Toxaris' Word macro for clean HTML code:

https://www.mobileread.com/forums/showthread.php?t=142530

(for Word on Windows or OS X)