Newbie text to epub conversion

[michaelbr]

I'm new to calibre and conversion, I'm trying to gather some pages/information from website and put them together into an "epub" ebook. I've tried to use the default setting in calibre to convert, unfortunately it didn't work. Tried to search on internet tutorials/youtube explaining it, and found very few tutorials available for this topic. Can someone please give me an hint where to start to learn this topic?

The trouble I'm having in converting is this,
original text:
Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Mauris gravida urna ac vulputate efficitur. Duis ultrices

nisl id tempor ultricies. Ut feugiat metus a ornare aliquam.

Integer vel accumsan elit, in facilisis libero.


Vestibulum mollis justo ut dictum tempor. Maecenas euismod

dui sed feugiat auctor. Aenean at accumsan mauris.

after conversion:
<p class="calibre2">Lorem ipsum dolor sit amet, consectetur adipiscing elit. </p>

<p class="calibre2">Mauris gravida urna ac vulputate efficitur. Duis ultrices </p>

<p class="calibre2">nisl id tempor ultricies. Ut feugiat metus a ornare aliquam. </p>

<p class="calibre2">Integer vel accumsan elit, in facilisis libero.</p>


<p class="calibre2">Vestibulum mollis justo ut dictum tempor. Maecenas euismod </p>

<p class="calibre2">dui sed feugiat auctor. Aenean at accumsan mauris. </p>


It seems for some reason, the webpage generated linebreak after each line instead of end of paragraph. It seems not too complicate to solve but I have no clue how.

[itimpi]

You probably want to enable the heuristics processing option for conversions that have that problem.

[Doitsu]

Originally Posted by michaelbr

I'm new to calibre and conversion, I'm trying to gather some pages/information from website and put them together into an "epub" ebook.

You also might want to check out dotepub.

[michaelbr]

Originally Posted by Doitsu

You also might want to check out dotepub.

Thanks for this tip, I'm not sure if dotpub will solve my problem, since nowadays a lot of sites has embeded frames with ads, so I assume all those frames/ads will be included/converted too?

[michaelbr]

Originally Posted by itimpi

You probably want to enable the heuristics processing option for conversions that have that problem.

Thanks for the tips, is there anywhere a tutorial which explains tweaks for conversion? I've found only one on youtube, and it's a very short one, using one regex to solve a specific problem.

[retiredbiker]

When you go to convert a text file, a "TXT Input" settings panel becomes available in the left side of the convert screen.

If your input text has indentation at each paragraph, or if there is a blank line between paragraphs, you can use the "Paragraph style" dropdown. "Block" will use blank lines to determine actual paragraphs, and "Print" will use indentations. (Look at the tool tips.)

If your input text is just a long list of short lines, with no indication of where actual paragraphs should be, then the heuristic processing is your next best bet. It will use the length of short lines to try and guess the paragraph boundaries. It will maybe give pretty good results, but will not be perfect.

How well any of this works depends on the consistency of the input text. If you have inconsistent indentations or blank lines, or if there are, by chance, many paragraphs that end in long lines rather than shorter ones, you should edit the input text first, to get good results.

If you are doing a copy/paste to gather text from a web page, your best bet is to paste it into Word or Writer first, fix it up there, and then convert the word processor doc to epub.

[michaelbr]

Originally Posted by itimpi

You probably want to enable the heuristics processing option for conversions that have that problem.

Thanks itimpi for this tip, unfortunately it did not work, maybe the linebreak is somehow hardcoded into the page when the webpage was generated.

[michaelbr]

Originally Posted by retiredbiker

When you go to convert a text file, a "TXT Input" settings panel becomes available in the left side of the convert screen.

If your input text has indentation at each paragraph, or if there is a blank line between paragraphs, you can use the "Paragraph style" dropdown. "Block" will use blank lines to determine actual paragraphs, and "Print" will use indentations. (Look at the tool tips.)

Thanks so much for your detailed explanation, it's much appreciated, I'll give it a try.

[deback]

Originally Posted by michaelbr

Thanks itimpi for this tip, unfortunately it did not work, maybe the linebreak is somehow hardcoded into the page when the webpage was generated.

Yes, conversion will create a separate line for each line or paragraph in the text file that has a CR at the end (carriage return, same as hitting the Enter key at the end of a sentence or line or paragraph). If you remove those CRs in your text file and prepare the text without the CRs (when they are in the wrong places), the Calibre conversion will turn out the way you want it.

[michaelbr]

Originally Posted by deback

Yes, conversion will create a separate line for each line or paragraph in the text file that has a CR at the end (carriage return, same as hitting the Enter key at the end of a sentence or line or paragraph). If you remove those CRs in your text file and prepare the text without the CRs (when they are in the wrong places), the Calibre conversion will turn out the way you want it.

Thanks deback, but the trouble is to remove the CR, I don't think it can be removed automatically (there're CR at the end of each line and at the end of paragraph), is there any other way than to remove manually?

[Doitsu]

@michaelbr you really might want to give dotepub a try. It's much easier to remove a couple of ads than having to deal with each single line.
If you're a Fireforx user, also try to enable the Reader View option, which'll remove a lot of clutter. (It's also available as a Chrome extension.)

[deback]

Originally Posted by michaelbr

Thanks deback, but the trouble is to remove the CR, I don't think it can be removed automatically (there're CR at the end of each line and at the end of paragraph), is there any other way than to remove manually?

Yes, remove manually in a text editor, but I would try this first: Find \n\n and replace it with \n. Hopefully, the line space that should be there after a paragraph would be at \n\n\n.

[retiredbiker]

Originally Posted by deback

Yes, remove manually in a text editor, but I would try this first: Find \n\n and replace it with \n. Hopefully, the line space that should be there after a paragraph would be at \n\n\n.

Doing this in a text editor just replacing \n\n with \n would remove the blank lines, making it worse. If I have a text file with "blank" lines, what I do is basically:

Replace \n\n with some unused symbol, like #
Replace \n with a space
Replace the # with \n
Replace space space with space a few times to get rid of multiple spaces.

It's almost always messier than just that, but that is the basic process.

[theducks]

I have 5 'Join type' REGEX S&R that I use.
Only 2 do I ever run in ALL mode. the other 3, I step thru and do a Find (aka skip) if the visual shows it should not be applied.
It takes me 5-10 minutes to do the whole book. There are ALWAYS a couple of cases the the keyboard Del or Enter is used.
Take a few and learn REGEX (there are a couple of tutorials here at MR). It will pay off and make fixing so much easier that doing multiple Conversions while trying to fine tune the Heuristics settings

[michaelbr]

Originally Posted by theducks

I have 5 'Join type' REGEX S&R that I use.
Only 2 do I ever run in ALL mode. the other 3, I step thru and do a Find (aka skip) if the visual shows it should not be applied.
It takes me 5-10 minutes to do the whole book. There are ALWAYS a couple of cases the the keyboard Del or Enter is used.
Take a few and learn REGEX (there are a couple of tutorials here at MR). It will pay off and make fixing so much easier that doing multiple Conversions while trying to fine tune the Heuristics settings

Thanks theducks for this tip, I start to have the same feeling, there are too many places for one to tweak to accomplish the desired results. If I learn REGEX, it'll solve my problems not on this topic, also in other places/apps where REGEX is required.