Breaking down a massive file

[lostinlodos]

Hi. I'm not sure where to post this so feel free to move me!
I created a document in Microsoft word that I then converted to ePub in Calibre. It's 56XX pages long and image heavy.
I now have two problems. First I accidentally deleted (shredded) the folder I was using for various page things including my source docx.
Second my file is too big for most readers, at 160MB.
iBooks crashes and Totalreader stalls for a long time (almost half an hour) on each load.
It displays fine on most of my PC apps so it's definitely a size or length problem.
I've tried converting the ePub back to docx and Microsoft Word can't load it, invalid file. Fact, Atlantis is the only WP that can load Calibre's docx. Resaving from AT doesn't help, still can't load in word.

SO: question is, is there an easy, direct, automated way to split my ePub into, say, 500 page chunks? (Not worried about where it splits at the moment I can fix sentence cutting and stuff manually). Like drag n drop type 500 click and walk away?
Or is there another text based format I could use to do this. PDF works and gives me a "valid" output but the formatting is messed up and I'm simply attempting to not have to rearrange and reformat more than 5000 pages.
Please, any help would be wonderful!

[fbrzvnrnd]

Originally Posted by lostinlodos

SO: question is, is there an easy, direct, automated way to split my ePub into, say, 500 page chunks? (Not worried about where it splits at the moment I can fix sentence cutting and stuff manually). Like drag n drop type 500 click and walk away?

You can use Sigil and the string Sigil uses to break chapters. For example, you can search for title tags (h1 or h2, you have to see the code) and add the string Sigil uses to break chapters. After you added the string, you can ask Sigil to split file in that places.

[BetterRed]

There's a plugin for Calibre that will split an epub into multiple epubs, you could give it a try ===>>> EpubSplit

BR

[Turtle91]

160 MB is immense when it comes to an ebook. Even very large - picture intense - children's books that I've seen are around 20-25MB. I would guess that the size of your image files are the main culprit. You can/should reduce the image size to about 1600px in height and/or use a better image format to compact the size. That is really all the image size/quality you need with today's reading apps/devices. If you absolutely must have the full size images then you could host them on a network server and include links in the ebook.

As far as splitting the ePub into smaller files....yes...definitely! The smaller the file size the faster the device will load and display each new file. Most people will split the html file at a new chapter - 20 Chapters = 20 files plus files for cover, title, acknowledgments, copyright, etc.

You can use a simple search and replace to find new chapter headings...or if you have the html tags set to use the <h1>-<h6> for different headings you can split there.

If you are using Sigil - and I'm pretty sure Calibre has the same or similar functionality - you can use the following search and replace:

search: "<h1" or "<h3>Chapter" etc. (without the quotes)
replace: "<hr class="sigil_split_marker" /><h1" or "<hr class="sigil_split_marker" /><h3>Chapter" etc.

Then you can use the function "Split at Markers (F6)" in Sigil to automatically split at all those marked locations.

Conversely...you can use the "Split at Cursor (Ctrl-return)" to manually split up the document.


Hope that helps!

[lostinlodos]

Edited, this isn't going to work: see photo.

Thanks both of you. It's not exactly what I had in mind but looks like it would work. Since there are boldened chapters which should have converted to headings. I hope.
Going to try ePub split shortly. Given I can't get Atlantis to do what I want here I'm running out of programming options before I attempt reworking via PDF.

Yes it's massive, though not the largest I've seen, but it's not intended for general distribution. Electronically it's only going to a few people. It's not up set to be consumable at the moment.
Well, at the moment it's not set up to be anything since nothing portable loads it. Lol.
I know the length is what's tossing totalreader since it can handle files many times this size, so it needs to be split.
Nothing out there like HTML join and split, for ePubs?

[BetterRed]

I've made occasional use of pdfSAM (Split And Merge) - the Lite version is free.

You cannot split HTML randomly and expect to do anything sensible with it - it needs to split on a boundary of some sort. Try using calibre's recently added calibre-debug --explode-book and --implode-book options, see ==>> calibre-debug.

That should allow you to use standard html and other text editing tools.

BR

[fbrzvnrnd]

Originally Posted by lostinlodos

Edited, this isn't going to work: see photo.

Try the Sigil way I suggested to you.

[JSWolf]

Use calibre to edit the eBook. Given that it came from Word, chances are there is a lot of garbage in the ePub that you can remove. Also check the size of the images. If they are beyond 1500 lines, reduce them in size to 1500 lines. Calibre can do this in the editor. If there are any embedded fonts, remove them. Make this as streamlined as possible. Also, check the code using epubcheck. There is a Calibre editor plugin to allow you to use epubcheck to check the eBook. Also, check the CSS and remove any excess lines from the classes. And if you see things like <p class="someclass">, replace it with <p> and and define <p> as what <p> is going to be most of the time.

[lostinlodos]

Thanks everyone. I've got more options now. I think I'll give sigil a try . For whatever reason I never got the hang of it, so maybe now is a good time to learn.
I've used HTML split and join since the 90s and never had a problem with it but never used it on anything fancier than standard text and graphics pages. So not really sure what would happen if I extracted the ePub and ran it through SaJ. I should probably try that too.
I'm trying not to mess with the images at all because they're maps.
Sigil here I come.

[salamanderjuice]

Originally Posted by Turtle91

160 MB is immense when it comes to an ebook. Even very large - picture intense - children's books that I've seen are around 20-25MB. I would guess that the size of your image files are the main culprit. You can/should reduce the image size to about 1600px in height and/or use a better image format to compact the size. That is really all the image size/quality you need with today's reading apps/devices. If you absolutely must have the full size images then you could host them on a network server and include links in the ebook.

As far as splitting the ePub into smaller files....yes...definitely! The smaller the file size the faster the device will load and display each new file. Most people will split the html file at a new chapter - 20 Chapters = 20 files plus files for cover, title, acknowledgments, copyright, etc.

You can use a simple search and replace to find new chapter headings...or if you have the html tags set to use the <h1>-<h6> for different headings you can split there.

If you are using Sigil - and I'm pretty sure Calibre has the same or similar functionality - you can use the following search and replace:

search: "<h1" or "<h3>Chapter" etc. (without the quotes)
replace: "<hr class="sigil_split_marker" /><h1" or "<hr class="sigil_split_marker" /><h3>Chapter" etc.

Then you can use the function "Split at Markers (F6)" in Sigil to automatically split at all those marked locations.

Conversely...you can use the "Split at Cursor (Ctrl-return)" to manually split up the document.


Hope that helps!

I don't think the total file size is the issue. I have graphic novels over 300MBs that my Kobo and it's various apps happily handle. Overly complex HTML/CSS is a more likely culprit especially at 5600+ pages and converted from a word document.

Unless the images are absolutely massive but with a 5600+ page document there could just be a few hundred 0.5MB images which would be pretty typical e-Reader sized images.

[lostinlodos]

Originally Posted by salamanderjuice

I don't think the total file size is the issue. I have graphic novels over 300MBs that my Kobo and it's various apps happily handle. Overly complex HTML/CSS is a more likely culprit especially at 5600+ pages and converted from a word document.

Unless the images are absolutely massive but with a 5600+ page document there could just be a few hundred 0.5MB images which would be pretty typical e-Reader sized images.

I figured the size isn't the issue for totalreader since I've opened books orders of magnitude larger. Such as some server manuals that were in the near gigabyte range. It's that total reader does a layout scan on open, and at 5600+ pages, that's what takes it forever.
Hence my attempting to spit this in the first place. I'm going to try SG as mentioned above shortly. And see if I can find a way using that.

[Turtle91]

Sigil has a merge and split function.


Once you get it to open in Sigil:

- Use the search and replace function mentioned above to insert the <hr class="sigil_split_marker" /> markers at your desired locations
- highlight all the files in the Book Browser, right click and select merge... you will have one very large file
- Select F6 (or Edit/Split at Markers) and Sigil will re-split the document at your marked locations


Although I would certainly listen to salamandarjuice's advice and check that your css/html is not overly complex....I've seen some very heinous code-bloat created by software programs that you would think are better than that.

Eliminating complex html/css can drastically reduce the number of "pages" - but I would still look at your images. Even maps can be losslessly reduced significantly. I get very good results with cover images that are 1600 x 600-ish pixels at around 200KB or less.

Cheers,