Is there Epub cleaning software (so much unneeded code inside)

Nick_1964 · 08-17-2015, 08:22 AM

Dear friends.
When I opened a commercial ebook that I bought (social DRM) I noticed the book is really slow.
So I looked to the code and that was terrible.. no way I can clean this by hand... is there software (I hope free because it is for only one book,until now) or an online tool that can clean it for me ??
Look at the spoiler for a example.
In real..this are only a couple of lines from the book...

Spoiler:

'Ik weet het,' zei Ro zacht. 'Ik heb alles gezien. Ik zat onder de tafel verstopt.

Elena keek hem verbijsterd aan.

'Mijn moeder zei dat ze mij een afkoelperiode wilde geven en dat ik dan wel weer redelijk zou worden. Ze stopte mij in een kamer, maar ik kon ontsnappen. Joxy, mijn privé- bediende heb ik kunnen overhalen en zij verstopte mij onder de trolley die ze naar de grote zaal bracht. Toen ik zag dat daar van alles stond te gebeuren besloot ik mij onder tafel te verbergen. Ik heb alles gezien en nu kom ik je halen. We moeten weg hier. We moeten William en Charlotte bevrijden. Dat… wezen is mijn moeder niet meer!'

Elena knikte en ze schreef opnieuw iets op het papier. 'Wat wil je doen? Samis heeft nu de twee boeken. Alles en iedereen is hier aangepast, gekruist of gemaakt. Greysdale is bewoond door zombies, Utopalta is half vergaan en Willowjinx is weg. Waar moeten we naartoe? Ik kan niets zo!' Elena wees naar haar keel.

'Shit- ja!' zei Ro net iets te hard. Meteen sloeg Elena een hand voor zijn mond.

Ro haalde haar hand weg. 'Sorry,' fluisterde hij. 'Ik weet het niet, maar we moeten iets doen.'

Met tegenzin schudde Elena haar hoofd. Ze pakte het papiertje en schreef verder. 'We hebben alleen het huis in het bos over. Misschien dat de professor ons kan helpen. Aan de Codex Trias hebben we niets. Die is veilig bij Bia en Luxis. Hier kunnen we niet blijven. We moeten William en Charlotte bevrijden en gaan dan naar het bos.'

'Goed plan,' fluisterde Ro. 

Elena pakte het papier en gooide het in de open haard. Ro nam haar bij de hand en trok haar door de deur waar hij vandaan was gekomen. Ze kwam in een gelijksoortige kamer als die van haar alleen was het hier een stuk rommeliger.

Ripplinger · 08-17-2015, 08:33 AM

This looks to be a typical paragraph from your selection, and if it's the same throughout the book, I think you could clean it by hand pretty easily using Sigil's Find and Replace:

Quote:

Ro haalde haar hand weg. 'Sorry,' fluisterde hij. 'Ik weet het niet, maar we moeten iets doen.'

I'd search for and replace all instances of it with (whatever you decide to use and set up in the stylesheet).

Then search for and in the replace box just leave it blank so it will delete all instance of it. Note the number of instances it finds here for the next step.

Then search for and do the same thing, leave the replace box blank to they'll all be deleted. Check that the number of instances found matches with the previous step, or there might be other span classes somewhere, some of which you might want to keep. You can find those after this step is done by just searching for "span" and see what comes up.

If there are a lot of other instances of "dict-###" with various numbers, you could use regex the same way and get them all.

Then just let Sigil clean up the stylesheet to delete any unused styles and try it.

I've never found any automated software that can make the proper decisions about what stays and what should go.

Nick_1964 · 08-17-2015, 08:47 AM

Quote:

Originally Posted by Ripplinger

If there are a lot of other instances of "dict-###" with various numbers, you could use regex the same way and get them all.

Then just let Sigil clean up the stylesheet to delete any unused styles and try it.

I've never found any automated software that can make the proper decisions about what stays and what should go.

I tried with Sigil which I use the most but the changing "dicts" are everywhere and by removing a lot of span's I always get the warning that the html is not well formed and need to be cleaned.

Due to my dyslection I really don't understand the regex way of searching.
In the very beginning Sigil was working with wildcards like dict* but that isn't the case anymore..
Also it doesn't use any .css all the rules are in the html:

Spoiler:

This is only a small part of one html... and they use the margins-right to position the words (or it seems that way) maybe it is better to put it all in Calibre and convert it to txt and then build it again proper with Sigil if that is possible.
I really don't know what the publisher tries to do with it...

Yes.. I really tried to understand the regex method.. but it just doesn't stay in the part I reserved for it in my brains

And Imagine what happens with all the classes when I put it out with the KoboTouchExtended driver to a kepub ..

MikeB1972 · 08-17-2015, 08:58 AM

In Sigil use the Regex mode for find/replace
In the find box
(.*?)
in the replace box
\1
Replace all

Nick_1964 · 08-17-2015, 09:08 AM

Quote:

Originally Posted by MikeB1972

In Sigil use the Regex mode for find/replace
In the find box
(.*?)
in the replace box
\1
Replace all

I gonna try,thank you, but first I gonna try again to read what you wrote to understand what it does.. for sure it will help, but I want to know why to understand that Regex mode.. just for the look it replaces the classes, but what does the \1 means..
The dlct-007 is just one.. they start with 1 and end with.. I even don't know where..and everyone is different and used.. some containing larger txt for chapters so the lay-out would be gone anyway... I rather scan a book with abbyy then this mess..

MikeB1972 · 08-17-2015, 09:19 AM

Quote:

Originally Posted by Nick_1964

I gonna try,thank you, but first I gonna try again to read what you wrote to understand what it does.. for sure it will help, but I want to know why to understand that Regex mode.. just for the look it replaces the classes, but what does the \1 means..
The dlct-007 is just one.. they start with 1 and end with.. I even don't know where..and everyone is different and used.. some containing larger txt for chapters so the lay-out would be gone anyway... I rather scan a book with abbyy then this mess..

To break it down
Search for this 
then any combination of items (.*?)
ending with this 

the brackets around .*? say "cut the text for later use"

to use in the replace start with \ then the instance of saved text so as it's the first (and only) instance in your case use this for replace
\1

from the look of the sample you posted I would just try replacing 007 for now as it seems to be the default paragraph and won't affect the layout.

Nick_1964 · 08-17-2015, 09:40 AM

Quote:

Originally Posted by MikeB1972

To break it down
Search for this 
then any combination of items (.*?)
ending with this 

the brackets around .*? say "cut the text for later use"

to use in the replace start with \ then the instance of saved text so as it's the first (and only) instance in your case use this for replace
\1

from the look of the sample you posted I would just try replacing 007 for now as it seems to be the default paragraph and won't affect the layout.

Gonna study this.. but the dclt-007 is only in the lines I copy'd, 2 paragraps below it is changing in dclt-008... tried to open the next html, Sigil is crashing.. there is just to much..

HarryT · 08-17-2015, 10:00 AM

Moved to the "Workshop" forum, where such questions belong.

rubeus · 08-17-2015, 10:07 AM

Delete in Sigil and let do tidy do the rest.

Nick_1964 · 08-17-2015, 10:08 AM

Quote:

Originally Posted by HarryT

Moved to the "Workshop" forum, where such questions belong.

Excuse me .. didn't finded that one..

Quote:

Originally Posted by rubeus

Delete in Sigil and let do tidy do the rest.

67712 replacements done.. but then html error expected end of tag 'p'. automatix fix doesn't work and there are a whole lots of there.. I guess about 67712

But I do try to figure out how the regex is working so such examples are gold for me.

Toxaris · 08-17-2015, 10:43 AM

If you want Nick, I can clean it up for you. Just give me a sign.

dickloraine · 08-17-2015, 10:54 AM

Is it a novel? If so, maybe use calibre to conver it to for example txt with markdown and then back to epub. Or convert it to docx and use toxaris word add-in. Good luck, it really looks terrible.

Nick_1964 · 08-17-2015, 10:57 AM

Quote:

Originally Posted by Toxaris

If you want Nick, I can clean it up for you. Just give me a sign.

Hou ik als backup

I keep that as a backup,thank you very much,because I want to learn... And the books are not even mine (in the way I paid for it but they are for the girl next door..she has one of my readers in use and brings me her pocketmoney and a list with books she want.. )

But I am a bit further now.. just to find out that between every line, almost between every line there is a blank one.
When I remove it by adding code to a .css I added..they are gone but also the paragraphs that does belong there.. but even visually I can't see what is a blanc line by a wrong html code and whats a paragraph, they almost all start with and at places where i expect a paragraph they are to... The files are not splitted in sections where a new chapter begins (I always do that...) but the chapters are divided by a bunch of enters.. ( ) man oh man..

And more worse.. al the hyphen - are there (i suppose they are there in the real paper book) just in the middle of lines.. but also some text is marked just by - txt here - so i can't just remove all the hyphen - marks.. and guess what.. she also asked me to buy the 2 other parts and they are made exactly the same..

Quote:

Originally Posted by dickloraine

Is it a novel? If so, maybe use calibre to conver it to for example txt with markdown and then back to epub. Or convert it to docx and use toxaris word add-in. Good luck, it really looks terrible.

That was what I was thinking before and I may end up doing this but now so many people add suggestions I first want to try it with Sigil and after that I try with Calibre and then compare them..

They are 3 child books, for as far as I can see it is a story about a couple of kids that are bookkeepers (dunno.. seems a bit odd compared to the mess the books are) and have to fight dragons and other things in a underworld, don't know why and for what goal..and I don't want to know it to..

rubeus · 08-17-2015, 01:41 PM

Quote:

Originally Posted by Nick_1964

automatix fix doesn't work

Thats not what i suggested.

Nick_1964 · 08-17-2015, 01:45 PM

Quote:

Originally Posted by rubeus

Thats not what i suggested.

Nope.. but tidy isn't an option anymore in the new Sigil, it now works as the automatic repair function.

08-17-2015, 08:22 AM	#1
Nick_1964 Bookworm Posts: 975 Karma: 768585 Join Date: Aug 2010 Location: Netherlands Device: Sony prs-650, Kobo Glo HD (2x), Kobo Glo	Is there Epub cleaning software (so much unneeded code inside) Dear friends. When I opened a commercial ebook that I bought (social DRM) I noticed the book is really slow. So I looked to the code and that was terrible.. no way I can clean this by hand... is there software (I hope free because it is for only one book,until now) or an online tool that can clean it for me ?? Look at the spoiler for a example. In real..this are only a couple of lines from the book... Spoiler: <p class="dlct-000"><span class="dlct-007">'Ik weet het,</span><span class="dlct-007">'</span> <span class="dlct-007">zei Ro zacht. 'Ik heb alles gezien</span><span class="dlct-007">. I</span><span class="dlct-007">k</span> <span class="dlct-007">zat onder de tafel verstopt</span><span class="dlct-007">.</span></p> <p class="dlct-000"><span class="dlct-007">Elena keek hem</span> <span class="dlct-007">verbijsterd</span> <span class="dlct-007">aan.</span></p> <p class="dlct-000"><span class="dlct-007">'Mijn moeder zei dat ze mij een afkoelperiode wilde geven en dat ik dan wel weer redelijk zou worden. Ze stopte mij in een kamer</span><span class="dlct-007">,</span> <span class="dlct-007">maar ik kon ontsnappen</span><span class="dlct-007">.</span> <span class="dlct-007">Joxy</span><span class="dlct-007">,</span> <span class="dlct-007">mijn privé</span><span class="dlct-007">-</span> <span class="dlct-007">bediende heb ik kunnen overhalen en z</span><span class="dlct-007">ij</span> <span class="dlct-007">verstopte mij onder de trolley die ze naar de grote zaal bracht. Toen ik zag dat daar van alles stond te gebeuren besloot ik mij</span> <span class="dlct-007">onder tafel</span> <span class="dlct-007">te ver</span><span class="dlct-007">bergen</span><span class="dlct-007">. Ik heb alles gezien en nu kom ik je halen. We moeten weg hier. We moeten William en Charlotte bevrijden. Dat</span><span class="dlct-007">… wezen</span> <span class="dlct-007">is mijn moeder niet meer!'</span></p> <p class="dlct-000"><span class="dlct-007">Elena knikte en ze schreef</span> <span class="dlct-007">opnieuw iets op het</span> <span class="dlct-007">papier.</span> <span class="dlct-007">'Wat wil je doen? Samis heeft nu de twee boeken. Alles en iedereen is hier aangepast, gekruist of gemaakt</span><span class="dlct-007">.</span> <span class="dlct-007">Greysdale is bewoond door zombies, Utopalta is half vergaan en Willowjinx is weg. Waar moeten we naartoe? Ik kan niets zo!' Elena</span> <span class="dlct-007">wees naar haar</span> <span class="dlct-007">keel.</span></p> <p class="dlct-000"><span class="dlct-007">'Shit- ja!'</span> <span class="dlct-007">zei Ro net iets te hard</span><span class="dlct-007">. M</span><span class="dlct-007">eteen sloeg Elena een hand voor zijn mond.</span></p> <p class="dlct-000"><span class="dlct-007">Ro haalde haar hand weg</span><span class="dlct-007">.</span> <span class="dlct-007">'Sorry,'</span> <span class="dlct-007">fluisterde hij</span><span class="dlct-007">. 'Ik weet het niet</span><span class="dlct-007">,</span> <span class="dlct-007">maar we moeten iets doen.'</span></p> <p class="dlct-000"><span class="dlct-007">Met tegenzin schudde Elena haar hoofd. Ze pakte het papiertje en</span> <span class="dlct-007">schreef verder</span><span class="dlct-007">.</span> <span class="dlct-007">'We hebben alleen het huis in het bos over. Misschien dat de professor ons kan helpen. Aan de Codex Trias hebben we niets. Die is veilig bij Bia en Luxis. Hier</span> <span class="dlct-007">kunnen we niet</span> <span class="dlct-007">blijven.</span> <span class="dlct-007">We</span> <span class="dlct-007">moeten</span> <span class="dlct-007">William en Charlotte</span> <span class="dlct-007">bevrijden</span> <span class="dlct-007">en gaan dan naar het bos.'</span></p> <p class="dlct-000"><span class="dlct-007">'</span><span class="dlct-007">Goed plan</span><span class="dlct-007">,' fluisterde Ro</span><span class="dlct-007">.</span><span class="dlct-007"> </span></p> <p class="dlct-000"><span class="dlct-007">Elena pakte het papier en gooide het in de open haard. Ro</span> <span class="dlct-007">nam haar bij de</span> <span class="dlct-007">hand en trok haar door de deur waar hij vandaa</span><span class="dlct-007">n was gekomen</span><span class="dlct-007">. Ze kwam in een gelijksoortige kamer als die van haar alleen was het hier een stuk rommeliger. </span></p>

08-17-2015, 08:58 AM	#4
MikeB1972 Gnu Posts: 1,222 Karma: 15625359 Join Date: Jul 2009 Location: UK Device: BeBook,JetBook Lite,PRS-300-350-505-650,+ran out of space to type	In Sigil use the Regex mode for find/replace In the find box <span class="dlct-007">(.*?)</span> in the replace box \1 Replace all

08-17-2015, 10:07 AM	#9
rubeus Banned Posts: 272 Karma: 1224588 Join Date: Sep 2014 Device: Sony PRS 650	Delete <span class="dlct-\d\d\d"> in Sigil and let do tidy do the rest.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Create / Optimize Cbz files for Kobo (software inside)	satsuki_yatoshi	Kobo Reader	20	06-22-2022 04:23 PM
conversion problem? - cleaning up epub	potestus	Calibre	1	05-31-2011 01:28 PM
Stop Automatic Code cleaning in Sigil	ericp20	Sigil	11	05-27-2011 08:52 AM
questions on epub and lrf and cleaning up book	Janette55	Sony Reader	1	03-11-2011 09:25 AM
Unutterably Silly A pug cleaning the inside of your monitor!	Dusty Bottoms	Lounge	4	05-03-2010 10:06 AM

08-17-2015, 10:00 AM	#8
HarryT eBook Enthusiast Posts: 85,544 Karma: 93383043 Join Date: Nov 2006 Location: UK Device: Kindle Oasis 2, iPad Pro 10.5", iPhone 6	Moved to the "Workshop" forum, where such questions belong.

08-17-2015, 10:43 AM	#11
Toxaris Wizard Posts: 4,520 Karma: 121692313 Join Date: Oct 2009 Location: Heemskerk, NL Device: PRS-T1, Kobo Touch, Kobo Aura	If you want Nick, I can clean it up for you. Just give me a sign.

08-17-2015, 10:54 AM	#12
dickloraine Guru Posts: 631 Karma: 7544080 Join Date: Apr 2013 Location: Berlin Device: PRS 350, Kobo Aura	Is it a novel? If so, maybe use calibre to conver it to for example txt with markdown and then back to epub. Or convert it to docx and use toxaris word add-in. Good luck, it really looks terrible.

Advert

Advert