[Plugin] CustomCleanerPlus

[Turtle91]

I'm not sure what you've defined as 'line' but I exclusively use the following style of CSS (where properties can all be on the same line). They may get passed over just because there is a : and a ; in the line...also, they may not have a space after the colon, and may, or may not, have a semi-colon at the end.
Of course, AFTER I'm done with the whole book I use the cssRemoveUnusedSelectors to clean out and organize them...that plugin organizes the css into 'property: value;' per line styling. That styling has a space after the colon, but not after the semi-colon...

Before Organizing:

p.first::first-line {font-variant:small-caps; font-size:1.15em}

After Organizing:

p.first::first-line {
	font-variant: small-caps;
	font-size: 1.15em;
}

The point is, you can't guarantee that people will have a ': ' and a "; " on the same line to prevent changing the '::' to ': : '.... you may want to run some kind of CSS organizer routine to make sure everything is in a standard format that you can code for.

[tamaracks]

Yes, I know. It’s not my code, I was just trying to get to a point where it would stop messing up my CSS.

[Turtle91]

Cool!

The other option is to run a find/replace on your css sheet after you use the plugin:

find: ": :"
replace: "::"

(without the quotes)

[KevinH]

Open Sigil's CV to an css file. Right click pop-up menu on that CV tab should allow you to Reformat CSS to single lines or multiple lines.

[Turtle91]

Originally Posted by KevinH

Open Sigil's CV to an css file. Right click pop-up menu on that CV tab should allow you to Reformat CSS to single lines or multiple lines.

Dang! ANOTHER feature of Sigil of which I wasn't aware!!




It just keeps getting better and better....or I'm getting older and older and not remembering stuff that I used to know

[tamaracks]

Originally Posted by KevinH

Open Sigil's CV to an css file. Right click pop-up menu on that CV tab should allow you to Reformat CSS to single lines or multiple lines.

I’ve not gotten all the Sigil terminology down yet, but I’m assuming you mean the tab of an open CSS file?

[Turtle91]

Not quite.

Just have a css page open in the codeview area. Then you can right-click anywhere in the codeview window to get the pop-up menu. Select "Reformat CSS" then choose between "Multiple Lines per Style" or "Single Line per Style"

[KevinH]

Yes, I was unclear. Anywhere in the CodeView window that has a css file loaded in it, not the little tab at the top.

[slowsmile]

Apologies for my late reply. The reported bug in cutils2.py has already been fixed along with several other plugin problems but I'm unable to release the new version(v0.5.0) yet for good reasons -- I'm in the middle of a major plugin upgrade. The new version of this plugin will not be able to process imported HTML files in Sigil as before -- but it will be able to clean up both EPUB2 and EPUB3 files. I have also fixed many other small problems in that plugin and have also extended it's automatic capabilities as well as added several more user options which I hope will be useful. I've also had to do a massive cleanup of all the plugin code -- in order to remove all the unnecessary code for html file cleanup.

Because I've removed the plugin's ability to process html files I've also had to write another new plugin -- tentatively called HTMLCleaner -- that will just be dedicated to cleaning up html files that are derived from Word or LibreOffice only. I have also added many new and extra features to this plugin as well, including a new "nuke" feature that acts to preserve all text but removes all classes and inline styling, ultimately removing or reverting all html tags to basic <p> tags.

Both the new version of the CustomCleanerPlus plugin and the new html cleaner plugin will be released at the same time, otherwise people who have been regularly using the original CustomCleanerPlus plugin for html cleanup might complain. So my intention is to release these two plugins at the same time which seems the fairest way of doing it. I'm currently busy testing both plugins at the moment, so please bear with me.

[slowsmile]

Major Update(v0.5.0):

Includes fixes and new added features shown below:

1. This plugin no longer supports HTML input. Now only supports Epub input.

2. This plugin can now be used for Epub 2 and Epub 3 versions.

3. Added a selectable feature that automatically removes all hyphen class properties from the CSS(e.g. hyphens, adobe-hyphenate, -moz-hyphens, -webkit-hyphens etc).

4. Added a selectable feature that removes all unnecessary ad hoc black text color declarations from the css and from the html.

5. Added a selectable feature that automatically removes all unnecessary ad hoc white background-color declarations from the css and from the html.

6. Added a selectable option that enables the user to remove all ids and all associated internal links.

7. Added a selectable option that enables the plugin user to remove page links only.

8. Fixed a problem with conversion of <i>, <b>, <u>, <em>, <s> etc to their span equivalents. If you select this option now it will act on ALL html tags containing these tags(whereas before it was only acting on <p> tags only).

11. Added a selectable option which removes any unused images stored in Sigil's 'Images' dir(epubs only).

12. New selectable feature that removes empty paragraphs that contain no text.

13. Added a selectable feature that removes all horizontal rule tags from the html.

14. Added a selectable feature that removes all non-breaking spaces from the html(ie &nbsp; )

15. Added a selectable feature that replaces all <div> tags with <p> tags.

16. Automatically removes all line-height declarations in the CSS.

17. Automatically repairs the image names by replacing all illegal spaces with underscores.

18. Updated plugin Release Notes.

[Leonatus]

Sigil v. 1.9.20, Plugin v. 0.5.0, Windows 10, 64 bit.
I applied the plugin to an epub2 and got the following message:

Status: failed

Python version:  3.9.9 (tags/v3.9.9:ccb0e6a, Nov 15 2021, 18:08:50) [MSC v.1929 64 bit (AMD64)] 

Running CustomCleanerPlus...

 -- Copy to work dir...wrap0000.html
 -- Copy to work dir...4791824656362272078_16425-8-0.txt.html
 -- Copy to work dir...4791824656362272078_16425-8-1.txt.html
 -- Copy to work dir...4791824656362272078_16425-8-2.txt.html
 -- Copy to work dir...4791824656362272078_16425-8-3.txt.html
 -- Copy to work dir...4791824656362272078_16425-8-4.txt.html
 -- Copy to work dir...4791824656362272078_16425-8-5.txt.html
 
 -- Copy images to work dir...C:\Users\Juerg\AppData\Local\Temp\tmpjeb5476q\6308072537968574473_16425-cover.png
 -- Write to work dir...C:\Users\Juerg\AppData\Local\Temp\tmpjeb5476q\pgepub.css
 -- Write to work dir...C:\Users\Juerg\AppData\Local\Temp\tmpjeb5476q\0.css

 -- Processing automatic tasks...
 -- Clean and reformat the html

>>> In getHREF_ID_Epub...
>>> xhtml file list...
wrap0000.html
4791824656362272078_16425-8-0.txt.html
4791824656362272078_16425-8-1.txt.html
4791824656362272078_16425-8-2.txt.html
4791824656362272078_16425-8-3.txt.html
4791824656362272078_16425-8-4.txt.html
4791824656362272078_16425-8-5.txt.html
TOC.NCX ids added to HREF_IDS..
pg-header
TOC.NCX ids added to HREF_IDS..
id00002
TOC.NCX ids added to HREF_IDS..
id00003
TOC.NCX ids added to HREF_IDS..
id00004
TOC.NCX ids added to HREF_IDS..
id00018
TOC.NCX ids added to HREF_IDS..
id00020
TOC.NCX ids added to HREF_IDS..
id00021
TOC.NCX ids added to HREF_IDS..
id00030
TOC.NCX ids added to HREF_IDS..
id00041
TOC.NCX ids added to HREF_IDS..
id00042
TOC.NCX ids added to HREF_IDS..
id00043
TOC.NCX ids added to HREF_IDS..
id00080
TOC.NCX ids added to HREF_IDS..
id00104
TOC.NCX ids added to HREF_IDS..
id00153
TOC.NCX ids added to HREF_IDS..
id00185
TOC.NCX ids added to HREF_IDS..
id00250
TOC.NCX ids added to HREF_IDS..
id00393
TOC.NCX ids added to HREF_IDS..
id00508
TOC.NCX ids added to HREF_IDS..
id00607
TOC.NCX ids added to HREF_IDS..
id00661
TOC.NCX ids added to HREF_IDS..
id00842
TOC.NCX ids added to HREF_IDS..
id00946
TOC.NCX ids added to HREF_IDS..
id01007
TOC.NCX ids added to HREF_IDS..
id01082
TOC.NCX ids added to HREF_IDS..
id01148
TOC.NCX ids added to HREF_IDS..
id01225
TOC.NCX ids added to HREF_IDS..
id01295
TOC.NCX ids added to HREF_IDS..
id01361
TOC.NCX ids added to HREF_IDS..
id01419
TOC.NCX ids added to HREF_IDS..
id01467
TOC.NCX ids added to HREF_IDS..
id01538
TOC.NCX ids added to HREF_IDS..
id01620
TOC.NCX ids added to HREF_IDS..
pgepubid00000

 >>> File type is: HTML
 >>> Doctype is: 
 >>> Epub version is: 
Traceback (most recent call last):
  File "C:\Program Files\Sigil\plugin_launchers\python\launcher.py", line 142, in launch
    self.exitcode = target_script.run(container)
  File "C:\Users\Juerg\AppData\Local\sigil-ebook\sigil\plugins\CustomCleanerPlus\plugin.py", line 160, in run
    processAllTasks(bk, WDIR, t_fnames, s_fnames, s_ids)
  File "C:\Users\Juerg\AppData\Local\sigil-ebook\sigil\plugins\CustomCleanerPlus\cutils2.py", line 61, in processAllTasks
    docTidyNoWrap(wdir, file)
  File "C:\Users\Juerg\AppData\Local\sigil-ebook\sigil\plugins\CustomCleanerPlus\doc_tidy.py", line 131, in docTidyNoWrap
    html, errors = tidy_document(xhtml, options=base_options)
  File "C:\Users\Juerg\AppData\Local\sigil-ebook\sigil\plugins\CustomCleanerPlus\tidylib\tidy.py", line 292, in tidy_document
    return get_module_tidy().tidy_document(text, options)
  File "C:\Users\Juerg\AppData\Local\sigil-ebook\sigil\plugins\CustomCleanerPlus\tidylib\tidy.py", line 304, in get_module_tidy
    _tidy = Tidy()
  File "C:\Users\Juerg\AppData\Local\sigil-ebook\sigil\plugins\CustomCleanerPlus\tidylib\tidy.py", line 158, in __init__
    raise OSError(
OSError: 
Could not load library: C:\Users\Juerg\AppData\Local\sigil-ebook\sigil\plugins\CustomCleanerPlus\tidylib\win64\tidy.dll
Error: 
Could not load library: C:\Users\Juerg\AppData\Local\sigil-ebook\sigil\plugins\CustomCleanerPlus\tidylib\win64\tidy.dll

I understood that it didn't work, because the content is HTML. But, what can I do, please?

[slowsmile]

@Leonatus. . . It looks like the plugin cannot find the tidylib file for the plugin. I've just downloaded the plugin from Sigil/MobileRead and tested it successfully on several different epubs without any problems. Perhaps you can try downloading and re-installing the plugin again from MR and retry.

I also wasn't sure whether you were trying to use my plugin to clean up an imported HTML file. Is that what you were trying to do? As I made clear in the update and in the Release Notes, this new version of the plugin(v0.5.0) no longer supports cleanup for HTML files imported into Sigil. I decided to remove that capability because I wanted to extend the plugin's use to include both Epub 2 and Epub 3 files and it was too complicated to do that with the html cleanup capability as well, so I removed it.

At some point, it's also my intention to write a new plugin that will be dedicated to just cleaning up imported HTML files in Sigil.

[Leonatus]

Originally Posted by slowsmile

. . . Perhaps you can try downloading and re-installing the plugin again from MR and retry.

Doing as you suggest without avail: It's the same error message.

I also wasn't sure whether you were trying to use my plugin to clean up an imported HTML file. Is that what you were trying to do?

No, sorry about my last observation in my post - that was a misunderstanding. It's a regular epub2, and meanwhile I tested the plugin with other epubs as well, always with the same result.

[slowsmile]

@Leonatus. . .I'm still not able to resolve your problem. I've already tested many Epub2 and Epub3 files on this plugin without any problems at all. I also note that, in the past, one person did report exactly the same problem as you and I was unable to resolve or fix that problem because I could not emulate the fault using the that plugin on my computer.

Since you get this failure with all epubs that are run with this plugin and since no one else has so far reported this problem recently, I suspect that your plugin problem has probably been caused by a setup problem or a system problem on your computer. I mention this because you've said that all your epubs fail with the same "Could not load [tidylib] library" using the plugin and yet when I run various epubs using this plugin on my computer they don't fail.

If you like, I could contact you and send you my email link via MR messaging and you could send me one of the epubs that fails and I will run the plugin on that epub on my computer system to see if I can emulate your problem or not. Please let me know.

[KevinH]

The real error in that output log is:

Could not load library: C:\Users\Juerg\AppData\Local\sigil-ebook\sigil\plugins\CustomCleanerPlus\tidylib\win6 4\tidy.dll


So my guess is the tidy.dll file inside the plugin could not be found, or if found, can not be loaded because it is in some way incompatible.

Things to explore:

- could your python be 32 bit and the tidy.dll be 64 bit, or visa-versa? Same for Sigil?

- could the provided tidy.dll be linked with some other dll that does not exist on the target system but does exist on slowsmile's system as it may have been built there?

I am thinking of something like tidy.dll might needs a specific lib c++ to exist on the target system?

You could try testing the plugin on a another system that was never used to build the tidy.dll just to see if it works there.