Editing metadata comments in bulk

[phossler]

Many time after downloading metadata, there are a lot of <tags> in comments, esp. the <hx> tags

Using bulk edit I can change or delete a lot of the fields, but there doesn't seem to be any way to just remove the metadata formatting in the comments field, but leave the text (at least as far as I can find)

As a little test I added some formatting to a number of books (first screen shot) and tried bulk edit [Set the Comments...] (screen shot 2) and selected remove formatting and normal (hoping it would just delete the <hx> tags and the bolding and italicing, leaving the basic text

But that did as I suspected it would and cleared the metadata comments text also.

Is there an option I'm missing or is there a work around to just clear the formats in bulk, or is it a one book at a time operation?

Thanks

[theducks]

Originally Posted by phossler

Many time after downloading metadata, there are a lot of <tags> in comments, esp. the <hx> tags

Using bulk edit I can change or delete a lot of the fields, but there doesn't seem to be any way to just remove the metadata formatting in the comments field, but leave the text (at least as far as I can find)

As a little test I added some formatting to a number of books (first screen shot) and tried bulk edit [Set the Comments...] (screen shot 2) and selected remove formatting and normal (hoping it would just delete the <hx> tags and the bolding and italicing, leaving the basic text

But that did as I suspected it would and cleared the metadata comments text also.

Is there an option I'm missing or is there a work around to just clear the formats in bulk, or is it a one book at a time operation?

Thanks

Not in Bulk mode, but in Single mode, yes.

You need to select the Text to affect (in Comments: Normal View) and use the recycle icon

That cleans just that

[phossler]

Originally Posted by theducks

Not in Bulk mode, but in Single mode, yes.

You need to select the Text to affect (in Comments: Normal View) and use the recycle icon

That cleans just that

Yea, that was the "one book at a time" I was trying to avoid.

1. Select all text
2. Remove formatting (to remove bold and italic)
3. Apply Normal (to remove h-tags)

[Terisa de morgan]

Perhaps you can try this, I think it works. I extracted this small piece of code as a plugin for a friend with a similar problem.

[phossler]

Nice - thanks

One tiny request

Is there a way to not add the '#' and '*' prefixes and suffixes?

If you prefer, you can tell me the lines to change, I'll change my copy ... please

[Terisa de morgan]

I don't understand, when is it adding those characters? I'll test it this night at home and I'll be back to you.

[phossler]

Thanks -- appreciate it

I had applied various formats in Normal View to several of my test epubs.

After running the PI, the bold and/or italic and/or underline text gained a '*' and the headings gained a '#' for each heading level

[Terisa de morgan]

Yes, I'm looking at the code and I see that it's not really a conversion to plain text but markup text. I'll have to add some replaces after the conversion.

[Terisa de morgan]

Well, I think this will clean '*' and '#'. Feel free to tell me if you find any problem (as I will send it to my friend, because I don't use it, au contraire, I format the comments for sending to my Kobo )

[phossler]

It works very well - thank you very much

It will make things easier for me

[phossler]

I was using the PI to normalize the comments before I re-generated a catalog. I was going by grouping authors

If there is not a comment in a book then there is a error

Spoiler:

calibre, version 2.52.0
ERROR: Unhandled exception: <b>TypeError</b>:cannot concatenate 'str' and 'NoneType' objects

calibre 2.52 [64bit] embedded-python: True is64bit: True
Windows-8-6.2.9200 Windows ('64bit', 'WindowsPE')
('Windows', '8', '6.2.9200')
Python 2.7.9
Windows: ('8', '6.2.9200', '', 'Multiprocessor Free')
Successfully initialized third party plugins: Quality Check && EpubMerge && Generate Cover && Manage Series && Open With && EpubSplit && NormComment && Diaps Editing Toolbag && View Manager && Modify ePub
Traceback (most recent call last):
File "calibre_plugins.norm_comment.action", line 124, in _normalize_comment
File "calibre_plugins.norm_comment.action", line 149, in _do_replace_text
File "site-packages\calibre\utils\html2text.py", line 376, in html2text
File "site-packages\calibre\utils\html2text.py", line 371, in html2text_file
File "sgmllib.py", line 103, in feed
TypeError: cannot concatenate 'str' and 'NoneType' objects

Any chance you could add another tweak for me please?

[Terisa de morgan]

Yeah, no problem, this should be controlled from the beginning . I'll be back with a new version

[Terisa de morgan]

Hi,

Try this and sorry for the newbie mistake.

[phossler]

You're the one doing me the favor, so absolutely no need be sorry about anything

Thanks again

It's working great and by stripping out all the html tags, my catalog looks much neater

[Capricorn]

Hi Terisa,

I just discovered your normcomment plugin and it works well for me, as I like to remove as much formatting as possible. Thank you very much for making it available.

Unfortunately, I have just run into this problem. One book has the following comment:
<div><p></p><blockquote><p>As startling and powerful as when first published more than two decades ago, André Brink's classic novel, <em>A Dry White Season</em>, is an unflinching and unforgettable look at racial intolerance, the human condition, and the heavy price of morality.</p></blockquote><p>Ben Du Toit is a white schoolteacher in suburban Johannesburg in a dark time of intolerance and state-sanctioned apartheid. A simple, apolitical man, he believes in the essential fairness of the South African government and its policies—until the sudden arrest and subsequent "suicide" of a black janitor from Du Toit's school. Haunted by new questions and desperate to believe that the man's death was a tragic accident, Du Toit undertakes an investigation into the terrible affair—a quest for the truth that will have devastating consequences for the teacher and his family, as it draws him into a lethal morass of lies, corruption, and murder.</p></div>

And when I run your plugin I get the following error:

calibre, version 3.32.0
ERROR: Unhandled exception: <b>ValueError</b>:invalid literal for int() with base 10: '8212a'

calibre 3.32 [64bit] embedded-python: True is64bit: True
Windows-10-10.0.17763-SP0 Windows ('64bit', 'WindowsPE')
('Windows', '10', '10.0.17763')
Python 2.7.12+
Windows: ('10', '10.0.17763', 'SP0', u'Multiprocessor Free')
Interface language: en_GB
Successfully initialized third party plugins: Gather KFX-ZIP (from KFX Input) (1, 9, 0) && DeDRM (6, 6, 1) && Package KFX (from KFX Input) (1, 9, 0) && Fantastic Fiction (1, 2, 0) && Modify ePub (1, 3, 13) && Find Duplicates (1, 6, 3) && BOL_DE (0, 7, 5) && Quality Check (1, 9, 11) && Goodreads (1, 1, 16) && NormComment (0, 0, 2) && Diaps Editing Toolbag (0, 3, 5) && FictionDB (1, 0, 10) && EpubCheck (0, 1, 2) && Barnes & Noble (1, 2, 15) && KFX metadata reader (from KFX Input) (1, 9, 0) && KFX Input (1, 9, 0)
Traceback (most recent call last):
File "calibre_plugins.norm_comment.action", line 125, in _normalize_comment
File "calibre_plugins.norm_comment.action", line 154, in _do_replace_text
File "site-packages\calibre\utils\html2text.py", line 436, in html2text
File "site-packages\calibre\utils\html2text.py", line 430, in html2text_file
File "sgmllib.py", line 104, in feed
File "sgmllib.py", line 186, in goahead
File "site-packages\calibre\utils\html2text.py", line 200, in handle_charref
File "site-packages\calibre\utils\html2text.py", line 72, in charref
ValueError: invalid literal for int() with base 10: '8212a'

I have no idea what it all means. If you have time, I would appreciate it if you could look at it.

The version of your plugin installed is 0.0.2


Many thanks