regex/search help needed to remove redundant metadata in different fields

[Sidetrack]

So I've got some redundant data across fields, such as ISBNs listed in identifiers and in tags. or Authors listed in Tags, or Series Names listed in Tags

Once I identify them I can to a regex search and replace something like this

template: {identifiers}, @@ {tags}
search for: .*\bisbn[0-9x-]+),.*@@(.*)\1(.*)
replace with: \2\3
field: tags

so if an ISBN in a tag matches the ISBN in identifiers it will delete the tag. However, if there is no matching tag, that regex dumps the whole "{identifiers}, @@ {tags}" into the tags field.

If I could a) search for matching items across fields, or b) refine the regex so it doesn't corrupt the tags field when the pattern doesn't match... I'd be happy.

Is there an elegant solution that I'm missing? I don't want a lot of built template columns slowing down operations.

[BetterRed]

@Sidetrack - see Mufti Column Search

I'm not certain not sure it will do what you want, but worth a look

BR

[Sidetrack]

Originally Posted by BetterRed

@Sidetrack - see Mufti Column Search

I'm not certain not sure it will do what you want, but worth a look

BR

That does help, doesn't seem to handle the idenitfiers field that I can see.

[BetterRed]

Originally Posted by Sidetrack

That does help, doesn't seem to handle the idenitfiers field that I can see.

Why don't you post something in the PI's thread outlining what you need, the developer is usually up for making kaizen style enhancements.

BR

[DaltonST]

Originally Posted by Sidetrack

That does help, doesn't seem to handle the idenitfiers field that I can see.

@Sidetrack:

Both the 'Final Filters' Tab and 'SQL Queries' Tab handle identifiers. For your purposes, the 'SQL Queries' Tab should be used.

See the image below of an example using the 'SQL Queries' Tab.

Copy and paste this into that Tab:

SELECT book FROM _mcs_tags_by_book WHERE tagname LIKE '%978%' OR tagname LIKE '%045%' OR tagname LIKE '%isbn%' OR tagname REGEXP '^[0-9][-]*[0-9][-]*[0-9][-]*[0-9][-]*[0-9][-]*[0-9][-]*[0-9][-]*[0-9][-]*[0-9][-]*[0-9]+$'

This will catch both 10-digit and 13-digit ISBNs with and without hyphens.

Table _mcs_tags_by_book is an MCS "Search Accelerator" table.

In lieu of using the latter table, you could JOIN standard Calibre tables tags and books_tags_link. However, if you have a lot of books with a lot of tags, that query will take much longer to execute. Hence, the MCS "Search Accelerator" table.

By the way, the (industrial-strength) QuarantineAndScrub add-on automatically fixes the issues you described in your first post. The example in the image is from an actual Q&S test library with ISBN test cases.


DaltonST