[GUI Plugin] Extract ISBN

[dm101]

@kiwidude,
thank you very much for this update :-)
greets dm101

[kiwidude]

Changes in this release:

• Ensure stripped HTML tags replaced with a ! to prevent ISBN running into another number making it invalid

I found a situation where the decision to strip all html tags in 1.3 caused some ISBNs not being detected. It is because the raw html had a <br/> tag shielding the ISBN number from the next line, which if it coincidentally happened to start with a number meant that the two numbers got merged together. As the combined length was not valid for an ISBN the number would get thrown away. This release fixes that problem.

[theaccountant]

I second xXTGMKXx idea!


"Now that I think about it though... it is as murky as you thought. Since search parameters would start to confuse each other. I think my solution of a yes/no column is more elegant... if you could somehow change your plugin to create a yes/no marker... let's call it "Extracted" and mark those updated with a checkmark, and those failed with an x, that would be pretty elegant. By that logic, you could still have the option to view the updated isbns at the end of the job - and you could also leave the user the option to search on their own terms... for example "identifiers:false & extracted:false" would return a clean list of documents yet to be scanned"

Is the any way that after the ISBN is extracted that it could be preserved and not oevrwriiten when downloading metadata? My 10 digits numbers and being replaced but 13 digit codes.

Also would it possible to have this as an external tool?

Thanks

[kiwidude]

Originally Posted by theaccountant

I second xXTGMKXx idea!

The problem is that any value I applied could only be temporary and would not survive a calibre restart. And the set of data being marked would give you different results every time you ran the scan against a different selection as it would have no memory of books you have scanned previously. At the moment I only mark the books that were updated from your selection. I could add marking of books that were scanned but it doesn't seem very useful. I could add marking books that were not updated but as I said above that is complicated by the reasons why they weren't updated.

There seems to be several different questions related to this floating around between the various posts. As looking at what books in your library that do not have an isbn and that you haven't scanned as yet is a totally different requirement from books that you just scanned but could get no isbn from. So until I see some clarity on what it is you are trying to achieve I am not going to change the current behaviour.

My usage of extract isbn is pretty simple as I just add a bunch of books, select them and extract then run download metadata. I don't get hung up on having an exact value from the book, to me it is just a tool to increase the chances of metadata download picking the right book. So if extract isbn fails I don't care so long as title author search gives me the right metadata result. And if that fails then I use my goodread sync plugin for it's link book feature to search Goodreads website for a more useful edition, drag drop the URL back onto the linked book dialog and have that plugin configured to overwrite the isbn. So then I can just fire the metadata download again.

Originally Posted by theaccountant

Is the any way that after the ISBN is extracted that it could be preserved and not oevrwriiten when downloading metadata? My 10 digits numbers and being replaced but 13 digit codes.

No. That isn't anything to do with this plugin, that is just the way the metadata download works, and it prefers 13 digit isbns.

Originally Posted by theaccountant

Also would it possible to have this as an external tool?

Thanks

No. Why would you want to, how are you going to tell it which books to scan?

However you could take a look at the scripts thread linked from the first post in this thread. Of course those scripts have very different internal code to what this plugin now does. I originally based the plugin on logic in one of the scripts but found a number of issues with it so since then it works very differently in it's approach to identifying isbn values in general as well as for better and faster PDF scanning.

[theaccountant]

Thanks for your response.

[QUOTE=kiwidude;1546108]The problem is that any value I applied could only be temporary and would not survive a calibre restart. QUOTE]

If the plugin created and used a 2nd column called ISBN Output. The output from the isbn extractor "show details " could be posted to the new column.
ISDN matched
ISBN not Found
ISBN Number result

In otherwords the results from the plugin would be posted to a new field in the database as well as to the ISBN field.

Then the results would be permanent in the DB and would survive a restart.

[kiwidude]

Yes I could have mentioned that a custom column would be a "permanent" solution. However I am reluctant to go down that route as we are talking a (in my opinion) extremely niche requirement. Very few users will be bothered with cluttering their view with yet another column that records something as trivial as whether they have run the extract ISBN plugin on a book. And for the sake of a couple of extra clicks, you can do this yourself manually.

[kiwidude]

Changes in this release:

• Run the ISBN extraction out of process to get around the memory leak issues

This release requires Calibre 0.8.2

I have decided to keep both methods of scanning (threaded job versus worker job) optionally available in the plugin. There is now a configurable threshold at which it will switch between them. By default this threshold is set to one selected book. So if you select just one book, the scan will run as a threaded job as per the changes I made for 1.3. This is the fastest way to get an ISBN, but will continue to suffer from the known memory leak issue if you scan hundreds of certain book formats over a long period of keeping Calibre open.

If you select more than one book, then the scan runs as a worker job, just like book conversions do. This would be a little slower for just a single book but faster overall if you select higher numbers of books at once. This method will not suffer from the memory leak issue.

You can adjust the threshold on the plugin configuration screen as per the screenshot.

EDIT: I left some debug code in the plugin which a couple of you had downloaded before I caught it - if you were one of the first two downloaders please just download it again.

[drMerry]

just one problem with the reverse lookup funciton.

One of my books has this info on one of the last pages:

ISBN

9789077740798 (ebook)
9789077740606 (gedrukte uitgave)

You implemented a function to do a reverse lookup for the last pages (in reaction on my question related to books with read also isbn numbers (and ISBN for current document as last one)).

The numbers above are two numbers of the same book. First is the number for ebooks, second for hard copies.
So in this case I would like to get the first number. But the plugin should give me the second because of stated implementation.

Curious enough, the first number is returned nice in this case but not desired in most others.

Spoiler:

Starting job: Extract ISBN for 1 books
Running scan for isbn query with parameters:
{u'paths': [(u'EPUB', u'H:\\Local (swart)\\Madelon Schoemaker\\Spanje Voorgoed (7546)\\Spanje Voorgoed - Madelon Schoemaker.epub')], u'timeout': 30, u'title': u'Spanje Voorgoed'}
Scanning: H:\Local (swart)\Madelon Schoemaker\Spanje Voorgoed (7546)\Spanje Voorgoed - Madelon Schoemaker.epub
Valid ISBN13: 9789077740798
Valid ISBN13: 9789077740606
Scan time: 4.80999994278 Spanje Voorgoed
The isbn was found in 4.81 seconds
New ISBN extracted of 9789077740798 for Spanje Voorgoed
Scan complete, with 0 failures

[xXTGMKXx]

looking at what books in your library that do not have an isbn and that you haven't scanned as yet

First of all, there's your clarification.

Second of all, I understand it's a niche. As a matter of fact I wouldn't even expect to use it much after I've sorted my 100,000 or so. However for any amount over 500... this is useful... and I know for a fact I'm not the only person who downloads large collections.

On the other hand, your argument of cluttering the view is flawed... I actually hid my custom #extracted column. I only need to know it's there for searching purposes.

Finally, as I said before... I've solved the problem from my perspective. I'm not on some sort of crusade to change YOUR EXTREMELY USEFUL AND APPRECIATED PLUGIN. I'm glad someone agreed with me, and I would like to point them back to my assertion that a custom, as needed solution is perfectly suitable. Anyway, I consider the case closed unless you want to contact me further on the issue.

Thanks again for the brilliant automation tool! I wish I had a credit card to activate my paypal account, I'd drop you a fin for your contribution.

All the best,
Matt

[kiwidude]

Reverse lookups only take place for PDFs.

[drMerry]

Originally Posted by kiwidude

Reverse lookups only take place for PDFs.

That's it. My file is an ePub.
Thanks!

(and it IS a great plugin! )

[dm101]

Hi kiwidude,

yes it's a very helpful tool, and the newest modification is great, calibre will not break down :-)

could you please add the option:
"delete existing isbn, if no isbn was found"

because of using the old version of your plugin, i have much isbn numbers, that will not match the books....

thank you
gereets
dm101

[theducks]

Originally Posted by dm101

Hi kiwidude,

yes it's a very helpful tool, and the newest modification is great, calibre will not break down :-)

could you please add the option:
"delete existing isbn, if no isbn was found"

because of using the old version of your plugin, i have much isbn numbers, that will not match the books....

thank you
gereets
dm101

That is a very bad idea

Not all books in the library may have had an ISBN included within the document
That does not make the ISBN you have in the metadata, incorrect. (nor, make correct )

[dm101]

in my library are only isbn numbers extracted with this plug-in (in an older version), and now i have much wrong numbers.
i wish only a checkbox for deleting the isbn numbers that not exist in this document.
if you don't want to use this option, you will not activate the checkbox.....
greets
dm101

[kiwidude]

Changes in this release:

• Add yet another unicode variation of the hyphen separator to the regex

Thanks to dm101 for sending me the PDFs to try this on. You would think there are only so many variations of the separator that could be used between numbers that all look the same to the naked eye...