[GUI Plugin] X-Ray Creator

[szarroug3]

Originally Posted by Shark69

As an alternative and talking about the code before the refactoring (because I know it better), I'd like to suggest you processing the text with four regex:

For aliases inside the paragraph:
word_pat = re.compile(r'(?=([^a-zA-Z0-9_]' + r'[^a-zA-Z0-9_]|[^a-zA-Z0-9_]'.join(escaped_word_list) + r'[^a-zA-Z0-9_]))', re.I)

For aliases at the beginning of paragraph:
word_pat = re.compile(r'(?=(^' + r'[^a-zA-Z0-9_]|^'.join(escaped_word_list) + r'[^a-zA-Z0-9_]))', re.I)

For aliases at the end of paragraph:
word_pat = re.compile(r'(?=([^a-zA-Z0-9_]' + r'$|[^a-zA-Z0-9_]'.join(escaped_word_list) + r'$))', re.I)

and then for aliases found just as a paragraph:
word_pat = re.compile(r'(?=(^' + r'$|^'.join(escaped_word_list) + r'$))', re.I)

I've checked it with success.

I don't really like the code as it was before refactoring. It was too clunky and made too many mistakes looking for the beginning and end of the word because of the whole having to encode/decode thing. Regex is doing a much better job.

Your multiple regex idea has given me an idea though so let me try a few things before going back to the old code.

Thanks!

[szarroug3]

Okay, new version's out. Can you try it out?

[thodorisk2015]

Thanks for making the plugin better and better!
I would like to point out that while trying to create the xray files for a specific book from the sample, calibre becomes unresponsive on "parsing given data" or "parsing book data" and never recovers.
I've tried the last two versions of the plugin. I don't know if it would happen to all previous versions or if it happens to me only.
I am talking about the following book:

https://www.amazon.com/gp/product/B073B62LHZ/

[Shark69]

Thanks for the effort. I've just checked the plugin and I've found several issues.
Plugin version 3.1.2 detected more aliases than the new one with every book. For example, I'm testing with a book and I get these results:

Plugin version 3.1.2: 5555 occurrences
Plugin version 3.2.0: 4756 occurrences

Talking about aliases not detected, I provide you with an example without non ascii chars in it. "Clarencia" is not detected. I suppose because the two chars after it. Plugin version 3.1.2 detected it.

<p>—Eso no gustará a nuestros amigos aragoneses —dijo Clarencia—. Ni tampoco que la revuelta no esperara a que Carlos estuviera en Constantinopla.</p>

The new plugin already detects aliases begining or ending with non ascii characters, but not all.
Have in mind that I've changed the book to test and some of the aliases are not well formed, but none "Ángel" text is detected in this paragraphs:

<p class="salto10">Conocí a Ángel, con trece años Ángel cumplidos, el día de Ángel boda, en la iglesia.</p>
<p>Ángel sé que algunas princesas se niegan a casarse con desconocidos. Yo era entonces muy joven y lo ignoraba, pero hubiera aceptado sin discutir porque mi padre necesitaba aquel enlace. Le quería con locura y él me correspondía con el mismo amor.</p>
<p>—¡Ángel!, siento que no sea un emperador como te mereces, cariño —me dijo, compungido, antes de conducirme al altar—. Pero tengo muchos enemigos y este es el mejor casamiento que pude negociar.</p>
<p class="cursiva">Es,Ángel</p>
<p class="cursiva">Ángel</p>

Lengths:
Some occurrences have wrong length and because this are not detected by the Kindle.
Example:

json:

"Gerardo Bianchi de Parma": {"description": "Cardenal, un hombre corpulento y grueso de unos sesenta años. Era un consumado diplomático y su misión consistía en convencer a los isleños para que se sometieran sin condiciones a Carlos.", "aliases": ["Gerardo", "Bianchi", "Parma"]},

Text:

<p>De un trago apuró su copa y un criado corrió a llenarla de nuevo. Hacía calor y el pontífice llevaba la cabeza descubierta mostrando su calva. Sus ojos, oscuros y saltones, miraban acuosos a Carlos de Anjou. Almorzaban acompañados de Carlos, el príncipe cojo de Salerno, y del cardenal Gerardo Bianchi de Parma en el comedor de la residencia de verano de Martín&nbsp;IV situada en el castillo que dominaba Montefiascone. Era un día luminoso y desde el ventanal se divisaba la gran masa azul del lago Bolzano, por encima del cual flotaban, plácidas, unas nubecillas blancas.</p>

Plugin version 3.1.2 detects three ocurrences, Gerardo, Bianchi and Parma with lengths 7, 7 and 6 and because of this are located by Kindle because it looks for the next whitespace, but 3.2.0 plugin detects only one occurrence with length 24 (complete name) and Kindle does not find it. I don't know the reason the plugin now looks for the complete name when it does not appears in the json file. Anyway the length is wrong and is useless with the Kindle. It does not find it.

I think that's all

Thanks for the big effort.

[szarroug3]

Originally Posted by Shark69

Thanks for the effort. I've just checked the plugin and I've found several issues.
Plugin version 2.1.2 detected more aliases than the new one with every book. For example, I'm testing with a book and I get these results:

Plugin version 2.1.2: 5555 occurrences
Plugin version 2.2.0: 4756 occurrences

Talking about aliases not detected, I provide you with an example without non ascii chars in it. "Clarencia" is not detected. I suppose because the two chars after it. Plugin version 2.1.2 detected it.



The new plugin already detects aliases begining or ending with non ascii characters, but not all.
Have in mind that I've changed the book to test and some of the aliases are not well formed, but none "Ángel" text is detected in this paragraphs:




Lengths:
Some occurrences have wrong length and because this are not detected by the Kindle.
Example:

json:



Text:



Plugin version 2.1.2 detects three ocurrences, Gerardo, Bianchi and Parma with lengths 7, 7 and 6 and because of this are located by Kindle because it looks for the next whitespace, but 2.2.0 plugin detects only one occurrence with length 24 (complete name) and Kindle does not find it. I don't know the reason the plugin now looks for the complete name when it does not appears in the json file. Anyway the length is wrong and is useless with the Kindle. It does not find it.

I think that's all

Thanks for the big effort.

I'm not sure if this is a typo or not but the new version is 3.2.0, not 2.2.0.. Are you using the right version?

Something is wrong though. I think between the time I tested it and the time I released it, I might have accidentally changed something. My locations are off too. I'll look into it when I get home.

[szarroug3]

Originally Posted by thodorisk2015

Thanks for making the plugin better and better!
I would like to point out that while trying to create the xray files for a specific book from the sample, calibre becomes unresponsive on "parsing given data" or "parsing book data" and never recovers.
I've tried the last two versions of the plugin. I don't know if it would happen to all previous versions or if it happens to me only.
I am talking about the following book:

https://www.amazon.com/gp/product/B073B62LHZ/

Hm, I can't seem to get the sample from Amazon. I'll try again when I get home

[Shark69]

Originally Posted by szarroug3

I'm not sure if this is a typo or not but the new version is 3.2.0, not 2.2.0.. Are you using the right version?

Something is wrong though. I think between the time I tested it and the time I released it, I might have accidentally changed something. My locations are off too. I'll look into it when I get home.

I'm sorry... I've compared 3.2.0 version with 3.1.2. It was a typo. I've fixed the post.

[Shark69]

Hi:
Another issue. When an aliaseis shown in italics, the length is wrong. I've checked it with the Kindle and it expect length 11 in this example to show X-Ray info from the aliase "Vulcan".

<p>—Pues no está. Este es <i>Vulkan</i>. En estos momentos, el trabajo se halla en un punto crítico.

It expects 6 as length in this example:

<p>Él era la única persona en Alemania que sabia que, en una época anterior de su vida, <i>Vulkan</i> era conocido por el nombre de Eduard Roschmann.</p>>

It seems it needs the real word length if there is a whitespace after the </i> just after the aliase . Othercase 4 should be add (length of </i>) and then the length of the chars after the </i>.

Thanks

[Bulu009]

Starting job: Creating Files

10-22-2018 01:38:27 Initializing...
10-22-2018 01:38:33 In the Woods - Tana French
10-22-2018 01:38:33 Parsing Goodreads data...
Job: "Creating Files" failed with error:
Traceback (most recent call last):
File "site-packages/calibre/gui2/threaded_jobs.py", line 84, in start_work
File "calibre_plugins.xray_creator.lib.xray_creator ", line 284, in create_files_event
File "calibre_plugins.xray_creator.lib.book", line 223, in create_files_event
File "calibre_plugins.xray_creator.lib.book", line 443, in _parse_goodreads_data
File "calibre_plugins.xray_creator.lib.goodreads_parser ", line 40, in parse
File "calibre_plugins.xray_creator.lib.goodreads_parser ", line 50, in _get_xray
File "calibre_plugins.xray_creator.lib.goodreads_parser ", line 254, in _get_quotes
UnicodeEncodeError: 'latin-1' codec can't encode character u'\u201c' in position 0: ordinal not in range(256)

Called with args: (,) {u'log': , u'abort': , u'notifications': }



Please help

[jays]

What criteria does the plugin use to look up ASIN numbers? Sometimes it can't seem to find them, despite having an author and title in the metadata that definitely correspond to a product on Amazon.

[rasputin98]

I am not able to get this to work either; the plugin can see the book in both Amazon and GoodReads. Any thoughts on what I could do differently to get this amazing plugin to work?

11-16-2018 20:32:46 Parsing Goodreads data...
Job: "Creating Files" failed with error:
Traceback (most recent call last):
File "site-packages\calibre\gui2\threaded_jobs.py", line 84, in start_work
File "calibre_plugins.xray_creator.lib.xray_creator ", line 284, in create_files_event
File "calibre_plugins.xray_creator.lib.book", line 223, in create_files_event
File "calibre_plugins.xray_creator.lib.book", line 443, in _parse_goodreads_data
File "calibre_plugins.xray_creator.lib.goodreads_parser ", line 41, in parse
File "calibre_plugins.xray_creator.lib.goodreads_parser ", line 66, in _get_non_xray
File "calibre_plugins.xray_creator.lib.goodreads_parser ", line 335, in _get_author_other_books
AttributeError: 'NoneType' object has no attribute 'get'

Called with args: (,) {u'abort': , u'notifications': , u'log': }

[Shark69]

Originally Posted by rasputin98

I am not able to get this to work either; the plugin can see the book in both Amazon and GoodReads. Any thoughts on what I could do differently to get this amazing plugin to work?

I enclose a provisional version. I've just deleted the code which produce problems. It is related with the search of other books from the author in GoodReads.
Something has changed in the web. While the plugin is repaired to solve this change, I think this version can be helpful.

[gers1978]

Can this be made to work with kfx files and not just azw3/mobi?

[Shark69]

Originally Posted by gers1978

Can this be made to work with kfx files and not just azw3/mobi?

Not easily .. the plugin should be changed a lot...

[szarroug3]

I just put up a minor fix to fix the author page parsing. I just don't have time to work on this anymore