Hacks X-Ray file generator for Kindle Touch

[Ephemerality]

Originally Posted by zcqsimon

Does this also work for kindle paper white?

Yes, it does.

[KingViper]

Originally Posted by Ephemerality

Could you post a screenshot/example of the first issue? Not sure what it's doing based on the description.

The "click a word to view xray" feature is relatively new in the firmware (make sure you're on the latest), so I haven't messed around with it a lot. It does seem a bit iffy for multi-word names. It matches based on the location data; if you go into the entry for that particular character, does it show an excerpt from that specific part of the page you're trying to click? If it does and the name is highlighted properly in that except, it should be working.

So here is an example of the aliases that sometimes include extra characters.
Mis-identified Aliases

Here is a short video showing an alias that seems to be properly identified, and yet doesn't allow me to select the alias directly to get to the xray data.
Video of non-functioning xray aliases

FYI. The video uploaded the wrong orientation by default, its currently reprocessing to rotate it.

[Ephemerality]

Originally Posted by KingViper

So here is an example of the aliases that sometimes include extra characters.
Mis-identified Aliases

That's really bizarre. It seems like the lengths are getting generated wrong somehow or they aren't matching the locs in the book (generating for a DRMed copy and then using it on a DRMed book causes weird issues like that)... Even the one that looks normal might include the space in front of it, which would explain why it doesn't work when you tap it.
If you send the book/xray/alias file I can take a look at it. Alternatively you could try the stand-alone (I'm assuming you used the plug-in) and see if you have better results to compare.

[KingViper]

Originally Posted by Ephemerality

That's really bizarre. It seems like the lengths are getting generated wrong somehow or they aren't matching the locs in the book (generating for a DRMed copy and then using it on a DRMed book causes weird issues like that)... Even the one that looks normal might include the space in front of it, which would explain why it doesn't work when you tap it.
If you send the book/xray/alias file I can take a look at it. Alternatively you could try the stand-alone (I'm assuming you used the plug-in) and see if you have better results to compare.

I got around to trying the command line tool and it worked perfectly fine. It actually detected more instances of the aliases and all functions related to X-Ray seem to work now. Thanks for the help!

[mjwilson]

Originally Posted by Ephemerality

That's really bizarre. It seems like the lengths are getting generated wrong somehow or they aren't matching the locs in the book (generating for a DRMed copy and then using it on a DRMed book causes weird issues like that)... Even the one that looks normal might include the space in front of it, which would explain why it doesn't work when you tap it.
If you send the book/xray/alias file I can take a look at it. Alternatively you could try the stand-alone (I'm assuming you used the plug-in) and see if you have better results to compare.

Yes, the plug-in has problems getting the positions in the excerpts right. I know basically what's causing it, I just have to nail down the fix.

[Ephemerality]

I decided to throw together a GUI version of the stand-alone builder. It requires .Net Framework 4, of course.
It should be a bit easier to use as there are less options and no command-line arguments to deal with. The output is still directed to a text box so you can see what's going on and deal with errors.

Interfaces are not my strong suit, so any suggestions are welcome
The source is available at the same place as the original, https://code.google.com/p/xray-builder/, under the "gui" repo.
I rushed through the readme file so it may not be entirely clear.
Let me know if there are any issues with it. It seemed to work fine on my system, but I haven't had a chance to test it in any other environments yet.

Latest version will be in my most recent post in the thread.

[KingViper]

Originally Posted by Ephemerality

I decided to throw together a GUI version of the stand-alone builder. It requires .Net Framework 4, of course.
It should be a bit easier to use as there are less options and no command-line arguments to deal with. The output is still directed to a text box so you can see what's going on and deal with errors.

Interfaces are not my strong suit, so any suggestions are welcome
The source is available at the same place as the original, https://code.google.com/p/xray-builder/, under the "gui" repo.
I rushed through the readme file so it may not be entirely clear.
Let me know if there are any issues with it. It seemed to work fine on my system, but I haven't had a chance to test it in any other environments yet.

I just attempted to test this. I selected my book, entered the mobi file and got the following error. (This may be because the book didn't have an ASIN properly)

"Running kindleunpack to get metadata"
"Error: Missing metadata."
"Database Name: A_Game_Of_Thrones"
"ASIN:"
"UniqueID: 2785203474"

After the error, if I click on any buttons, the window that should popup does, but cannot be focused on. For instance, I clicked on "Settings" after the error and now the gui acts like it's hung, but really the setting menu is opened behind the main window. I know this, because I can see it in my Aero Peek on the windows toolbar, but clicking on it does not bring focus. I can close it from Aero Peek and then the GUI doesn't act hung anymore, but I still can't click on anything.

Also, I don't know if it's possible, but being able to drag'n'drop a file to the "Book" field would be nice too.

[Ephemerality]

Originally Posted by KingViper

I just attempted to test this. I selected my book, entered the mobi file and got the following error. (This may be because the book didn't have an ASIN properly)

"Running kindleunpack to get metadata"
"Error: Missing metadata."
"Database Name: A_Game_Of_Thrones"
"ASIN:"
"UniqueID: 2785203474"

After the error, if I click on any buttons, the window that should popup does, but cannot be focused on. For instance, I clicked on "Settings" after the error and now the gui acts lie it's hung, but really the setting menu is opened behind the main window. I know this, because I can see it in my Aero Peek on the windows toolbar, but clicking on it does not bring focus. I can close it from Aero Peek and then the GUI doesn't act hung anymore, but I still can't click on anything.

Also, I don't know if it's possible, but being able to drag'n'drop a file to the "Book" field would be nice too.

Here is a new version (1.1), hopefully the focus issue is gone. You can also drag a book onto the form to populate the path into the book field, let me know if it doesn't work.
For the meta data problem you had, you'll need the book to have an ASIN. You can add the book to Calibre and when you populate the meta-data there, it should put an ASIN in automatically. You can also use a metadata editor to put one in there (I like to use the ASIN from Amazon).

Latest version will be in my most recent post in the thread.

[elchamaco]

Thanks Ephemerality for the gui version. I've been trying and it works fine but:
- I miss the offset in settings, i can't assing it so all the bolds terms are bad. The offset can't be calculated?
- Also if you select a name x-ray it's not enabled. But if you press menu then it's avaliable.
- Althought i read most of the forum messages i don't get how to use chapters. It'll be probably usefull to export the chapters like in the alias section
- It'll be fantastic if this option is coded https://www.mobileread.com/forums/sho...&postcount=290. Generate like a ini file or xml with terms, and descriptions so you can later add terms, change descriptions and use this file to generate the x-ray

[Ephemerality]

Originally Posted by elchamaco

Thanks Ephemerality for the gui version. I've been trying and it works fine but:
- I miss the offset in settings, i can't assing it so all the bolds terms are bad. The offset can't be calculated?
- Also if you select a name x-ray it's not enabled. But if you press menu then it's avaliable.
- Althought i read most of the forum messages i don't get how to use chapters. It'll be probably usefull to export the chapters like in the alias section
- It'll be fantastic if this option is coded https://www.mobileread.com/forums/sho...&postcount=290. Generate like a ini file or xml with terms, and descriptions so you can later add terms, change descriptions and use this file to generate the x-ray

I didn't realize that anyone was actually using the offset option! I had it in there just for the sake of people being able to keep their DRM books on their Kindle. I'll add it back in there.

The chapters are exported like the aliases (to the /ext dir) and it (should?) prompts if you want to edit them. The format is "Name|start loc|end loc".

COVER|241|316
TITLE PAGE|316|403
COPYRIGHT|403|2494
DEDICATION|2494|10080

You have to scan through the rawml if you want to customize your chapter locations properly. Currently, it will only auto-find chapters if they're a standard format (mainly the one created via Calibre's conversion function).

I will look into adding the feature you mentioned. I'm thinking it would be a toggle, so either pull from Shelfari or use the saved file, but not both. Unless it pulled from both, then the saved file would take priority for any conflicts...

Added offset support to v1.20:
Latest version here: https://www.mobileread.com/forums/sho...d.php?t=245754

[elchamaco]

Hi Ephemerality, thkx for the new version.

If i don't use the offset in an epub calibre converted to azw3 the bold names are not properly located.

I suppose the problem i 've is the names, i'm using the program in a spanish book so probably as there are no "chapter" text probably the program it's not finding them?. I use heading1 mostly to the beginning of the chapters, also heading2 when there are parts. And the book is epub to azw3 calibre converted. As i see the definition uses locations but i don't know how to calculate them if i want to use from an empty file.


Yes probably the idea it's to generate the x-ray form shelfrai and as in the alias generate a text/ini/xml or something similar with the terms and descriptions. And later if you want you can generate the xray from that file that probably (in my case for sure) you have edited and the program search the locations using this file and not the web, and generate the output with it.
Or if you create one file from scrath use it as source and not the web.
But as you say it has not too much sense to use web and that file.

Great work.

[Ephemerality]

Originally Posted by elchamaco

Hi Ephemerality, thkx for the new version.

If i don't use the offset in an epub calibre converted to azw3 the bold names are not properly located.

I suppose the problem i 've is the names, i'm using the program in a spanish book so probably as there are no "chapter" text probably the program it's not finding them?. I use heading1 mostly to the beginning of the chapters, also heading2 when there are parts. And the book is epub to azw3 calibre converted. As i see the definition uses locations but i don't know how to calculate them if i want to use from an empty file.

That's weird, not sure why the offsets would be off if it's not a DRM book, it would be interesting to take a look at it.

Not sure what could be happening with clicking names, it seems to be working on the ones I have built. Clicking a name or alias brings up the X-Ray for it.
Weird that the chapters wouldn't be located if it were put through Calibre. If you want to PM me the book I can take a look and try to see what's going on.
If you want to make the chapters yourself, you have to save and open the .rawml file and search for where the chapters start; I use Notepad++. Find where the chapter starts, put the cursor there, then highlight everything from there to the beginning of the file. The "sel" entry on the status bar will tell you what location to use.

Edit: I just realized the code to save the rawml to the output directory is missing from the GUI version... new version attached.
You can now dump the info to an XML file, then select it then switch to XML mode. The file is saved with the same name as the book and the XML field is filled in with the path to that file (or you can pick one manually). Documentation for the XML file is included as well, hopefully it's easy enough to deal with. Let me know if there are any issues or suggestions, as it got thrown together pretty quickly.

Latest version here: https://www.mobileread.com/forums/sho...d.php?t=245754

[elchamaco]

Man you're the boss.

It works perfect. Now i can generate from the xml file.
The chapters If i do manually they work.
The offset still necessary, i send you the book so you can test it.

I only find one problem with the xml file origin, if i use special characters the xmray ouput its invalid. For example in spanish we use áéíóúÁÉÍÓÚüÜ probably also with ñÑ, if i put one of those the result is invalid. I see in notepad that a very strange character it's used. For example: "Médico" and the result in the xray is "MxE9dico" If i change back the word to "Médico" editing directly in the xray thhen works. It's a problem from the xml source reading and writing in a strange way.

Now the gui version is almost perfect

[Ephemerality]

Originally Posted by elchamaco

Man you're the boss.

It works perfect. Now i can generate from the xml file.
The chapters If i do manually they work.
The offset still necessary, i send you the book so you can test it.

I only find one problem with the xml file origin, if i use special characters the xmray ouput its invalid. For example in spanish we use áéíóúÁÉÍÓÚüÜ probably also with ñÑ, if i put one of those the result is invalid. I see in notepad that a very strange character it's used. For example: "Médico" and the result in the xray is "MxE9dico" If i change back the word to "Médico" editing directly in the xray thhen works. It's a problem from the xml source reading and writing in a strange way.

Now the gui version is almost perfect

Grr I always forget about the encoding of the files. Aliases, chapters, and the XML should all save/load with accented characters now, let me know if it doesn't work.

I can see why the app isn't finding the chapters, they don't seem to be in the normal format. Possibly because Calibre isn't finding them either due to the lack of 'chapter' keywords that it looks for. It looks for H1 tags that have chapter, prologue, etc in them. I couldn't get it to work with that one either! But with the rawml you should be able to throw them together.

Latest version available here: https://www.mobileread.com/forums/sho...d.php?t=245754

[elchamaco]

Now it works fantastic. Thanks.

I only find a mini bug in the gui, when you generate a xray from a file and there is no shelfari http xray it's not generated. But no problem for me at all.