Collections Manager

[ixtab]

Originally Posted by chipf0rk

Jeez, was that thing sold out? Since when are Amazon so slow?

Nah, not sold out. But obviously the logistics company prefers to let it lay around for 10+ days in some of its dispatch centers instead of shipping it.

Still unsure about whether/when it will eventually arrive - but Amazon has now arranged a replacement shipping. So I assume that I'll eventually get two of them, of which I'll have to send one back

[manawydan]

I think I need help here pls.

I tried to follow the instructions in the first post.
Before that I used jailbreak for my PW2 (5.4).

Before I waste mine and your time - I assume at the moment none of these instructions are of any use since the CM won't work with PW2 vs. 5.4x, right?

But if it would or if it will soon, my problem is this:

1.
Jailbreak - done

2. CM
Followed links to
[Kindle] Unified Application Launcher. KUAL (V 2.2)
UPDATE v 2.2:
followed link again, got the tar, installed the
update_kindlet-dev-20130710-k5-ALL_install.bin

went back to the CM thread, and am stuck with

Kindlet Jailbreak (mirror: update_k5_ALL_PW_ALL_kindlet_jailbreak_install.bin from prerequisites-all-supported-devices.zip in KUAL thread)

There is no such file
update_k5_ALL_PW_ALL_kindlet_jailbreak_install.bin

in the
prerequisites-all-supported-devices.zip

All I have is
update_kindlet-jailbreak-20121029_install.bin

Shall I install this?

I am confused and lost, jumping forward and backward ..

But maybe it wouldn't work anyway and everything starts again when there are updates for vs 5.4 available.

[ixtab]

Originally Posted by manawydan

I think I need help here pls.

I tried to follow the instructions in the first post.
Before that I used jailbreak for my PW2 (5.4).

Before I waste mine and your time - I assume at the moment none of these instructions are of any use since the CM won't work with PW2 vs. 5.4x, right?

But if it would or if it will soon, my problem is this:

1.
Jailbreak - done

2. CM

[...]

You're on the right track , and thanks for the feedback!

I have now modified the initial post to be more up to date. It's not really difficult - CM really just needs a jailbroken device with the dev certificates and the Kindlet Jailbreak installed. But as Amazon is constantly trying to screw us, all of these three are somewhat moving targets.

The current first post is probably as good as it can be right now (BTW, NiLuJe, you deserve a MASSIVE thank you for keeping up with the constant changes! Really, everybody who reads this, give NiLuJe some Karma for his continued effort!)

That said, yes, the best that you will get in CM right now is something along the lines of "Your Kindle is running Firmware 5.4.x, which is not currently supported by Collections Manager". If you get to that message, it means that CM as such can -- in principle -- run, but doesn't know how to handle your particular Firmware. Thanks to Amazon's "fuck independent developers, let's screw them over again and again" policy, CM has to needlessly be adapted to every firmware.

And all of that said: yes, I will be looking into the FW 5.4.x changes. But as mentioned before, I can't promise anything, and I can only do that when I have a PW2 in my hands.

[eureka]

@ixtab, did you look at Lua scripts in /usr/lib/ccat? In the end, stock Java code works with catalog database through these scripts anyway (and so is Collections Manager). Scripts are more readable than Java bytecode and even if they are slightly changed with firmware updates, changes are easlily tracked with simple diff'ing.

/usr/lib/ccat/worker.lua is executed for answering to HTTP requests to http://127.0.0.1:9101/. It recognizes POST requests to predefined endpoints with JSON-encoded POST data. This JSON request is transfromed by Lua script into SQL query to execute. Useful endpoints inculde http://127.0.0.1:9101/query endpoint for selecting from catalog database and http://127.0.0.1:9101/change endpoint for updating catalog database.

Example of HTTP request:

curl http://127.0.0.1:9101/query -d '{"type": "QueryRequest", "maxResults": 2}'

Here is a small hack to display system JSON-encoded requests received by worker.lua in log. Backup /usr/lib/ccat/worker.lua. Add this at the top of /usr/lib/ccat/worker.lua (but after all requires):

local function logged (handler_name, post_data_handler)
  return function (post_data)
    llog.info("ccat_worker", handler_name, post_data, "")
    return post_data_handler(post_data)
  end
end

Then run

sed -ie 's/^\(_G.\+=\s*\)\([^.]\+\)\(\..\+\)$/\1logged("\2", \2\3)/g' /usr/lib/ccat/worker.lua

Then mntroot ro, restart framework and showlog -f | grep ccat_worker

Format of log entries is 'ccat_worker:<endpoint>:<JSON-encoded data>:'

Here are sample log entries recorded right after invoking of Collections Manager from Home screen:

Spoiler:

cvm[4642]: I ccat_worker:query:{"filter":{"And":[{"ValueIn":{"choices":["\/mnt\/us\/documents\/CollectionsManager.azw2"],"path":"location"}},{"Not":{"ValueIn":{"choices":[true],"path":"isArchived"}}}]},"type":"QueryRequest","maxResults":1,"sortOrder":[{"order":"ascending","path":"titles[0].collation"},{"order":"ascending","path":"credits[0].name.collation"},{"order":"descending","path":"publicationDate"}],"startIndex":0,"id":21,"resultType":"fast"}:
cvm[4642]: I ccat_worker:change:{"commands":[{"update":{"uuid":"78a758fa-3e2f-4081-b4bd-4c01ede9dd35","displayTags":["DEV"],"lastAccess":1386802717}}],"type":"ChangeRequest","id":5}:
cvm[4642]: I ccat_worker:query:{"filter":{"And":[{"ValueIn":{"choices":[true],"path":"isVisibleInHome"}},{"Not":{"ValueIn":{"choices":[true],"path":"isDownloading"}}},{"Equals":{"value":"Entry:Item:Notice","path":"type"}}]},"type":"QueryRequest","maxResults":40,"sortOrder":[{"order":"descending","path":"lastAccess"},{"order":"ascending","path":"titles[0].collation"}],"startIndex":0,"id":22,"resultType":"fast"}:
cvm[4642]: I ccat_worker:query:{"filter":{"And":[{"ValueIn":{"choices":[true],"path":"isVisibleInHome"}},{"Not":{"ValueIn":{"choices":[true],"path":"isArchived"}}},{"Not":{"ValueIn":{"choices":[true],"path":"isDownloading"}}},{"ValueIn":{"choices":[true],"path":"isLatestItem"}},{"Not":{"ValueIn":{"choices":["Entry:Item:AI","Entry:Item:Dictionary","Entry:Item:Notice"],"path":"type"}}}]},"type":"QueryRequest","maxResults":40,"sortOrder":[{"order":"descending","path":"lastAccess"},{"order":"ascending","path":"titles[0].collation"}],"startIndex":0,"id":23,"resultType":"fast"}:
cvm[4642]: I ccat_worker:query:{"filter":{"And":[{"StartsWith":{"prefix":"","path":"titles[0].display"}},{"And":[{"Not":{"ValueIn":{"choices":[true],"path":"isArchived"}}},{"Not":{"ValueIn":{"choices":[true],"path":"isDownloading"}}},{"Not":{"Equals":{"value":"application\/x-kindle-acx","path":"mimeType"}}},{"Or":[{"Equals":{"value":"Collection","path":"type"}},{"StartsWith":{"prefix":"Entry:Item","path":"type"}}]}]}]},"type":"QueryRequest","maxResults":5,"sortOrder":[{"order":"ascending","path":"titles[0].display"},{"order":"ascending","path":"credits[0].name.collation"},{"order":"descending","path":"publicationDate"}],"startIndex":0,"id":24,"resultType":"full"}:
cvm[4642]: I ccat_worker:query:{"filter":{"And":[{"StartsWith":{"prefix":"","path":"titles[0].display"}},{"And":[{"Not":{"ValueIn":{"choices":[true],"path":"isArchived"}}},{"Not":{"ValueIn":{"choices":[true],"path":"isDownloading"}}},{"Not":{"Equals":{"value":"application\/x-kindle-acx","path":"mimeType"}}},{"Or":[{"Equals":{"value":"Collection","path":"type"}},{"StartsWith":{"prefix":"Entry:Item","path":"type"}}]}]}]},"type":"QueryRequest","maxResults":5,"sortOrder":[{"order":"ascending","path":"titles[0].display"},{"order":"ascending","path":"credits[0].name.collation"},{"order":"descending","path":"publicationDate"}],"startIndex":0,"id":25,"resultType":"full"}:
cvm[4642]: I ccat_worker:query:{"filter":{"And":[{"StartsWith":{"prefix":"","path":"titles[0].display"}},{"And":[{"Not":{"ValueIn":{"choices":[true],"path":"isArchived"}}},{"Not":{"ValueIn":{"choices":[true],"path":"isDownloading"}}},{"Not":{"Equals":{"value":"application\/x-kindle-acx","path":"mimeType"}}},{"Or":[{"Equals":{"value":"Collection","path":"type"}},{"StartsWith":{"prefix":"Entry:Item","path":"type"}}]}]}]},"type":"QueryRequest","maxResults":5,"sortOrder":[{"order":"ascending","path":"titles[0].display"},{"order":"ascending","path":"credits[0].name.collation"},{"order":"descending","path":"publicationDate"}],"startIndex":0,"id":26,"resultType":"full"}:
cvm[4642]: I ccat_worker:query:{"filter":{"And":[{"StartsWith":{"prefix":"","path":"titles[0].display"}},{"And":[{"Not":{"ValueIn":{"choices":[true],"path":"isArchived"}}},{"Not":{"ValueIn":{"choices":[true],"path":"isDownloading"}}},{"Not":{"Equals":{"value":"application\/x-kindle-acx","path":"mimeType"}}},{"Or":[{"Equals":{"value":"Collection","path":"type"}},{"StartsWith":{"prefix":"Entry:Item","path":"type"}}]}]}]},"type":"QueryRequest","maxResults":5,"sortOrder":[{"order":"ascending","path":"titles[0].display"},{"order":"ascending","path":"credits[0].name.collation"},{"order":"descending","path":"publicationDate"}],"startIndex":0,"id":27,"resultType":"full"}:

This is not my discovery, there was Kindle_Touch_Hacking#Collections and some old threads mentioned this way too.

[NiLuJe]

@ixtab: You're not doing so bad yourself .

@eureka: Ooh, shiny!

[knc1]

Originally Posted by ixtab

Thanks to Amazon's "fuck independent developers, let's screw them over again and again" policy, CM has to needlessly be adapted to every firmware.

That is indeed getting more than a bit wearing to put up with.

My own answer to the problem:
"Find a manufacturer willing to cooperate with owner's after-market development - then pick the hobby device."

The progress to date (and why readers are seeing less of twobob and myself here for awhile):

The good news:
A manufacturer has been found, and is convinced this type of relationship with the owners of their products is an advantage to themselves. (Yeah! Big win!)

Work is underway to make this idea a reality, with the active support of the company.

The bad (?) news:
This manufacturer does not make e-ink display devices.
(What can I say? I tried, but none of the e-ink device manufacturers I contacted want to be first.)

When will the world know more about this?
Probably sometime next spring, it's a big project.

Just what is it all about:
To keep the open source software Open (even after modification for the device) ;
To keep company proprietary software Private ;

And the big, catch-22 sounding goal:

Keeping the device and firmware security system in-place that protects against malicious code at the same time as meeting the GPLv3 requirement that the owner can install code of their own choosing (including updates to the GPLv3 items).

Leaving firms that are stuck with aging GPLv2 code due to their corporate policies in the dust of history.

One possible side-effect - -
The death of the Amazon Kindle Fire product line.
But hey - it was Amazon's own policies that brought this all about.
(Talk about shooting one's self in the foot. )

[chipf0rk]

But... isn't that part of the fun for many devs here, finding out a way to give back an even bigger fuck you to Amazon by cracking their "protection" from anything non-Amazon?

(well I guess mostly for the JB and system-"intrusive" hack developers, not for the rest...)


But in any case, that sounds pretty awesome, knc1. A company that is interested in giving their customers full access to their product's potential? Shall they be praised!

[manawydan]

Originally Posted by ixtab

That said, yes, the best that you will get in CM right now is something along the lines of "Your Kindle is running Firmware 5.4.x, which is not currently supported by Collections Manager". If you get to that message, it means that CM as such can -- in principle -- run, but doesn't know how to handle your particular Firmware. Thanks to Amazon's "fuck independent developers, let's screw them over again and again" policy, CM has to needlessly be adapted to every firmware.

Thank you and thank you also for constantly working on this.

Question: would it be of use (and possible) to downgrade the firmware then?

The thought alone makes me sweat but I really really miss the easy working with Calibre and Kindle Collections.

Originally Posted by knc1

The bad (?) news:
This manufacturer does not make e-ink display devices.
(What can I say? I tried, but none of the e-ink device manufacturers I contacted want to be first.)

Yes, that's bad indeed. :/

If Amazon would listen to users and improve the Kindle users would be happy, Amazon would be happy (because sales might increase) and only developers will be bored.

[knc1]

Originally Posted by manawydan

Question: would it be of use (and possible) to downgrade the firmware then?

No, not on the Kindle Paperwhite Gen(2).

Different hardware inside, different CPU, different GPU.
About the only thing the same between the PW2 and the earlier devices is the "Kindle" name.

[gweminence]

Did you get your pw2 yet ixtab?

[levante]

Originally Posted by gweminence

Did you get your pw2 yet ixtab?

A kindle without ixtab's JBPatch is less kindle.

[ixtab]

Originally Posted by gweminence

Did you get your pw2 yet ixtab?

Nope. By now, I don't expect anything to happen anytime soon. So I guess that I will only get to start looking into the FW in mid-January...

[cryzed]

Shit... I hope you get it earlier, that really sucks. What about the one that got stuck somewhere? Did it just magically disappear? Because if I recall correctly, you said that you had already contacted Amazon and they were planning on shipping another.

I also hope someone still has a 5.4.0 version, by the time you have your device and get around to developing the rescue pack...

[ixtab]

Originally Posted by cryzed

Shit... I hope you get it earlier, that really sucks. What about the one that got stuck somewhere? Did it just magically disappear? Because if I recall correctly, you said that you had already contacted Amazon and they were planning on shipping another.

Well, seems like they are now *both* stuck somewhere. Since Amazon was smart enough to let the same logistics company handle both shipments (the original, and the "replacement" one), the status for the first one was "Your shipment has been flagged for review of the contents prior to export" for almost two weeks - and that's exactly where the second one is now stuck too.

In the mean time, the first one's status is now "Exported and in transit to country of destination" - and has been so for *another* almost two weeks.

I don't know if they transport the Kindles by bicycle, or fly them 10 times around the earth, or what.

[chipf0rk]

It's a conspiracy, they don't want you to make a new Collections Manager version.