Tagging files to show up in the "Docs"-lists

[katzz]

Hello there,

before asking my question, I humbly ask you to read these posts:
Number 1
Number 2

As you just (hopefully) read, I manage my content locally and actually do want to use the "Docs"-category on my PW2. Simply because it makes managing collections on the device easier.

Putting everything in the "Books"-category, switching to a collection and then use "Add/Remove Items", will show me a list where sideloaded ebooks and cloud-stored ebooks are mixed up, without any possibility to filter them.

Putting stuff into the "Docs"-category will show me a list with sideloaded ebooks only.

Doing this with Calibre and mobi-7-files is quite easy, since it has an output-option called "Personal Doc Tag". This doesn't even alter the original file in the library, since it just tags the copy transferred to the device.

__________________________________________________ ________________________________________________

But what about KF8-formatted AZW3-files? How to I force them to appear in the "Docs"-category? Reading this post looks like I have to alter the original AZW3-file.

An example would be a fixed-layout epub or similar. Some kind of file that is not supported by the PW2 natively, but would be fine using the KF8-formatting.

This is all giving me a headache.

[NiLuJe]

I vaguely remember thinking that the KF8 (AZW3) output plugin honours the PDOC setting, but I might be wrong. I'll check.

Do keep in mind that on FW 5.X, a PDOC cdeType necessarily means ending up with the "Personal" banner.

EDIT: Yup, IIRC, the setting will be honoured when the metadata is updated during the "Send To Device" process.

[katzz]

Something strange just happened. Yesterday I managed to put several books into the "Docs"-category and they didn't even show a "Personal"-banner: http://imgur.com/a/ytAaw

Today the PDOC-setting is completely ignored. I can't even put mobi-files in the "Docs"-category any longer. I'm not sure if I yesterday changed the output-settings from mobi (old) to mobi (both) and back afterwards, I don't remember.

Fact is, as you can see in the screenshots above, that I managed to put files in the "Docs"-category, without a banner, on firmware 5.4.3.2 and today it doesn't even work with Non-KF8-mobi-files.

[NiLuJe]

You're showing a *Collection* in that screenshot. I'd wager that the books inside are indeed tagged EBOK, except the ones sent by Amazon.

How the Homescreen shows collection has somehow always been confusing to some, so it's not unheard of .

(Basically, Collections, Clippings, Kindlets (apps) & Booklets (super-apps) are special types of items, and may or may not appear as you would expect depending on which filter you're using on the Homescreen).

To recap how Calibre's PDOC feature work: in the mobi output settings, it asks you for the name of a tag. Any books containing that tag will be tagged as PDOC instead of EBOK during the "Send To Device" process.

As usual, when updating books (and not uploading new ones), keep in mind that the Kindle DB is tricky, I recommend deleting the books from the Kindle's UI first, to avoid weirdness.

[katzz]

#Update

Just found this thread and now I'll give it another try.

_________________________________________________
Original Post

Originally Posted by NiLuJe

You're showing a *Collection* in that screenshot. I'd wager that the books inside are indeed tagged EBOK, except the ones sent by Amazon.

Look at the first screenshot. It says: Docs (3).

These books have been in the Docs-category.

Originally Posted by NiLuJe

How the Homescreen shows collection has somehow always been confusing to some, so it's not unheard of .

(Basically, Collections, Clippings, Kindlets (apps) & Booklets (super-apps) are special types of items, and may or may not appear as you would expect depending on which filter you're using on the Homescreen).

The second screenshot should just show, that no "personal"-badge was applied to the sideloaded book.

Originally Posted by NiLuJe

To recap how Calibre's PDOC feature work: in the mobi output settings, it asks you for the name of a tag. Any books containing that tag will be tagged as PDOC instead of EBOK during the "Send To Device" process.

This worked yesterday, now it doesn't.

This post confuses me even more.

What matters is the internal metadata tag. The advise to use [EBOK] as the Personal Doc Tag is wrong, in the sense that whatever that tag is set to the output is PDOC if that tag is present in the ebook entry. So [EBOK] only works because you don't have [EBOK] as a tag. The best way to get an ebook into the Books app is to set Preferences => Conversion => Output Options => Mobi Output => Personal Doc Tag to blank (empty) and do a MOBI to MOBI conversion.

I just kept [PDOC] and did a MOBI to MOBI - conversion, still no result.

[NiLuJe]

Yup, you got the same explanation as I just gave from the posts you linked (from Kovid, of course ), and summarised by wallcraft in the second link.

As I said, the fact that you filter for "Docs" doesn't directly impact collections, which is why you see the collection on the Homescreen. And inside that collection, no more filtering on the cdeType is done, which is why you see both EBOK & PDOC files (and yeah, you might have side loaded them, but I'm guessing they have an EBOK cdeType, so you'll have to check how you proceeded to set up Calibre if you intended for them to end up as PDOC ).

The Homescreen and the "Inside-a-collection" screen behave completely differently, and have different filtering capabilities/set of filters.

I agree that the usual given examples of using "[PDOC]" can be confusing. Personally, I prefix "special" tags (those used to make automagic stuff happens, like PDOC or Kindle Collections) with a "+", so that output setting is set to "+Doc" for me. If I ever tag a book with "+Doc" in Calibre, and then send it *over USB, via Calibre's "Send To Device"*, it will end up with a PDOC cdeType (and thus fall in the "Docs" filter of the *Homescreen*, and get the "Personal" banner on FW 5.X).

You don't need to do a conversion, and it won't actually change a thing, the change is applied to the metadata of the file during the "Send To Device" process (or a Save To Disk). (i.e. it only applies to the on-device file, or the exported file).

(And, again, if you keep testing stuff on the exact same book, keep the last paragraph of my previous post in mind: the Kindle catalog can sometime be tricky in its acceptance of changes to an existant file).

EDIT: If my mumbling about cdeType isn't clear, that's because I keep using the 'internal' naming of the file format itself, because the issue is so often mistreated/misunderstood that I prefer using 'exact' words, which happen to be keywords you can look for in the wiki, Calibre's source code, and the code of the various tools dealing with the Mobi/KF8 format, in particular its internal metadata.

So the cdeType field is what that output setting is controlling in Calibre, and what the Kindle's OS actually checks when it decides how to filter a book, and it if it should get a special banner or not.
EBOK is the default: this is both what you get when buying books from Amazon, and what you get with Calibre (when sent over USB from Calibre, with default settings).
PDOC is what you get when you get stuff via Amazon's various Personal documents-related services, and also what you get out of KindleGen. You can also get this with Calibre, if the aforementioned output setting is used correctly.

[katzz]

Originally Posted by NiLuJe

As I said, the fact that you filter for "Docs" doesn't impact collections, which is why you see the collection on the Homescreen. And inside that collection, no more filtering on the cdeType is done, which is why you see both EBOK & PDOC files (and yeah, you might have side loaded them, but I'm guessing they have an EBOK cdeType, so you'll have to check how you proceeded to set up Calibre if you intended for them to end up as PDOC ).

I don't know how to explain myself any further. Look at this screenshot again and then at the second one.

The whole "Vampire Chronicles"-collection was listed under the "Docs"-category. When clicking the collection, no personal-tags have been visible on the ebooks themselves.


Just seconds ago I sent the files over again, following this advice:

Originally Posted by kovidgoyal

you have [PDOC] in the setting *and* [PDOC] as a tag for the book you want marked as PDOC?

and now look at what happened.

As you can see, I managed to but them into "Docs" again, but now there's the [personal]-badge visible, which hasn't been there before.

Originally Posted by NiLuJe

I agree that the given examples of using "[PDOC]" can be confusing. Personally, I prefix "special" tags (those used to make automagic stuff happens, like PDOC or Kindle Collections) with a "+", so that output setting is set to "+Doc" for me. If I ever tag a book with "+Doc" in Calibre, and then send it *over USB, via Calibre's "Send To Device"*, it will end up with a PDOC cdeType (and thus fall in the "Docs" filter of the Homescreen, and get the "Personal" banner on FW 5.X).

You don't need to do a conversion, and it won't actually change a thing, the change is applied to the metadata of the file during the "Send To Device" process (or a Save To Disk). (i.e. it only applies to the on-device file, or the exported file).

I actually already knew this, but was distracted for a moment. Like you said, the tags are just applied to the copy that is stored on the device. Not the file in my library.

[NiLuJe]

Yup, one of those screen is the Homescreen and the other is the (Inside-a-)Collection screen, which, for all intents and purposes, and as weird as it may sound, are two different beasts altogether (see how, inside the collection, you're not filtering by "Docs", but sorting by "Title"? Although mostly irrelevant to your puzzlement, it proves that they behave differently).

As far as I can see, everything is behaving properly, and what happened yesterday is just that you had *ONE* file in that collection 'tagged' (more accurately, with the cdeType set to) PDOC, while the others were in fact EBOK.
That one PDOC most likely made the collection appear as *containing* (but not *completely composed of*) PDOCs by the filter "Docs" (that, or the "Docs" filter doesn't care at all about collections, and you see collections anyway, no matter what they contain: you can answer that by telling me if you happen to have other collections on your device ).

And yeah, a file with a PDOC cdeType automatically gains an unsightly "Personal" banner on FW 5.X.

If you want to replicate the behaviour of your first screenshots, just put a bogus PDOC book in the same collection as a bunch of EBOK books.

[katzz]

Originally Posted by NiLuJe

Yup, one of those screen in the Homescreen and the other is the (Inside-a-)Collection screen, which, for all intents and purpose, and as weird as it may sound, are two different beasts altogether.

As far as I can see, everything is behaving properly, and what happened yesterday is just that you had *ONE* file in that collection tagged PDOC, while the others were in fact EBOK.
That one PDOC most likely made the collection appear as *containing* (but not *completely composed of*) PDOCs by the filter "Docs" (that, or the "Docs" filter doesn't care at all about collections, and you see collections anyway, no matter what they contain).

There is still one detail that doesn't add up, but I can live with that. You're explanation is perfectly valid and 100% correct since I didn't add a "PDOC"-metatag yesterday. Of course the output-option couldn't work its magic.

Glad I'm not messing with the Amazon-cloud yet.

Thanks for taking your time and helping me out!

If I ever find a way to contribute to these forums, I will surely do.

PS: AZW3-files do work as well, as you mentioned above!

[NiLuJe]

@katzz: It *should* use the exact same bit of metadata handling code, and inherit the setting from the Mobi output settings, but I admit it's been a while since I actually used that exact tweak, so do tell me if you hit a snag, and I'll try on my end .

[katzz]

It does work indeed, just tried it with different input-files.

[NiLuJe]

Cool, glad we cleared this up . (As I'm sure you realised when looking stuff up, you're not the first to be confused by this ).