Clara HD Tips for a Beginner

[xtmpq]

Hey guys, new to eReaders in general but I like tinkering stuff. Just got the Clara and want to use it to the fullest that I can but I need your help. I have a bunch of questions if you can answer them I will be thankful.

1. I used This to use my Kobo without creating an account. Is there any harm in it?

2. I used This to find out how to remove the Walmart logo. Can I just straight just remove the Affiliate.conf file?

3. What is the most optimal way that I can update my Kobo without connecting it to the wifi?

4. What are the best Kobo plugins that I should use with Calibre

5. What are the best hacks for Kobo. I know that I am gonna use One-Click Install Packages. I also know of Kobopatches but I don't know what to do with them.

6. I think I saw somewhere that a Calibre plugin could backup and update Kobo devices. Am I wrong. If so how to properly backup your kobo device and update their firmware.

Thanks

[xtmpq]

Also can anyone tell me how to Install a dictionary without connecting the device to Wifi. Thanks

[geek1011]

1. Things may break unexpectedly, but the offline features should work fine.
2. No, it'll just be recreated again.
3. Connect over USB and use my firmware page.
4. That's a question for someone else to answer, as I don't use Calibre.
5. I made NickelMenu, which is included in the OCP packages. You'll probably be interested in some of the other features like invert screen, usb connection (since you say you won't be using WiFi much), screen orientation, and so on. You might also be interested in NickelSeries if you read a lot of fiction in series or if you read a lot of nonfiction with subtitles. For the patches, you just read through the files in src and decide which ones you might want to enable. A popular set of patches are the header/footer ones and the line spacing ones. For you, I'd also recommend the "Always show confirmation dialog before upgrading" and "Change Browse Kobo home screen link target - *" patches.
6. I don't use Calibre, but IIRC, it refers to syncing state with the Kobo.

Dictionaries: You can download them manually, but I'll need to know your firmware version to give more details.

[xtmpq]

Originally Posted by geek1011

1. Things may break unexpectedly, but the offline features should work fine.
2. No, it'll just be recreated again.
3. Connect over USB and use my firmware page.
4. That's a question for someone else to answer, as I don't use Calibre.
5. I made NickelMenu, which is included in the OCP packages. You'll probably be interested in some of the other features like invert screen, usb connection (since you say you won't be using WiFi much), screen orientation, and so on. You might also be interested in NickelSeries if you read a lot of fiction in series or if you read a lot of nonfiction with subtitles. For the patches, you just read through the files in src and decide which ones you might want to enable. A popular set of patches are the header/footer ones and the line spacing ones. For you, I'd also recommend the "Always show confirmation dialog before upgrading" and "Change Browse Kobo home screen link target - *" patches.
6. I don't use Calibre, but IIRC, it refers to syncing state with the Kobo.

Dictionaries: You can download them manually, but I'll need to know your firmware version to give more details.

Thank you so much for answering. From searching the forum I found that you're a veteran so I really appreciate you helping me.

1. So I'm gonna stick with the offline. Thank you
2. For some reason it went away automatically. I don't see it when powering off or when device is sleeping
3. I just did
4. No worries. I am thinking of ditching calibre and using your website for firmware plus kepubify
5. Oh wow. Thanks for the answer. I couldn't figure out how to customized NickleMenu correctly but making a config file and adding a single orientation action taught me how to do it. I don't understand what NickelSeries does? Lastly I am just starting so I might stay away from Kobo Patches. Just stick to NickelMenu which is already too Powerful (and series if I understand what it does correctly).
6. I found out how to download the latest dictionary for latest firmware using this link.

Thankyou so much for the reponse. One last question, should I backup my sdcard image and how should I go about doing that?

[ZodWallop]

Originally Posted by xtmpq

2. I used This to find out how to remove the Walmart logo. Can I just straight just remove the Affiliate.conf file?

As mentioned, deleting will just prompt a new on. Just change the affiliate and never touch it again. You should be fine.

4. What are the best Kobo plugins that I should use with Calibre

There's not so much a best one as there is the ones you like.

I use DeDRM of course.

I also use KoboTouchExtended. I let it create collections based on tags, upload black and white covers, update the metadata on the device and convert ePub books to kepub when they are sent from Calibre to the Kobo.

Kobo Source is nice. It adds Kobo as a source when Calibre searches for metadata and covers. That is not specifically for Kob, it can benefit any user.

There's also Kobo Utilities which saves reading location from Kobo to Calibre.

Lastly, I'd recommend JSWolf's Better Hyphenation Dictionary. I don't like Kobo's default hyphenation much.

5. What are the best hacks for Kobo. I know that I am gonna use One-Click Install Packages. I also know of Kobopatches but I don't know what to do with them.

Again, it's not about what's best. It's what you might use. For instance, I (and most other patchers) turn on advanced features for side-loaded fonts. It's nice to have. But to be honest, I never use those advanced features. So really I just do it because I can.

I'd recommend spending time reading on your Kobo and figure out what changes you would like. Smaller header and footer? Larger cover images?

At any rate, here's a link for instructions on patching firmware.

[geek1011]

Originally Posted by xtmpq

Thank you so much for answering. From searching the forum I found that you're a veteran so I really appreciate you helping me.

You're welcome .

4. No worries. I am thinking of ditching calibre and using your website for firmware plus kepubify

If you have more than one device or if you already use Calibre extensively, I'd recommend using it (you can still mass-preconvert your books with kepubify; there is quite a bit about this on MR and GitHub). Otherwise, kepubify would the best option. If you use kepubify, consider also using covergen (it comes with kepubify) to pre-generate the covers, which will improve performance of the library browser.

5. Oh wow. Thanks for the answer. I couldn't figure out how to customized NickleMenu correctly but making a config file and adding a single orientation action taught me how to do it. I don't understand what NickelSeries does? Lastly I am just starting so I might stay away from Kobo Patches. Just stick to NickelMenu which is already too Powerful (and series if I understand what it does correctly).

Regarding NickelMenu, you can also look at my own configuration on GitHub.

NickelSeries essentially adds built-in support for series metadata and subtitles on sideloaded books. It works across firmware versions and unlike seriesmeta and calibre, doesn't require connecting the device twice.

For the patches, I'd recommend seeing how you like the firmware as-is first. You can always enable them later if you want, and they're a bit more involved to set up and maintain than NickelMenu, NickelSeries, kepubify, and dictutil.

6. I found out how to download the latest dictionary for latest firmware using this link.

That's correct.

Thankyou so much for the reponse. One last question, should I backup my sdcard image and how should I go about doing that?

You can either open the case (start from the side, but be careful about the bottom snaps) and make an image or you can do it over SSH/telnet (put a minimal rootfs in a tmpfs mount, pivot_root, then make an image over the network). But why do you want to do that?

[xtmpq]

I will look into your config of Nickelmenu. Also will play around with Nickelseries to see how it works. Thanks.

I wanted to backup so that if something happens to my sdcard or kobo due to my foolishness, I can recover it

[xtmpq]

Is this the config you're talking about?

[xtmpq]

Also is there anyway to convert the v3 of dicthtml.zip to stardict?

[JSWolf]

Originally Posted by xtmpq

Hey guys, new to eReaders in general but I like tinkering stuff. Just got the Clara and want to use it to the fullest that I can but I need your help. I have a bunch of questions if you can answer them I will be thankful.

1. I used This to use my Kobo without creating an account. Is there any harm in it?

2. I used This to find out how to remove the Walmart logo. Can I just straight just remove the Affiliate.conf file?

3. What is the most optimal way that I can update my Kobo without connecting it to the wifi?

4. What are the best Kobo plugins that I should use with Calibre

5. What are the best hacks for Kobo. I know that I am gonna use One-Click Install Packages. I also know of Kobopatches but I don't know what to do with them.

6. I think I saw somewhere that a Calibre plugin could backup and update Kobo devices. Am I wrong. If so how to properly backup your kobo device and update their firmware.

Thanks

1. Yes, there is harm. In the past, Kobo has changed something in the firmware that prevented the current fake registration from working. That meant you had to lose everything and start over. Also, you cannot sync eBooks from Kobo or use Overdrive on your Kobo. There are a lot of negatives and no positives. I've had an account with Kobo for over 6 years. My H2O has been registered to my account for almost 6 years. I have not once had any problem. So I suggest you sign out of your Kobo, create a real account and use that to register.

2. Do not delete affiliate.conf. Edit the contents and change it to Kobo instead of Walmart.

3. Check Geek1011's Kobo firmware site and if there is a new firmware, download the version for your Kobo and manually install it. To manually install, you unzip the contents of the firmware ZIP to the .kobo directory and when you eject, the Kobo will install the firmware.

4. If you want to remove the DRM, you'll need to run Calibre 4.23 because the DeDRM plugin is not yet compatible with Calibre 5.x. Other plugins that are good to run are Kobo Utilities, Reading List, Find Duplicates, Quality Check, Modify ePub, EpubCheck, Diaps Editor Toolbag, Amazon Hi-Res Covers, Kobo Books, Annotations, Count Pages, EpubSplit, Favorites Menu, and ScrambleEbook. There may be others you'll want. Check the plugins forum. Oh and you'll want to decide in you want to read KePub or ePub. I suggest installing all of the KePub plugin and the KoboTouchExtended. I read mostly ePub because I prefer the better typesetting that you get with ePub.

5. NickelMenu and the patches are the best patches.

6. You backup your Kobo database using the Kobo Utilities plugin.

[JSWolf]

Originally Posted by xtmpq

Thank you so much for answering. From searching the forum I found that you're a veteran so I really appreciate you helping me.

1. So I'm gonna stick with the offline. Thank you
4. No worries. I am thinking of ditching calibre and using your website for firmware plus kepubify

Thankyou so much for the reponse. One last question, should I backup my sdcard image and how should I go about doing that?

1. Really bad idea. Someday you could end up having to start from scratch.
4. Ditching Calibre will lose you so much automation and you'll lose the editor, viewer, easy eBook management, and a lot of other really useful features (including being able to remove DRM). As for kepubify, there's an issue with hyphenation with KePub. That has been fixed in the KoboTouchExtended driver for Calibre.

[geek1011]

Originally Posted by JSWolf

1. Really bad idea. Someday you could end up having to start from scratch.

To be more specific, JSWolf is referring to the possibility that an update could change how the users table is handled in a way incompatible with the hack, causing a sign-out at best and an automatic full reset at worst.

4. Ditching Calibre will lose you so much automation and you'll lose the editor, viewer, easy eBook management, and a lot of other really useful features (including being able to remove DRM).

To be fair, I automate everything without Calibre (and prefer to edit my ebooks from the command line). DRM removal doesn't require Calibre (and IIRC, the tools won't even work with py3 in the latest version of Calibre). It's really up to preference.

As for kepubify, there's an issue with hyphenation with KePub. That has been fixed in the KoboTouchExtended driver for Calibre.

And in kepubify v3.1.6.

[geek1011]

Originally Posted by xtmpq

Also is there anyway to convert the v3 of dicthtml.zip to stardict?

Firstly, the Kobo ones are encrypted. I will not discuss that topic. Secondly, they're not intended to be used outside Nickel.

If you did happen to want to convert an unencrypted dicthtml to another format, you can use dictutil to decompile it into a dictfile, then PyGlossary to convert that into another format. I haven't fully implemented v3 support yet, but this should work fine in general.

A bit off topic: If you are clever enough, the ones with come with Microsoft Edge (the older, non-chromium, one) are also Oxford, and they can be extracted (they aren't encrypted or protected). They are downloaded to AppData/ProgramData as a sqlite DB, which consists of an index of words to a chunk+shard+offset and a table of chunks and binary data. The binary data consists of a list of compressed JSON separated by headers containing the shard index, size, and some extra junk. Here's some of the code I put together a while ago:

Spoiler:

package main 

import (
	"bytes"
	"compress/gzip"
	"database/sql"
	"encoding/json"
	"errors"
	"fmt"
	"io/ioutil"
	"os"
	
	_ "github.com/mattn/go-sqlite3"
) 

func queryWord(db *sql.DB, word string) (shardID, idx int, found bool, err error) {
	row := db.QueryRow("SELECT JsonShardID, JsonIndex FROM WordLookup WHERE Name == ?", word)
	if err = row.Scan(&shardID, &idx); err != nil {
		if err == sql.ErrNoRows {
			return 0, 0, false, err
		}
		return 0, 0, false, err
	}
	return shardID, idx, true, nil
}

func listWords(db *sql.DB) ([]string, error) {
	rows, err := db.Query("SELECT Name FROM WordLookup")
	if err != nil {
		return nil, err
	}
	defer rows.Close()

	var words []string
	for rows.Next() {
		var word string
		if err := rows.Scan(&word); err != nil {
			return nil, err
		}
		words = append(words, word)
	}
	if err = rows.Err(); err != nil {
		return nil, err
	}
	return words, nil
}

func getShard(db *sql.DB, shardID int, shardCache map[int][]byte) ([]byte, error) {
	if shardCache != nil {
		if buf, ok := shardCache[shardID]; ok {
			return buf[:], nil
		}
	}

	var buf []byte
	row := db.QueryRow("SELECT ShardData FROM Shard WHERE ID == ?", shardID)
	if err := row.Scan(&buf); err != nil {
		if err == sql.ErrNoRows {
			return nil, err
		}
		return nil, err
	}

	r, err := gzip.NewReader(bytes.NewReader(buf))
	if err != nil {
		return nil, err
	}

	nbuf, err := ioutil.ReadAll(r)
	if err != nil {
		return nil, err
	}

	if shardCache != nil {
		shardCache[shardID] = nbuf[:]
	}

	return nbuf, nil
}

// splitShard splits a shard into its segments.
func splitShard(shard []byte) (map[int][]byte, error) {
	hdr := []byte{0x00, 0x01, 0x00, 0x00, 0x00, 0xFF, 0xFF, 0xFF, 0xFF, 0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x11, 0x01, 0x00, 0x00, 0x00}
	if !bytes.HasPrefix(shard, hdr) {
		return nil, errors.New("unknown shard header")
	}

	if !bytes.HasSuffix(shard, []byte{0x0B}) {
		return nil, errors.New("unexpected shard end")
	}

	segments := map[int][]byte{}
	// 06 ## 00 00 00 ## ## __...
	//    |           |     ^ content (should start and end with { and })
	//    ^ index     ^ some sort of size
	spl := bytes.Split(shard[:len(shard)-1], []byte{0x06})[1:]
	for i, segment := range spl {
		if len(segment) <= 6 {
			// handle 0x06 in header edge case
			spl[i+1] = append(segment, append([]byte{0x06}, spl[i+1]...)...)
			continue
		}
		if segment[1] != 0x00 || segment[2] != 0x00 || segment[3] != 0x00 {
			return nil, errors.New("unhandled non-zero bytes in shard header")
		}

		start := 6
		if segment[6] != '{' || segment[len(segment)-1] != '}' {
			if segment[7] == '{' {
				start = 7
			} else {
				fmt.Fprintf(os.Stderr, "%#v\n", string(segment)[:20])
				fmt.Fprintf(os.Stderr, "%#v\n", string(segment)[6])
				fmt.Fprintf(os.Stderr, "%#v\n", segment[:20])
				panic("well, it seems like my guess at the header was wrong: invalid json afterwards")
			}
		}

		idx := int(segment[0]) - 2
		if idx < 0 {
			panic("well, it seems like my guess at the header was wrong: the index does not always start at 2")
		}
		if _, seen := segments[idx]; seen {
			panic("well, it seems like my guess at the header was wrong: the second byte is NOT the index")
		}
		segments[idx] = segment[start:]
	}

	return segments, nil
}

// extractIdx extracts the indexed segment from the shard.
//
// Test with:
// - ruffed: last word in file
// - salinency: problematic
// - curvet: slightly different format
// - test
// - example
// - edge
func extractIdx(shard []byte, idx int) ([]byte, error) {
	segments, err := splitShard(shard)
	if err != nil {
		return nil, err
	}
	segment, ok := segments[idx]
	if !ok {
		return nil, errors.New("no such segment")
	}
	return segment, nil
}

type Word struct {
	Name               string
	PhoneticName       string
	Pronunciation      string
	PronunciationAudio struct {
		ContentURL string
	}
	MeaningGroups []struct {
		Meanings []struct {
			// First is the main meaning, next are submeanings
			RichDefinitions []struct {
				Fragments []struct {
					Text string // Is there EVER more than one fragment!?!
				}
				Examples []string
				Synonyms []struct {
					Name string
				}
			}
		}
		WordForms []struct {
			Form string
			Word struct {
				Name string
			}
		}
		PartsOfSpeech []struct {
			Name string
		}
	}
	WordOrigin string
}

func parseWord(buf []byte) (*Word, error) {
	var w Word
	return &w, json.Unmarshal(buf, &w)
}

[WormGod]

Originally Posted by JSWolf

3. Check Geek1011's Kobo firmware site and if there is a new firmware, download the version for your Kobo and manually install it. To manually install, you unzip the contents of the firmware ZIP to the .kobo directory and when you eject, the Kobo will install the firmware.

I also have a new Clara and am looking to customize it a bit. When I look at Device Information, it shows my software version as 4.16.13268. I assume this is the firmware version, but I do not see it listed as a possible firmware version for the Clara on Geek1011's site.

Am I confusing something?

And should I update to the latest firmware as a general rule?

Thanks!

[DNSB]

Originally Posted by WormGod

I also have a new Clara and am looking to customize it a bit. When I look at Device Information, it shows my software version as 4.16.13268. I assume this is the firmware version, but I do not see it listed as a possible firmware version for the Clara on Geek1011's site.

Am I confusing something?

And should I update to the latest firmware as a general rule?

Thanks!

I would recommend updating to the latest firmware. The only mention of 4.16.13268 I remember seeing, if my memory is correct, was a Japan only version.