Instructions for patching firmware 4.10.11591

[geek1011]

GeoffR, I'm going to remove the 24 line spacing values patch which you sent me, as it seems to be the one causing crashes. Also, the dictionary pop-up size patch has been updated by jackie_w (see the kobopatch thread).

[GeoffR]

Originally Posted by Dirk-71

Do you mean addition like this? See last row! It doesn't work neither!

Yes that is the one. Thanks for testing.

I have another idea to try, but it means big changes to the patch so will take some time to work it out.

[jackie_w]

Originally Posted by GeoffR

I recommend using the new patching method in post #1, but here are the instructions for the old method, for us old dogs who find it hard to learn new tricks.

Originally Posted by JIPG-reader

Thank you GeoffR. You are right that the new method should be better(?), but I find your method simpler and it works perfect, so as we say in Spain, "if something works, don´t touch it!".

At the risk of an over-emotional reaction, my heart sinks to read this. Are we really planning to keep both systems running in parallel ad infinitum?

As an occasional contributor of new & updated patches I had not been planning to have to do everything twice.

I'm not trying to be lazy or difficult but my experience working in IT tells me that trying to maintain parallel equivalent systems usually turns out badly for everyone.

[geek1011]

I have removed the 24 line spacing values patch for fw 4.10.11591 which GeoffR sent me. For those who are fine with the potential crashes on the Adobe epub reader, here it is:

Spoiler:

My 24 line spacing values:
  - Enabled: no
  - PatchGroup: Line spacing values alternatives
  - Description: |
      This patch changes the values on the line spacing adjustment slider,
      increasing the number of spacing options from 15 to 24, and allowing
      much narrower spacing values to be set.
    # save r5, initialise loop counter
  - ReplaceBytes: {Offset: 0x65A034, FindH: BC F5 84 EA 6D A3, ReplaceH: A9 46 00 25 ED 00}
    # loop to load 14 vals from new table
  - ReplaceBytes: {Offset: 0x65A03A, FindH: D3 E9 00 23 07 F1, ReplaceH: 20 46 04 A1 29 44}
  - ReplaceBytes: {Offset: 0x65A040, FindH: 80 01 20 46,       ReplaceH: BC F5 7E EA}
  - ReplaceBytes: {Offset: 0x65A044, FindH: 61 E9 1C 23 BC F5, ReplaceH: 08 35 70 2D F7 D1}
    # jump over new table
  - ReplaceBytes: {Offset: 0x65A04A, FindH: 7A EA 6A A3 D3 E9, ReplaceH: 7C E0 00 BF 00 BF}
    # new table: 14 8-aligned doubles
  - BaseAddress: 0x65A050
  - ReplaceBytes: {Offset: 0x00, FindH: 00 23 07 F1 80 01 20 46, ReplaceH: 00 00 00 00 00 00 F0 3F}
  - ReplaceBytes: {Offset: 0x08, FindH: 61 E9 1A 23 BC F5 70 EA, ReplaceH: 00 00 00 00 00 00 F0 3F}
  - ReplaceBytes: {Offset: 0x10, FindH: 67 A3 D3 E9 00 23 07 F1, ReplaceH: 00 00 00 00 00 00 F0 3F}
  - ReplaceBytes: {Offset: 0x18, FindH: 80 01 20 46 61 E9 18 23, ReplaceH: 00 00 00 00 00 00 F0 3F}
  - ReplaceBytes: {Offset: 0x20, FindH: BC F5 66 EA 43 F2 33 33, ReplaceH: 00 00 00 00 00 00 F0 3F}
  - ReplaceBytes: {Offset: 0x28, FindH: 4F F0 33 32 C3 F6 F3 73, ReplaceH: 00 00 00 00 00 00 F0 3F}
  - ReplaceBytes: {Offset: 0x30, FindH: 07 F1 80 01 20 46 61 E9, ReplaceH: 00 00 00 00 00 00 F0 3F}
  - ReplaceBytes: {Offset: 0x38, FindH: 16 23 BC F5 5A EA 5E A3, ReplaceH: 00 00 00 00 00 00 F0 3F}
  - ReplaceBytes: {Offset: 0x40, FindH: D3 E9 00 23 07 F1 80 01, ReplaceH: 00 00 00 00 00 00 F0 3F}
  - ReplaceBytes: {Offset: 0x48, FindH: 20 46 61 E9 14 23 BC F5, ReplaceH: 00 00 00 00 00 00 F0 3F}
  - ReplaceBytes: {Offset: 0x50, FindH: 50 EA 00 23 00 22 C3 F6, ReplaceH: 00 00 00 00 00 00 F0 3F}
  - ReplaceBytes: {Offset: 0x58, FindH: F8 73 07 F1 80 01 20 46, ReplaceH: 00 00 00 00 00 00 F0 3F}
  - ReplaceBytes: {Offset: 0x60, FindH: 61 E9 12 23 BC F5 44 EA, ReplaceH: 00 00 00 00 00 00 F0 3F}
  - ReplaceBytes: {Offset: 0x68, FindH: 55 A3 D3 E9 00 23 07 F1, ReplaceH: 00 00 00 00 00 00 F0 3F}
  - BaseAddress: 0
    # initialise loop counter
  - ReplaceBytes: {Offset: 0x65A144, FindH: 00 23 00 22, ReplaceH: 00 BF 00 25}
    # loop to load 10 vals from old table
  - ReplaceBytes: {Offset: 0x65A148, FindH: C4 F2 08 03 07 F1, ReplaceH: 20 46 29 A1 29 44}
  - ReplaceBytes: {Offset: 0x65A14E, FindH: 80 01 20 46, ReplaceH: BC F5 F8 E9}
  - ReplaceBytes: {Offset: 0x65A152, FindH: 61 E9 02 23 BC F5, ReplaceH: 08 35 50 2D F7 D1}
    # restore r5
  - ReplaceBytes: {Offset: 0x65A158, FindH: F4 E9, ReplaceH: 4D 46}
    # continue at 65A15A
    #
    # Replacement value: the number of initial line spacing values to exclude from
    # the line spacing slider; minimum 0, maximum 13. E.g. 9 will result in a
    # slider with 15 positions, starting from the 10th replace_float value below.
  - ReplaceInt: {Offset: 0x65A036, Find: 0, Replace: 0}
    # 24 replacement line spacing values: change these replace_float values to suit
    # your device and preferences:
  - BaseAddress: 0x65A050
  - ReplaceFloat: {Offset: 0x00, Find: 1.0, Replace: 0.75}
  - ReplaceFloat: {Offset: 0x08, Find: 1.0, Replace: 0.78}
  - ReplaceFloat: {Offset: 0x10, Find: 1.0, Replace: 0.8}
  - ReplaceFloat: {Offset: 0x18, Find: 1.0, Replace: 0.82}
  - ReplaceFloat: {Offset: 0x20, Find: 1.0, Replace: 0.84}
  - ReplaceFloat: {Offset: 0x28, Find: 1.0, Replace: 0.86}
  - ReplaceFloat: {Offset: 0x30, Find: 1.0, Replace: 0.88}
  - ReplaceFloat: {Offset: 0x38, Find: 1.0, Replace: 0.9}
  - ReplaceFloat: {Offset: 0x40, Find: 1.0, Replace: 0.92}
  - ReplaceFloat: {Offset: 0x48, Find: 1.0, Replace: 0.94}
  - ReplaceFloat: {Offset: 0x50, Find: 1.0, Replace: 0.96}
  - ReplaceFloat: {Offset: 0x58, Find: 1.0, Replace: 0.98}
  - ReplaceFloat: {Offset: 0x60, Find: 1.0, Replace: 1.0}
  - ReplaceFloat: {Offset: 0x68, Find: 1.0, Replace: 1.02}
  - BaseAddress: 0x65A1F0
  - ReplaceFloat: {Offset: 0x00, Find: 1.05, Replace: 1.05}
  - ReplaceFloat: {Offset: 0x08, Find: 1.07, Replace: 1.07}
  - ReplaceFloat: {Offset: 0x10, Find: 1.1,  Replace: 1.1}
  - ReplaceFloat: {Offset: 0x18, Find: 1.35, Replace: 1.15}
  - ReplaceFloat: {Offset: 0x20, Find: 1.7,  Replace: 1.2}
  - ReplaceFloat: {Offset: 0x28, Find: 1.8,  Replace: 1.25}
  - ReplaceFloat: {Offset: 0x30, Find: 2.2,  Replace: 1.3}
  - ReplaceFloat: {Offset: 0x38, Find: 2.4,  Replace: 1.35}
  - ReplaceFloat: {Offset: 0x40, Find: 2.6,  Replace: 1.4}
  - ReplaceFloat: {Offset: 0x48, Find: 2.8,  Replace: 1.5}
    # The first 14 replace_float values above should allow the line spacing to be
    # adjusted in increments of one pixel when using a medium sized font on a high
    # resolution device. If you use a smaller font size and/or a lower resolution
    # device you might find that increasing the slider one step at a time doesn't
    # always result in a line spacing increase at each step. In that case, you
    # might want to use replacement values that are not quite so close together.
    #
    # For example: (I use these values on my 758x1024 Glo)
    #
    #  0.7 0.725 0.75 0.775 0.8 0.825 0.85 0.875 0.9 0.925 0.95 0.975 1.0 1.025 ...
    #
    # or: (these might better suit the 600x800 Touch)
    #
    #  0.6 0.633 0.667 0.7 0.733 0.767 0.8 0.833 0.867 9.0 9.33 9.67 1.0 1.025 ...

Later today, I'm going to investigate the reports that the brightness slider one doesn't work, as I thought it did when I tested it. I think I'll be able to fix whatever the problem is.

[geek1011]

Originally Posted by jackie_w

I'm not trying to be lazy or difficult but my experience working in IT tells me that trying to maintain parallel equivalent systems usually turns out badly for everyone.

Mine is not meant to be equivalent, it is meant to be more (and patch32lsb is not going to try to keep up with it, so it isn't totally parallel). So far (except for this version, but probably not too many others), GeoffR has been updating the patches (either in the new or the old format), and I have been adding them to kobopatch. One is usually the source of truth for the patches, except for the few kobopatch-exclusive ones.

As for the zlib patches, since kobopatch makes it WAY easier to maintain them, I can write a converter to the old format if anyone wants it.

The thing is, I've designed kobopatch so it can be as complex or as simple as you need it. If you only want the basics (like the old version), you can modify the patch files directly and ignore kobopatch.yaml. The new syntax should be pretty simple to pick up. I'm also willing to add anything to make it simpler (if it even can be made simpler). Some of my future plans include: automatic firmware downloading, an automated patch removal tool, optional automated installation, and if I can get it to work (I've been unsucessful so far), a virtual kobo in QEMU.

[geek1011]

Originally Posted by JIPG-reader

Thank you GeoffR. You are right that the new method should be better(?), but I find your method simpler and it works perfect, so as we say in Spain, "if something works, don´t touch it!".

What is more complex about kobopatch? Most people have said it's simpler due to the combined instructions (like FindReplaceString), zlib support, and the overrides in kobopatch.yaml.

[jackie_w]

Originally Posted by geek1011

Mine is not meant to be equivalent, it is meant to be more. So far (except for this version, but probably not too many others), GeoffR has been updating the patches (either in the new or the old format), and I have been adding them to kobopatch. One is usually the source of truth for the patches.

As for the zlib patches, since kobopatch makes it WAY easier to maintain them, I can write a converter to the old format if anyone wants it.

The thing is, I've designed kobopatch so it can be as complex or as simple as you need it. If you only want the basics (like the old version), you can modify the patch files directly and ignore kobopatch.yaml. The new syntax should be pretty simple to pick up. I'm also willing to add anything to make it simpler (if it even can be made simpler). Some of my future plans include: automatic firmware downloading, an automated patch removal tool, optional automated installation, and if I can get it to work (I've been unsucessful so far), a virtual kobo in QEMU.

No need to be so defensive, geek1011, my problem is not with the new system, which I like a lot, not least because of its improvements in patching the nickel file.

My problem is that if people are reluctant to even try to move forward to the new system "because the old works just fine" (and I agree, it does work fine and I've been very grateful for it for several years) then we will forever have 2 systems. I get it that, on the whole, people hate change but I'm sceptical that keeping 2 systems in-sync is workable in the long run (BTW, I'm happy to be proved wrong )

For the record, any future contributions from me will be for kobopatch in .yaml format. If someone else wants to translate them to .patch format and do the testing that's perfectly OK by me.

[GeoffR]

I find the new system more difficult than the old one when it comes to updating the patches, that is very likely just that I have become so used to the old way, but there are a couple of specific things that might help:

Does anyone know of a working yaml-mode for Emacs 24? The one on the Debian repository is a bit glitchy.

Is there a way to run kobopatch on a single yaml file that is equivalent to:

patch32lsb -i libnickel.so.1.0.0 -p libnickel.so.1.0.0.patch -o out

I do this a lot, combined with objdump, to compare before and after and to check for mistakes as I go.

[geek1011]

Originally Posted by GeoffR

I find the new system more difficult than the old one when it comes to updating the patches, that is very likely just that I have become so used to the old way, but there are a couple of specific things that might help:

Does anyone know of a working yaml-mode for Emacs 24? The one on the Debian repository is a bit glitchy.

Is there a way to run kobopatch on a single yaml file that is equivalent to:

patch32lsb -i libnickel.so.1.0.0 -p libnickel.so.1.0.0.patch -o out

I do this a lot, combined with objdump, to compare before and after and to check for mistakes as I go.

As for emacs, I would suggest using VSCode instead for editing kobopatch files, as it's yaml (and Go) support is really great (outline, syntax highlighting, folding, performance, git integration, etc). Also, it's diffs are useful.

As for the last thing you do, I can put together a tool later today. It's pretty simple, as the patching functionality is separated into a Go package.

My workflow is slightly different. I usually first run "./scripts/build.sh;./scripts/test.sh" after making a folder for the new firmware and copying the previous patches (note that a tarball needs to be out in the testdata dir with the binaries for the firmware). This tells me the most obviously broken patches. I then fix the easy ones with objdump (passed through c++filt), and try applying them. If they work, I continue on to the more complex ones. I use the Hopper dissembler for this, as it allows me to easily make modifications, and export executables (which I copy and test over telnet). It also has really good tools for viewing the assembly in different forms (hex, cfg, pseudocode, custom types, etc). Finally, I update the patch files and build a zip. I enable all the patches, and test the functionality of each one. To release the patches, I make a new git tag.

If you have any further questions, I'd be happy to answer them.

Also note that I probably won't have time to update all the patches from scratch in the next firmware version, as I don't usually have so much spare time. I will still be able to port your changes, update the other patches, and make a release, but you'll need to do most of the updating like usual.

[geek1011]

Originally Posted by jackie_w

No need to be so defensive, geek1011, my problem is not with the new system, which I like a lot, not least because of its improvements in patching the nickel file.

It's okay. I'm happy to discuss this topic.

Originally Posted by jackie_w

My problem is that if people are reluctant to even try to move forward to the new system "because the old works just fine" (and I agree, it does work fine and I've been very grateful for it for several years) then we will forever have 2 systems. I get it that, on the whole, people hate change but I'm sceptical that keeping 2 systems in-sync is workable in the long run (BTW, I'm happy to be proved wrong )

I partly agree with people being reluctant, but at the same time, I don't want to force the transition. I have been making it as easy as possible to switch to the kobopatch, and have made quite a few tools to ease the process. At first, I just implemented the old format in kobopatch, and added the zlib instructions. Then, I started releasing patch zips, but still in the old format, but with kobopatch. Finally, in 4.9.11311, I changed the default format to the new one.

Originally Posted by jackie_w

For the record, any future contributions from me will be for kobopatch in .yaml format. If someone else wants to translate them to .patch format and do the testing that's perfectly OK by me.

Do you (or anyone else) want me to make a tool to translate them to .patch format automatically?

[jackie_w]

Originally Posted by geek1011

Do you (or anyone else) want me to make a tool to translate them to .patch format automatically?

No, not on my behalf. Just a personal, but pretty strong, opinion, I think all the effort (including mine) should go in helping people to move forward, not in maintaining the status quo.

Having said that, I'd be happy to compromise on fully supporting a transition period as long as it's length is agreed and, most of all, short & finite (e.g. 4.10.11591 only).

I'm the wrong person to ask about what needs to be done to help with the transition because, speaking only for myself, I had few problems making it. Perhaps those who have had trouble will help by elaborating on their actual problems (like GeoffR already did). My concern is about those who won't even bother trying.

[JIPG-reader]

Originally Posted by geek1011

What is more complex about kobopatch? Most people have said it's simpler due to the combined instructions (like FindReplaceString), zlib support, and the overrides in kobopatch.yaml.

As Jackie_w says, people (in this case me ) is reluctant to change, specially when something is working and you have a known process that have not been changed for a long time. It is just that I am not a programmer and some of these things you mention sound like "Chinese" for me .

Your system seems to be more "elaborated" and with additional options, and probably in the future I will move to it, but today I only use six or seven patches, not the full set, and with GeoffR patcher system, I only change numbers in some txt files following the instructions inside and run it. The system has not changed for a long time, whilst yours is moving very quickly, with some features that are tricky for me (e.g. it has overrides, that depend in a specific number of spaces in the line, different way to visually separate the patches inside the file ...), just little things but with a "higher" probability for me to mesh the fw, and just now I prefer spending more time than .

Anyway, I encourage you to follow the development of kobopatch, as it seems to be a good tool, and probably in the near future I will use it, as I do now with kepubify (that by the way, I prefer over the use of calibre conversion through kobo addons).

[geek1011]

Originally Posted by JIPG-reader

Your system seems to be more "elaborated" and with additional options, and probably in the future I will move to it, but today I only use six or seven patches, not the full set, and with GeoffR patcher system, I only change numbers in some txt files following the instructions inside and run it. The system has not changed for a long time, whilst yours is moving very quickly, with some features that are tricky for me (e.g. it has overrides, that depend in a specific number of spaces in the line, different way to visually separate the patches inside the file ...), just little things but with a "higher" probability for me to mesh the fw, and just now I prefer spending more time than .

kobopatch is only as elaborate as you need it to be. You can still use it the old way (by changing the file directly), and you only need to update when you feel like it. I'm always careful to avoid breaking previously-working patches or features, and even spent a whole 2 months of waiting to make sure. kobopatch isn't anymore likely to cause issues than patch32lsb. Also note that the main reason why mine seems to iterate faster is because I make the changes more visible (to prevent confusion) instead of just changing it without saying anything.

In addition, I strongly recommend you try out the new features, as they really do make things simpler. Here are the only changes I need to make to the patch zips to configure it the way I want.

Also, @GeoffR, I've finished making the tool you requested (it took me ~10 minutes), and released it in kobopatch v0.9.2. It's called kobopatch-apply, and has the same syntax you requested. If you need anymore tools, I'd be happy to help.

[jackie_w]

Originally Posted by JIPG-reader

As Jackie_w says, people (in this case me ) is reluctant to change, specially when something is working and you have a known process that have not been changed for a long time. It is just that I am not a programmer and some of these things you mention sound like "Chinese" for me .

In the spirit of trying to be helpful rather than confrontational...

If/when you're ready to move forward let me offer you an open personal invitation to help you create your new set of .yaml patch files. You can do it via PM or in open forum in a new thread, your choice.

All I would need to start with is:

• a list of which patch names you normally enable
• an indication which of those you manually edit to change the default settings (details can follow later)

[JIPG-reader]

Originally Posted by geek1011

Also note that the main reason why mine seems to iterate faster is because I make the changes more visible (to prevent confusion) instead of just changing it without saying anything.

In addition, I strongly recommend you try out the new features, as they really do make things simpler. Here are the only changes I need to make to the patch zips to configure it the way I want.

OK. I will try with both methods and will compare them until I feel comfortable changing the files.

In theory, should they produce the same final koboroot file or not?