Selective binarization and re-encoding program for use with KoReader

[rantanplan]

Originally Posted by LegeApps

Unless you want to help with the project there is no point to compiling from source. If you did want to help, you would already know how to fix those issues;

You provide 64-bit binaries for Linux which I don’t have, which is why I need to compile from source. It’s that easy.

And how can we trust your app if simply compiling it in rust just fails?

[embryo]

The 3rd try didn't work either, but for different reasons.

Normal processing worked OK, but had some white margins I wanted to crop.
Enabling "Crop margins" did cut the pages but cut part of the text too.
So, I let the margins and tried to make it all BW by disabling the "Layout detection".
That did make it BW, but also let almost half the pages blank.
When I used the "CCITT4" the blank pages were black, with "JBIG2" they were white.

Maybe it's not my lucky day..😁 I'll try again tomorrow.
Thanks for your efforts though..🙏

[LegeApps]

Originally Posted by embryo

The 3rd try didn't work either, but for different reasons.

Normal processing worked OK, but had some white margins I wanted to crop.
Enabling "Crop margins" did cut the pages but cut part of the text too.
So, I let the margins and tried to make it all BW by disabling the "Layout detection".
That did make it BW, but also let almost half the pages blank.
When I used the "CCITT4" the blank pages were black, with "JBIG2" they were white.

Maybe it's not my lucky day..😁 I'll try again tomorrow.
Thanks for your efforts though..🙏

These are the issues I was just working on, when i noticed you had crop margins selected. with layout mode on it was finicky but I hadn't bothered with it. Now gpt 5.5 fixed it up and it works pretty good. new commit is up and new release is here -

https://github.com/LegeApp/Lege/rele...inux-universal

Can try again when youre ready. Thank you for your motivation. Everything you described should be fixed, unless you are running into OOM issues somehow. But the program has been tested on a Windows sandbox with 4gb ram and integrated VRAM and it works fine there.

You provide 64-bit binaries for Linux which I don’t have, which is why I need to compile from source. It’s that easy.

And how can we trust your app if simply compiling it in rust just fails?

OK i didnt know 32bit binaries were needed by anyone. It should be compilable directly from a git clone now. I missed your message about rfd before, that is also a forked repo that wasn't included in the repo and now is.

For anyone reading from MacOS, it is compilable there too, you just need to get the pdfium binary from here

https://github.com/bblanchon/pdfium-binaries

and put it next to the compiled binaries, and grab the other files from the Linux release distribution, then it should work in MacOS too. Let me know if it doesn't. It worked on a macbook in an earlier version just fine.

[rantanplan]

No 32-bit binaries needed, I use macOS but there were no binaries that work on Darwin.

Edit
To make it more clear, the existing 64-bit binaries don’t work on macOS.

[LegeApps]

Originally Posted by rantanplan

No 32-bit binaries needed, I use macOS but there were no binaries that work on Darwin.

Edit
To make it more clear, the existing 64-bit binaries don’t work on macOS.

I wouldnt expect the Linux binaries to work on MacOS...you can recompile for MacOS and it should all work. you need yolo-layout.onnx, sauvola.onnx, eng.tessdata, pdfium.dylib (get from the blanchon github), the docs folder optionally, and it should all work. Just replace the linux executables with the ones you compile. Can even compile in German, which was my tester language for the langauge pack feature after seeing a bunch of activity from there via the Microsoft Store.

Also the Windows release has now been updated.

[Frenzie]

Originally Posted by LegeApps

Don't worry about the code unless you want to help with the project. Otherwise you're an end user, give me end user feedback.

It's just a question about your experiences.

[embryo]

Tried the newer version. Much improved..
Now the log is telling me the page that got a problem and stopped, so I can fix it and continue.
The Layout detection works better, and it works without it too..👍
The cropping has some problems still, but hopefully will get fixed also.

Some bugs I found:

• The window maximize button doesn't work
• When I press the Layout detection button to disable it, it gets disabled but the CCITT4 button is hidden. Resizing the window puts it in the right position
• When the window is at its smaller size, some help tips are cropped

Also, I want to make a wish.
Can you please add a dark skin (or just invert the colors of the app)?
My eyes are hurt when I use the app at night..🙏

[LegeApps]

Originally Posted by embryo

Tried the newer version. Much improved..

The cropping has some problems still, but hopefully will get fixed also.

All the new fixes to cropping were done in that new "free aspect" button that you may not have tried, because most of the problems in cropping are from the program trying to keep aspect ratio, which can't be done 95% of the time. So free aspect gives it free reign to crop, so the results are better.

I'll work on all those fixes over the next day or 2. It's hard to find a GUI framework that works well on both Windows and Linux so I am going to keep trying to work with Freya's limitations around object placement.

Dark mode huh...I can add it as a compile-time feature, I don't want a toggle in the GUI itself.

1.46 will have all these fixes plus SIMD coverage for all encoders for less CPU usage and slightly higher speed.

[LegeApps]

Alrighty now 1.4.6 is out and i built the release for Linux universal.

https://github.com/LegeApp/Lege/rele...inux-universal

other OSes can be built from latest commit.

Changes -
-the requested GUI fixes. the layout detection off output type was off the canvas and now it appears correctly. added some hiding logic so it disappears when irrelevant. grayed out the maximize button. no reason to maximize the GUI. added a night mode. i do not like dark modes as a rule. it changes in 4 gradients from day to late night, making the color palette darker. can auto set in the About page.
-3 of 5 encoders (jp2, jbig2, djvu) have considerable SIMD coverage now.
-exposed jbig2 halftone as an option for dithering, it was in the CLI already but i decided it should be in the GUI also now.
- added AppImage support due to a user request. not really finalized yet. could theoretically package it yourself if you modify cargo.toml and collect the files.

now if you excuse me i want to work on other projects for a while. Enjoy your binarizing

[embryo]

Nice.
Since I'm on Windows I won't be checking this version.
I hope it will get released in the future for Windows too..
Have fun..

[LegeApps]

Originally Posted by embryo

Nice.
Since I'm on Windows I won't be checking this version.
I hope it will get released in the future for Windows too..
Have fun..

Oh I thought you were on Linux..Windows release is now here -

https://github.com/LegeApp/Lege/rele.../Windows-local

it's now 1.4.6, SIMD plus prior GUI changes plus Fable just identified a bunch of bugs and fixed them. also changed default k_factor to .05. .2 was way too high. it could be set manually but no one would think to change it. .05 works much better on all types of pages, light or dark. I could even remove the option entirely and it would be fine as the default. That was an oversight, but it should be good now.

Most recent commit on github is also 1.4.6, so MacOS users can build the newest version.

[embryo]

Great update, thanks!
Some first observations:
- The "Free aspect" checkbox disappeared
- Checking the GPU utilization, I noticed that most of the processing is happening in the CPU with only some spikes of GPU processing.
I have a RTX 3060, but my monitors are using the AMD Radeon graphics of the CPU (AMD Ryzen G).
Do I have to do something to offload the processing there? When I use e.g. ComfyUI, the GPU is full on..
- I used the heavy model (without GPU processing as I said), but when I unchecked it, it was still used for the conversions. When I closed and re-opened the app, things were working fine again.
- Fixed threshold is a great option that was handy for a book whose scanned pages had letters leaking from the flip side. But this means that we are missing some automatic detection of the different pages. Can we have an offset value that will offset the automatic detection's value, after the detection for a darker or lighter threshold?
- The "Late night" skin is a Light skin with a little more pal colors than total white..
I can understand that you don't like dark colors, but since you did most of the work, can you please make the "Late night" skin with white text on a dark/black background? Not the default of course, but an options for us that have eye sight problems..🙏

Overall a great update. Thank you for your time..

[LegeApps]

Originally Posted by embryo

Great update, thanks!
Some first observations:
- The "Free aspect" checkbox disappeared
- Checking the GPU utilization, I noticed that most of the processing is happening in the CPU with only some spikes of GPU processing.
I have a RTX 3060, but my monitors are using the AMD Radeon graphics of the CPU (AMD Ryzen G).
Do I have to do something to offload the processing there? When I use e.g. ComfyUI, the GPU is full on..
- I used the heavy model (without GPU processing as I said), but when I unchecked it, it was still used for the conversions. When I closed and re-opened the app, things were working fine again.
- Fixed threshold is a great option that was handy for a book whose scanned pages had letters leaking from the flip side. But this means that we are missing some automatic detection of the different pages. Can we have an offset value that will offset the automatic detection's value, after the detection for a darker or lighter threshold?
- The "Late night" skin is a Light skin with a little more pal colors than total white..
I can understand that you don't like dark colors, but since you did most of the work, can you please make the "Late night" skin with white text on a dark/black background? Not the default of course, but an options for us that have eye sight problems..🙏

Overall a great update. Thank you for your time..

Yes i consolidated free aspect into the standard crop function since crop without free aspect is effectively useless.

for GPU processing, the WGPU initializer selects the largest GPU by VRAM and other factors. it will use your 3060 and in the CLI you can see that it selected it.

3rd one is a bug, whoops.

fixed threshold issue..i dont understand. there is no detection for a lighter or darker threshold in fixed threshold. it thresholds by the number you pick. there is no current option for interlacing adaptive+fixed in the same document, which seems too granular to me. or what do you mean

for the late night skin, sorry no I dont want to add it to a standard release. I tested a dark mode (it is even still in the code, you can compile with --dark-mode), and it looks bad.

Glad you like the release, ill fix that bug for the next release. Overall I would suggest just using the CLI for processing, it uses the natural dark mode of your terminal and has an interactive mode that is easy to use. the GUI even farms its processing to the CLI under the hood.

[embryo]

Originally Posted by LegeApps

Yes i consolidated free aspect into the standard crop function since crop without free aspect is effectively useless.

True, nice..

for GPU processing, the WGPU initializer selects the largest GPU by VRAM and other factors. it will use your 3060 and in the CLI you can see that it selected it.

Hmm. So, that means that it doesn't utilize the GPU that much after all.
I did see some spikes in the GPU processing, but overall the processing was done at the CPU. Pity..

fixed threshold issue..i dont understand. there is no detection for a lighter or darker threshold in fixed threshold. it thresholds by the number you pick. there is no current option for interlacing adaptive+fixed in the same document, which seems too granular to me. or what do you mean

Let's say that auto detection (adaptive), gives for the 1st page threshold 160, for the 2nd 170 and for the 3rd 130.
If I use an offset value of -20, the 1st will be processed with 140, the second with 150 and the 3rd with 110..

for the late night skin, sorry no I dont want to add it to a standard release. I tested a dark mode (it is even still in the code, you can compile with --dark-mode), and it looks bad.

Of course it looks bad to you, that you don't like dark mode..😁
But it might look good for people like me who do like it.
But then again it might not. It might look horrible.
The point is that it is an option (and not the default), so whoever doesn't like it has the already existing 4 light styles to choose..

Glad you like the release, ill fix that bug for the next release. Overall I would suggest just using the CLI for processing, it uses the natural dark mode of your terminal and has an interactive mode that is easy to use. the GUI even farms its processing to the CLI under the hood.

I don't like the CLI that much. I use it only if I have no other option.
But as they say "There is no accounting for taste", or whatever..😉

Anyway, apart of all this nitpicking, overall it is a useful tool, so thanks for your work.. 🙏